Robots & Directives Handling
Automatically follows robots.txt, meta robots, and x-robots headers, with options to ignore or override them for testing.
Please authenticate to join the conversation.
Upvoters
Status
Completed
Board
💡
Feature Request
Date
5 months ago
Author
Signup.Only
Subscribe to post
Get notified by email when there are changes.
Upvoters
Status
Completed
Board
💡
Feature Request
Date
5 months ago
Author
Signup.Only
Subscribe to post
Get notified by email when there are changes.