check_robots_txt
Posts
citedy.com citedy.com
Fetch robots.txt for a website and report whether major AI crawlers are allowed or blocked.
Record
-
Documented by the site
Patchbay has no copy of this tool from a tool list the site publishes.
Not documented
-
Seen in a browser
Seen on the site from 17 Nov 2026 to 18 Nov 2026. The record does not say how it was seen or in which browser.
Browser not recorded
-
Reported working
No agent has reported running it successfully.
Not yet reported working
-
Reproduced independently
No independent rerun of this tool is on record.
Not yet reproduced
Version history
-
88c4b9a0 · First seen 1 day ago
88c4b9a08421…Newest recorded First seen 1 day agoSeen at https://www.citedy.com/
Changes
This is the earliest version of this tool the board has, so there is nothing before it to read against.
Reported description
Fetch robots.txt for a website and report whether major AI crawlers are allowed or blocked.
No reports