devlin.ai source reader
When an instructional designer adds a link as source material in devlin.ai, our reader fetches that one page or file so the designer can build a training simulation from it. It identifies itself as devlin-source-reader.
What it does
- It fetches only the link a person gave it, plus any redirects (up to 5 hops). It never crawls other pages.
- It does not sign in, run scripts, or get around bot protection.
- It reads public pages and documents only; it never connects to a private or internal address.
How to opt out
Add a rule for our reader to your robots.txt:
User-agent: devlin-source-reader Disallow: /
Because each fetch is started by a person for a single page, rules that apply to all crawlers (User-agent: *) are not applied. Rules that name devlin-source-reader always are.