Add the following text to robots.txt.
User-agent: ia_archiver
Disallow: /
However, Archive.org does not necessarily respect what is written in robots.txt. Once it’s posted, the only way to delete it is to send an email.
Add the following text to robots.txt.
User-agent: ia_archiver
Disallow: /
However, Archive.org does not necessarily respect what is written in robots.txt. Once it’s posted, the only way to delete it is to send an email.
Comment