Serving a robots.txt file
New feature•
We're happy to announce it's now possible to serve robots.txt files on your domains!
This can help block (bad) bots and get's you back in control how you want your domains to be indexed.
Serveral options are available:
No robots.txt (default)
Explicitly allow all
Block bad bots
Disallow all
Custom (Dedicated IP only)
Read more here.