(36) Please observe the maximum crawl rate of 90 seconds per page, as indicated in /robots.txt