Meta’s AI crawler sent more than 550,000 requests over two days to the website of Chelsea Paulus, a children’s book author and illustrator, using up the CPU allowance on the SiteGround plan that runs her four sites, according to Business Insider. Robots.txt and firewall changes would have taken up to a day to work, so Paulus stopped the crawler, Meta External Agent, with code from SiteGround, but a second Meta bot, Meta Web Indexer, then pushed her usage to 100%. Paulus said she can’t afford a bigger plan for bots that never buy anything, and she pulled her site link from Instagram so human visitors wouldn’t knock it offline. Meta said site owners can control its crawlers through robots.txt and that it respects their settings.
