@michael_nielsen In your post How to Crawl a 1/4 Billion pages, you actually ran the crawler on EC2, correct? There were no TOS issues?
@maesto The TOS forbids crawlers that impair or disrupt other sites. I worked pretty hard to avoid doing that.
-
-
@michael_nielsen Got it. Thanks for both the swift reply and the outstanding blog post.Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.