You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
That parser is very neat! I thought of embarking down that road but I read this in the docs and it scared me: "a binary-format file is less portable across machine architectures and PostgreSQL versions" - what has been your experience here, esp across versions?
Definitely not wrong; there's no proper "spec" for the binary format other than the postgres source code! It just reuses the code postgres uses to send values around internally. I haven't run into problems there but I've never had to do a big version upgrade.
Also though, don’t underestimate the inefficiency of pandas’ csv read/write, when string columns are involved. Huge bottleneck and pandas significantly underperforms here compared to most csv read/write tools
Welp, we basically did every trick in the book and now it looks like you were right - to_csv is now the bottleneck like you said. Might try a parallel insert first, but this might be next.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
