Best practice for saving a value that applies to whole dataframe? A column with the same value repeating — a separate file — something else?
-
-
-
Replying to @Research_Tim
Sorry to be more specific saving as geoJSON from within geopandas but willing to change to other compatible filetype. But has to be GEOpd.
2 replies 0 retweets 0 likes -
Replying to @o_guest @Research_Tim
why not save normally with to_csv(…, compression=‘gzip’)? standard compression algs should be able to handle this just fine
1 reply 0 retweets 1 like -
if you want to be more systematic about it, used composition and create your own class that stores the DF internally along w/ metadata
1 reply 0 retweets 1 like -
-
Replying to @o_guest @Research_Tim
I took your concern to be about storage on disk. I would just add a constant column to the DF, and save it with compression.
1 reply 0 retweets 0 likes -
but if you want something more robust, write your own class that specifies how to load/save the DF (appending/removing the extra columns)
1 reply 0 retweets 0 likes -
Replying to @talyarkoni @Research_Tim
Yes, that is one of the things I considered and currently my code does that but was going to switch to separate file. No concern for disk
1 reply 0 retweets 0 likes
space at all I think I have upwards of 5TB at disposal locally and more on server probably. Not an issue though at all neither is RAM.
-
-
So you think adding a column is best given I want plain text?
1 reply 0 retweets 0 likes - 2 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.