pandas PSA: s.isnull() on series of 100k int64s: 235 µs, s.isnull() on series of 100k 'object's: 5.91 seconds. that's like ... six orders of magnitude slower. /cc @BagelDaughterpic.twitter.com/UftrfvSYSD
-
Oh shit this explains a lot about the code I saw in there. What's weird is that converting to a categorical FIRST and then doing the null check is actually still 3 orders of magnitude faster than a straight isnull() in this casepic.twitter.com/zgIOeInG11
-
-
OK so now it makes sense - I think the bug is that the categorical code was trying to use a uint8 numpy array to index a pandas dataframe which was causing the weird typeerror. Doing a series.mask(numpyarray.astype('bool')) seems to workpic.twitter.com/XbCyaNOGZH
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.