"tl;dr: Multiprocessing in Python is crippled by pickle's poor function serialization." @mrocklin http://matthewrocklin.com/blog/work/2013/12/05/Parallelism-and-Serialization/ …
-
-
@avibryant Can't get it to stick in my brain, is problem capturing too much that you don't need? Or that you actually need too much? -
@peterseibel mostly the former. But also having good ways to need less (lazy init helps a lot). -
@avibryant Closing over in scope vars that aren't actually used? Or over things that are used but which also lead to big things that aren't? -
@peterseibel in Scala, eg, it's hard to avoid closing over "this" which can then pull in tons of stuff you don't need. -
@peterseibel but scala at least has a hard line you don't cross at this.class - in some languages, hard not to pull in the entire program.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.