Today I learned: Python’s concurrency-safe queue structures have a hard limit of 32,768 items. I totally did not deadlock a piece of code by not paying attention to that, why do you ask?
Prikaži ovu nit
And to stave off some questions: I wasn’t paying attention to it because the documentation states that if “maxsize is less than or equal to zero, the queue size is infinite”. I did not in fact assume “infinite”. But did assume > 32,768. Which turned out to be naïve.
-
-
Debugging it was a fun story, too. At first I thought it was something else stalling out (since all the failures happened at suspicious powers of 2). Eventually isolated just the queue setup and saw it start to block put() at 32,768 items.
Prikaži ovu nit -
So then I tried creating a queue with a higher maxsize, and got an immediate exception saying that’s not allowed. Turns out to be a hard-coded limit. Perhaps this should be documented better.
Prikaži ovu nit -
(it’s not entirely Python’s fault, since this is actually handed off to system-specific non-Python semaphore implementations, and that’s where the hard-coded limit actually lives, but still, could be documented better)
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.