Could use some help from Rust Twitter on this one: I want to provide each thread with a Vec in a thread-local. Then from another thread I want to periodically take ownership of the thread-local vecs and swap them with empty ones. Is there a good pattern/lib for this?
-
Show this thread
-
I'm thinking I could make this work with an Arc<Mutex<Vec>> stored in the thread-local, and sending a clone over the the driver thread. But it feels like this should be doable using a compare-and-swap of sorts, but unsure how to make this work correctly. This should be common?
6 replies 0 retweets 0 likesShow this thread -
Replying to @yoshuawuyts
Does it have to be in TLS? Which side controls the swapping, worker threads or the collecting thread?
1 reply 0 retweets 1 like -
Replying to @jakubvaltar
Good questions! We need to instantiate one instance per thread, so thread-local seemed like the best match. Collection thread does the swapping.
1 reply 0 retweets 1 like -
Replying to @yoshuawuyts
First idea: each worker thread gets a pair of a channel sender and a receiver. As soon as an empty Vec arrives on the receiver, sends the full Vec through the sender. Collector receives full Vecs on its receiver and sends empty Vecs to threads when it's time to swap.
1 reply 0 retweets 2 likes -
-
Replying to @yoshuawuyts @jakubvaltar
hmmm, actually this runs into one tricky part tho: if for some reason no new messages are produced for a while then the channel can't be checked -- which means we never receive the new vec, and never push the filled vec. Hmm....
1 reply 0 retweets 1 like -
Replying to @yoshuawuyts
Ah ok. I assume you can't use a channel instead of a Vec. Then you need to have some lock, either mutex or an atomic bool flag, because two threads can write to the same memory.
1 reply 0 retweets 1 like -
Replying to @jakubvaltar @yoshuawuyts
Though I would strongly consider having a crossbeam channel for each thread and sending every element to the collector, which can then use select.
1 reply 0 retweets 1 like
heh, the problem with that is that we'd need to do lots of allocations -- which is something I'm purposefully trying *not* to do. Ideally I could have two bump allocators per thread, with the driver thread switching which one's being written to and read from.
-
-
Replying to @yoshuawuyts
Then I'd go with Arc<Mutex<Vec<T>>> with a good Mutex. The one from parking_lot should be fast, it's going to just compare and swap when not contended (which it won't be if you swap once every 250 ms).
0 replies 0 retweets 1 likeThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.