Example for thread-local variables

[spease]

I wanted to use Rayon to perform operations on files being decompressed using the zip-rs crate. The problem is that the zip-rs files require a mutable reference to the archive. Reading an issue, it sounds like what's going on is that there's some pointer to the current file position that limits things.

The alternative I came up with was to create a ziparchive object per thread (since all the operations are read-only, multiple file handles to the zip file should be fine). The problem I ran into is that there doesn't seem to be an obvious way to have a per-thread variable with the par_iter approach. The other approach described in the documentation seems to only hold for two sets of data.

Is there a way to do this elegantly with Rayon?

[cuviper]

The broadcast API of #492 would allow you to set thread-local variables. You'd still have to use something like RefCell to get mutable access though, just like any TLS.

Another option, if you don't need to directly return output, is to use ParallelIterator::fold with the accumulator object being the zip archive handle. If you do need to return output, then perhaps you can use map_with and your own custom wrapper that opens a new zip archive handle each time it's cloned. Either of these methods will actually create more objects than just the number of threads, since they'll actually be per job split, but that will be roughly the same order of magnitude. You can use with_min_len to limit how small it splits up.

I don't have ready examples for any of this, but I can try to whip something up if it's not clear.

[spease]

Eventually I discovered thread_local! and this seemed to give me the performance I expected.

https://www.reddit.com/r/rust/comments/7kk49h/best_practices_for_using_combinators/

An example someplace discoverable would be nice, I had to search around before I came up with something that led me to try this solution.

Regardless, was mostly filing this (and other things) more for the sake of calling attention to an issue to either leave a comment trail or improve documentation to make it easier for other people who come later (or me when I forget I suppose...)