why is binary size still large after disabling unicode?

[d3lm]

I have noticed in an application I am working on which gets compiled to WebAssembly that the binary size increases by quite some amount (150 KB). I find this a lot and was wondering if this is normal in terms of the regex engine without unicode support.

So basically I have disabled all default features and have re-enabled std and perf.

[BurntSushi]

See: #583

See: #613

I don't know whether it's "normal." All regex engines that I think of as "fast" have a boatload of code in them. A slower or less featureful regex engine could likely be implemented in much less code.

So basically I have disabled all default features and have re-enabled std and perf.

Well, perf adds to the binary size and the number of dependencies. If you care most about binary size, then disable perf.

If you don't need Unicode and your regexes are known statically, then you could also use regex-automata to embed a DFA into your binary.

[d3lm]

Thanks for your reply! Unfortunately my regexes are not known statically but I could indeed try to disable perf and see how much overhead it adds to my program.

[RReverser]

I imagine, alternatively, if you're targeting Web environment with your Wasm, then you could conditionally compile to js_sys::RegExp instead. It will allow you to significantly reduce Wasm size by utilising native RegExp engine built into the host environment, but watch out for syntactic differences. For simple regexes it might be worth it.

UPD and, of course, also watch out for speed overhead if you're going to convert large strings from Rust to pass to the regex when using this technique. However, if your data comes as an input from JS, then conversion can be avoided altogether.

[BurntSushi]

Why can't you use a thread local for your case? The regex crate itself allocates memory proportional to the number of simultaneous searches per regex.

I would also generally expect js naive regexes to be quite fast.

[d3lm]

It's a little more complicated than that. Basically, the same WASM module and memory gets instantiated in multiple workers and the module is the single source of truth for some data that is maintained by Rust (compiled to WASM). JS can then subscribe to changes and provide a path, e.g. /foo/bar/*.js. My idea was to convert this into a regex first but because I can't really store this in a static I implemented very minimal and not complete globbing myself. Any existing crate was also adding too much to the binary size and had way more features that I don't really need. The reason I am storing this in a static is that, if that particular file is changed in some other thread then the change has to be propagated to the threads that are interested in this change. Let's assume there are 3 workers and each use the same WASM module + memory: A, B and C.

The JS side of A wants to listen for changes in /foo/bar/*.js. C now changes a JS file inside /foo/bar. What I am doing currently is to use a mpsc but I first look at all the static paths to find the sender of the threads that need to be notified (there's a listener). But if there is no listener then I don't even want to send the event from C to A over the channel. That's the reason why I need to store the, ideally regex, in a static so every thread can do the work of finding out if the event has to be propagated.

Correct me if I am wrong here. Curious to hear your thoughts but that's basically why I want it to be Send + Sync.

[RReverser]

You can prepare the regex as a string and store it in a static instead - Strings can be shared between threads easily as they live in the shared memory. Then put actual RegExp construction in a thread-local, so in each thread it will take the source from the static and construct the actual object.

[d3lm]

That means each thread has to watch for changes in that static list of strings and re-construct the regexes. Not sure if that's ideal. Or you'd have to notify each thread to update the regexes.

[RReverser]

Or you'd have to notify each thread to update the regexes.

Yeah, that. It shouldn't be a big problem though, constructing regex is cheap (done lazily by JS engines and the actual regex is usually cached behind the scenes).