How to get the performance of re2 in regex

[freekeeper]

I want to compare the performance of regex with re2,
how to get the result in https://github.com/rust-lang/regex/blob/master/bench/log/05/re2-vs-rust

[BurntSushi]

$ cd bench
$ ./run rust sherlock::name_alt | tee rust.log
$ ./run re2 sherlock::name_alt | tee re2.log
$ cargo benchcmp re2.log rust.log                                                              
 name                        re2.log ns/iter       rust.log ns/iter     diff ns/iter   diff %  speedup 
 sherlock::name_alt1         46,296 (12850 MB/s)   35,976 (16536 MB/s)       -10,320  -22.29%   x 1.29 
 sherlock::name_alt2         970,626 (612 MB/s)    85,602 (6949 MB/s)       -885,024  -91.18%  x 11.34 
 sherlock::name_alt3         996,394 (597 MB/s)    96,751 (6149 MB/s)       -899,643  -90.29%  x 10.30 
 sherlock::name_alt3_nocase  1,001,320 (594 MB/s)  377,102 (1577 MB/s)      -624,218  -62.34%   x 2.66 
 sherlock::name_alt4         972,360 (611 MB/s)    121,969 (4877 MB/s)      -850,391  -87.46%   x 7.97 
 sherlock::name_alt4_nocase  988,821 (601 MB/s)    184,097 (3231 MB/s)      -804,724  -81.38%   x 5.37 
 sherlock::name_alt5         982,030 (605 MB/s)    90,062 (6605 MB/s)       -891,968  -90.83%  x 10.90 
 sherlock::name_alt5_nocase  981,768 (605 MB/s)    349,229 (1703 MB/s)      -632,539  -64.43%   x 2.81

If you remove the sherlock::name_alt bit, then all benchmarks will be run.

You can get cargo benchcmp by running cargo install cargo-benchcmp. Its repo is: https://github.com/BurntSushi/cargo-benchcmp.

---

Putting that aside, I would strongly encourage you to ask more questions. Particularly if you're trying to understand a result. I would also strongly encourage you to benchmark your actual use case, instead of relying on only the benchmarks I've put together. I can also help try to explain differences in perf since I'm pretty familiar with RE2.

[BurntSushi]

Ummm... I don't know? You need to give me a full repro. You haven't provided enough information for me to run what you're running. And I wouldn't necessarily expect them to be the same anyway. Your first regex is anchored. So it will match different things.

[freekeeper]

The added code is line 163 to 210 in this link https://github.com/freekeeper/regex/blob/master/bench/src/misc.rs

[BurntSushi]

The answer is right here, in the docs for the macro: https://github.com/freekeeper/regex/blob/32bc38fa56bf8dc41bf23e8c46d50b69e3971805/bench/src/bench.rs#L124-L126

[freekeeper]

I did the following bench in regex，The complete code can be found at this link

macro_rules! dot_star_capture {
    () => {
        r"((.*)()()($))"
    };
}
bench_match!(is_match_dot_star_capture, dot_star_capture!(), get_text(TXT_1K, empty_suffix()));

The bench result is as follows:

test misc::is_match_dot_star_capture           ... bench:          24 ns/iter (+/- 19) = 42708 MB/s

And I want to bench regex's interface to C's rure_is_match https://github.com/rust-lang/regex/blob/master/regex-capi/include/rure.h#L185 function with the same test text and regex, the core code is as follows:

void Rure_is_Match_RE2(benchmark::State& state, const char *regexp)
{
  std::ifstream in("re2/testing/text_re2_1KB.txt");
  std::stringstream buffer;
  buffer << in.rdbuf();
  std::string s = buffer.str().substr(0, state.range(0));
  rure_error *err = rure_error_new();
  rure *re1 = rure_compile((const uint8_t *)regexp, strlen(regexp), RURE_DEFAULT_FLAGS, NULL, err);
  for (auto _ : state) {
    bool matched = rure_is_match((rure *)re1, (const uint8_t *)s.c_str(), strlen(s.c_str()), 0);
    CHECK(matched);
  }
  state.SetBytesProcessed(state.iterations() * state.range(0));
}

void Rure_is_Match_RE2_DotStarCapture_text_re2_1KB(benchmark::State& state)  { Rure_is_Match_RE2(state, "(?s)((.*)()()($))"); }

BENCHMARK_RANGE(Rure_is_Match_RE2_DotStarCapture_text_re2_1KB, 2<<6, 2<<9);

The bench result is as follows:

Rure_is_Match_RE2_DotStarCapture_text_re2_1KB/128        5000000               385 ns/op         332.07 MB/s
Rure_is_Match_RE2_DotStarCapture_text_re2_1KB/256        5000000               382 ns/op         669.85 MB/s
Rure_is_Match_RE2_DotStarCapture_text_re2_1KB/512        5000000               378 ns/op        1354.12 MB/s
Rure_is_Match_RE2_DotStarCapture_text_re2_1KB/1K         5000000               385 ns/op        2659.34 MB/s

I use the same test text and the same regular expression, why is the performance so bad in rure_is_match

[BurntSushi]

Dunno. I mean, you are running strlen inside the benchmark, but I wouldn't expect that to fully explain the difference here.

The best I can say is that something about your measurement process is flawed. But you aren't really providing a way for me to easily reproduce your tests. For example, I'd expect, "clone this repo and run this script" and it should do exactly what you're trying. Otherwise, I'm not digging through RE2's benchmark harness to try and understand it for you. I've already explained what I could.