first phase of migrating to regex-automata

.github/workflows/ci.yml

@@ -52,7 +52,7 @@ jobs:
         include:
         - build: pinned
           os: ubuntu-latest
-          rust: 1.41.1
+          rust: 1.60.0
         - build: stable
           os: ubuntu-latest
           rust: stable
@@ -159,11 +159,6 @@ jobs:
         cd regex-capi
         ./test
 
-    - if: matrix.build == 'nightly'
-      name: Compile regex-debug
-      run: |
-        ${{ env.CARGO }} build --verbose --manifest-path regex-debug/Cargo.toml $TARGET
-
     - if: matrix.build == 'nightly'
       name: Run benchmarks as tests
       run: |

CHANGELOG.md

@@ -1,3 +1,120 @@
+1.8.0 (TBD)
+===========
+This is a sizeable release that will be soon followed by another sizeable
+release. Both of them will combined close over 40 existing issues and PRs.
+
+This first release, despite its size, essentially represent preparatory work
+for the second release, which will be even bigger. Namely, this release:
+
+* Increases the MSRV to Rust 1.60.0, which was released about 1 year ago.
+* Upgrades its dependency on `aho-corasick` to the recently release 1.0
+version.
+* Upgrades its dependency on `regex-syntax` to the simultaneously released
+`0.7` version. The changes to `regex-syntax` principally revolve around a
+rewrite of its literal extraction code and a number of simplifications and
+optimizations to its high-level intermediate representation (HIR).
+
+The second release, which will follow ~shortly after the release above, will
+contain a soup-to-nuts rewrite of every regex engine. This will be done by
+bringing [`regex-automata`](https://github.com/BurntSushi/regex-automata) into
+this repository, and then changing the `regex` crate to be nothing but an API
+shim layer on top of `regex-automata`'s API.
+
+These tandem releases are the culmination of about 3
+years of on-and-off work that [began in earnest in March
+2020](https://github.com/rust-lang/regex/issues/656).
+
+Because of the scale of changes involved in these releases, I would love to
+hear about your experience. Especially if you notice undocumented changes in
+behavior or performance changes (positive *or* negative).
+
+Most changes in the first release are listed below. For more details, please
+see the commit log, which reflects a linear and decently documented history
+of all changes.
+
+New features:
+
+* [FEATURE #501](https://github.com/rust-lang/regex/issues/501):
+Permit many more characters to be escaped, even if they have no significance.
+More specifically, any character except for `[0-9A-Za-z<>]` can now be
+escaped. Also, a new routine, `is_escapeable_character`, has been added to
+`regex-syntax` to query whether a character is escapeable or not.
+* [FEATURE #547](https://github.com/rust-lang/regex/issues/547):
+Add `Regex::captures_at`. This filles a hole in the API, but doesn't otherwise
+introduce any new expressive power.
+* [FEATURE #595](https://github.com/rust-lang/regex/issues/595):
+Capture group names are now Unicode-aware. They can now begin with either a `_`
+or any "alphabetic" codepoint. After the first codepoint, subsequent codepoints
+can be any sequence of alpha-numeric codepoints, along with `_`, `.`, `[` and
+`]`. Note that replacement syntax has not changed.
+* [FEATURE #810](https://github.com/rust-lang/regex/issues/810):
+Add `Match::is_empty` and `Match::len` APIs.
+* [FEATURE #905](https://github.com/rust-lang/regex/issues/905):
+Add an `impl Default for RegexSet`, with the default being the empty set.
+* [FEATURE #908](https://github.com/rust-lang/regex/issues/908):
+A new method, `Regex::static_captures_len`, has been added which returns the
+number of capture groups in the pattern if and only if every possible match
+always contains the same number of matching groups.
+* [FEATURE #955](https://github.com/rust-lang/regex/issues/955):
+Named captures can now be written as `(?<name>re)` in addition to
+`(?P<name>re)`.
+* FEATURE: `regex-syntax` now supports empty character classes.
+* FEATURE: `regex-syntax` now has an optional `std` feature. (This will come
+to `regex` in the second release.)
+* FEATURE: The `Hir` type in `regex-syntax` has had a number of simplifications
+made to it.
+* FEATURE: `regex-syntax` has support for a new `R` flag for enabling CRLF
+mode. This will be supported in `regex` proper in the second release.
+* FEATURE: `regex-syntax` now has proper support for "regex that never
+matches" via `Hir::fail()`.
+* FEATURE: The `hir::literal` module of `regex-syntax` has been completely
+re-worked. It now has more documentation, examples and advice.
+* FEATURE: The `allow_invalid_utf8` option in `regex-syntax` has been renamed
+to `utf8`, and the meaning of the boolean has been flipped.
+
+Performance improvements:
+
+Bug fixes:
+
+* [BUG #514](https://github.com/rust-lang/regex/issues/514):
+Improve `Debug` impl for `Match` so that it doesn't show the entire haystack.
+* BUGS [#516](https://github.com/rust-lang/regex/issues/516),
+[#731](https://github.com/rust-lang/regex/issues/731):
+Fix a number of issues with printing `Hir` values as regex patterns.
+* [BUG #610](https://github.com/rust-lang/regex/issues/610):
+Add explicit example of `foo|bar` in the regex syntax docs.
+* [BUG #625](https://github.com/rust-lang/regex/issues/625):
+Clarify that `SetMatches::len` does not (regretably) refer to the number of
+matches in the set.
+* [BUG #660](https://github.com/rust-lang/regex/issues/660):
+Clarify "verbose mode" in regex syntax documentation.
+* BUG [#738](https://github.com/rust-lang/regex/issues/738),
+[#950](https://github.com/rust-lang/regex/issues/950):
+Fix `CaptureLocations::get` so that it never panics.
+* [BUG #747](https://github.com/rust-lang/regex/issues/747):
+Clarify documentation for `Regex::shortest_match`.
+* [BUG #835](https://github.com/rust-lang/regex/issues/835):
+Fix `\p{Sc}` so that it is equivalent to `\p{Currency_Symbol}`.
+* [BUG #846](https://github.com/rust-lang/regex/issues/846):
+Add more clarifying documentation to the `CompiledTooBig` error variant.
+* [BUG #854](https://github.com/rust-lang/regex/issues/854):
+Clarify that `regex::Regex` searches as if the haystack is a sequence of
+Unicode scalar values.
+* [BUG #884](https://github.com/rust-lang/regex/issues/884):
+Replace `__Nonexhaustive` variants with `#[non_exhaustive]` attribute.
+* [BUG #893](https://github.com/rust-lang/regex/pull/893):
+Optimize case folding since it can get quite slow in some pathological cases.
+* [BUG #895](https://github.com/rust-lang/regex/issues/895):
+Reject `(?-u:\W)` in `regex::Regex` APIs.
+* [BUG #942](https://github.com/rust-lang/regex/issues/942):
+Add a missing `void` keyword to indicate "no parameters" in C API.
+* [BUG #965](https://github.com/rust-lang/regex/issues/965):
+Fix `\p{Lc}` so that it is equivalent to `\p{Cased_Letter}`.
+* [BUG #975](https://github.com/rust-lang/regex/issues/975):
+Clarify documentation for `\pX` syntax.
+
+
+
 1.7.3 (2023-03-24)
 ==================
 This is a small release that fixes a bug in `Regex::shortest_match_at` that
@@ -743,7 +860,7 @@ Bug gixes:
 ==================
 This release includes a ground-up rewrite of the regex-syntax crate, which has
 been in development for over a year.
-
+731
 New features:
 
 * Error messages for invalid regexes have been greatly improved. You get these

Cargo.toml

@@ -14,11 +14,12 @@ finite automata and guarantees linear time matching on all inputs.
 categories = ["text-processing"]
 autotests = false
 exclude = ["/scripts/*", "/.github/*"]
-edition = "2018"
+edition = "2021"
+rust-version = "1.60.0"
 
 [workspace]
 members = [
-  "bench", "regex-capi", "regex-debug", "regex-syntax",
+  "bench", "regex-capi", "regex-syntax",
 ]
 
 [lib]
@@ -106,7 +107,7 @@ pattern = []
 
 # For very fast prefix literal matching.
 [dependencies.aho-corasick]
-version = "0.7.18"
+version = "1.0.0"
 optional = true
 
 # For skipping along search text quickly when a leading byte is known.

bench/log/10-last-frontier/rust-after-literal.log

@@ -0,0 +1,124 @@
+
+running 119 tests
+test misc::anchored_literal_long_match       ... bench:          18 ns/iter (+/- 0) = 21666 MB/s
+test misc::anchored_literal_long_non_match   ... bench:          20 ns/iter (+/- 0) = 19500 MB/s
+test misc::anchored_literal_short_match      ... bench:          18 ns/iter (+/- 0) = 1444 MB/s
+test misc::anchored_literal_short_non_match  ... bench:          20 ns/iter (+/- 0) = 1300 MB/s
+test misc::easy0_1K                          ... bench:          51 ns/iter (+/- 2) = 20607 MB/s
+test misc::easy0_1MB                         ... bench:          56 ns/iter (+/- 1) = 18725053 MB/s
+test misc::easy0_32                          ... bench:          51 ns/iter (+/- 0) = 1156 MB/s
+test misc::easy0_32K                         ... bench:          53 ns/iter (+/- 1) = 618773 MB/s
+test misc::easy1_1K                          ... bench:          41 ns/iter (+/- 0) = 25463 MB/s
+test misc::easy1_1MB                         ... bench:          44 ns/iter (+/- 1) = 23831727 MB/s
+test misc::easy1_32                          ... bench:          40 ns/iter (+/- 1) = 1300 MB/s
+test misc::easy1_32K                         ... bench:          40 ns/iter (+/- 1) = 819700 MB/s
+test misc::hard_1K                           ... bench:          51 ns/iter (+/- 2) = 20607 MB/s
+test misc::hard_1MB                          ... bench:          56 ns/iter (+/- 1) = 18725053 MB/s
+test misc::hard_32                           ... bench:          51 ns/iter (+/- 2) = 1156 MB/s
+test misc::hard_32K                          ... bench:          51 ns/iter (+/- 1) = 643039 MB/s
+test misc::is_match_set                      ... bench:          61 ns/iter (+/- 2) = 409 MB/s
+test misc::literal                           ... bench:          13 ns/iter (+/- 0) = 3923 MB/s
+test misc::long_needle1                      ... bench:       3,242 ns/iter (+/- 79) = 30845 MB/s
+test misc::long_needle2                      ... bench:     350,572 ns/iter (+/- 6,860) = 285 MB/s
+test misc::match_class                       ... bench:          62 ns/iter (+/- 6) = 1306 MB/s
+test misc::match_class_in_range              ... bench:          14 ns/iter (+/- 0) = 5785 MB/s
+test misc::match_class_unicode               ... bench:         259 ns/iter (+/- 15) = 621 MB/s
+test misc::matches_set                       ... bench:         462 ns/iter (+/- 9) = 54 MB/s
+test misc::medium_1K                         ... bench:          53 ns/iter (+/- 0) = 19849 MB/s
+test misc::medium_1MB                        ... bench:          58 ns/iter (+/- 1) = 18079379 MB/s
+test misc::medium_32                         ... bench:          53 ns/iter (+/- 1) = 1132 MB/s
+test misc::medium_32K                        ... bench:          53 ns/iter (+/- 1) = 618792 MB/s
+test misc::no_exponential                    ... bench:         423 ns/iter (+/- 13) = 236 MB/s
+test misc::not_literal                       ... bench:          89 ns/iter (+/- 0) = 573 MB/s
+test misc::one_pass_long_prefix              ... bench:          52 ns/iter (+/- 0) = 500 MB/s
+test misc::one_pass_long_prefix_not          ... bench:          52 ns/iter (+/- 1) = 500 MB/s
+test misc::one_pass_short                    ... bench:          38 ns/iter (+/- 1) = 447 MB/s
+test misc::one_pass_short_not                ... bench:          41 ns/iter (+/- 1) = 414 MB/s
+test misc::reallyhard2_1K                    ... bench:          81 ns/iter (+/- 1) = 12839 MB/s
+test misc::reallyhard_1K                     ... bench:       1,592 ns/iter (+/- 1) = 660 MB/s
+test misc::reallyhard_1MB                    ... bench:   1,575,822 ns/iter (+/- 39,203) = 665 MB/s
+test misc::reallyhard_32                     ... bench:         102 ns/iter (+/- 0) = 578 MB/s
+test misc::reallyhard_32K                    ... bench:      49,328 ns/iter (+/- 2,598) = 664 MB/s
+test misc::replace_all                       ... bench:         132 ns/iter (+/- 3)
+test misc::reverse_suffix_no_quadratic       ... bench:       4,171 ns/iter (+/- 134) = 1918 MB/s
+test misc::short_haystack_1000000x           ... bench:     132,251 ns/iter (+/- 729) = 60491 MB/s
+test misc::short_haystack_100000x            ... bench:      13,184 ns/iter (+/- 408) = 60680 MB/s
+test misc::short_haystack_10000x             ... bench:       6,036 ns/iter (+/- 167) = 13255 MB/s
+test misc::short_haystack_1000x              ... bench:         602 ns/iter (+/- 14) = 13307 MB/s
+test misc::short_haystack_100x               ... bench:         230 ns/iter (+/- 7) = 3526 MB/s
+test misc::short_haystack_10x                ... bench:         218 ns/iter (+/- 3) = 417 MB/s
+test misc::short_haystack_1x                 ... bench:         210 ns/iter (+/- 8) = 90 MB/s
+test misc::short_haystack_2x                 ... bench:         225 ns/iter (+/- 6) = 120 MB/s
+test misc::short_haystack_3x                 ... bench:         211 ns/iter (+/- 8) = 165 MB/s
+test misc::short_haystack_4x                 ... bench:         212 ns/iter (+/- 6) = 202 MB/s
+test regexdna::find_new_lines                ... bench:  12,245,066 ns/iter (+/- 117,141) = 415 MB/s
+test regexdna::subst1                        ... bench:     786,357 ns/iter (+/- 14,200) = 6464 MB/s
+test regexdna::subst10                       ... bench:     788,550 ns/iter (+/- 26,456) = 6446 MB/s
+test regexdna::subst11                       ... bench:     782,161 ns/iter (+/- 15,583) = 6499 MB/s
+test regexdna::subst2                        ... bench:     784,902 ns/iter (+/- 23,379) = 6476 MB/s
+test regexdna::subst3                        ... bench:     786,640 ns/iter (+/- 27,063) = 6462 MB/s
+test regexdna::subst4                        ... bench:     785,591 ns/iter (+/- 20,498) = 6470 MB/s
+test regexdna::subst5                        ... bench:     787,447 ns/iter (+/- 20,892) = 6455 MB/s
+test regexdna::subst6                        ... bench:     784,994 ns/iter (+/- 19,687) = 6475 MB/s
+test regexdna::subst7                        ... bench:     801,921 ns/iter (+/- 15,391) = 6339 MB/s
+test regexdna::subst8                        ... bench:     785,541 ns/iter (+/- 11,908) = 6471 MB/s
+test regexdna::subst9                        ... bench:     785,848 ns/iter (+/- 28,020) = 6468 MB/s
+test regexdna::variant1                      ... bench:   2,195,058 ns/iter (+/- 44,066) = 2315 MB/s
+test regexdna::variant2                      ... bench:   3,219,968 ns/iter (+/- 59,372) = 1578 MB/s
+test regexdna::variant3                      ... bench:   3,776,467 ns/iter (+/- 54,326) = 1346 MB/s
+test regexdna::variant4                      ... bench:   3,803,674 ns/iter (+/- 95,281) = 1336 MB/s
+test regexdna::variant5                      ... bench:   2,661,333 ns/iter (+/- 46,408) = 1910 MB/s
+test regexdna::variant6                      ... bench:   2,645,716 ns/iter (+/- 38,659) = 1921 MB/s
+test regexdna::variant7                      ... bench:   3,228,352 ns/iter (+/- 69,155) = 1574 MB/s
+test regexdna::variant8                      ... bench:   3,305,563 ns/iter (+/- 59,321) = 1537 MB/s
+test regexdna::variant9                      ... bench:   3,225,039 ns/iter (+/- 49,720) = 1576 MB/s
+test rust_compile::compile_huge              ... bench:     100,381 ns/iter (+/- 2,052)
+test rust_compile::compile_huge_bytes        ... bench:   5,899,989 ns/iter (+/- 114,363)
+test rust_compile::compile_huge_full         ... bench:  11,650,995 ns/iter (+/- 172,285)
+test rust_compile::compile_simple            ... bench:       4,082 ns/iter (+/- 88)
+test rust_compile::compile_simple_bytes      ... bench:       4,153 ns/iter (+/- 120)
+test rust_compile::compile_simple_full       ... bench:      20,414 ns/iter (+/- 1,860)
+test rust_compile::compile_small             ... bench:       9,114 ns/iter (+/- 216)
+test rust_compile::compile_small_bytes       ... bench:     183,049 ns/iter (+/- 9,917)
+test rust_compile::compile_small_full        ... bench:     361,291 ns/iter (+/- 11,045)
+test sherlock::before_after_holmes           ... bench:     907,103 ns/iter (+/- 12,165) = 655 MB/s
+test sherlock::before_holmes                 ... bench:      62,501 ns/iter (+/- 1,880) = 9518 MB/s
+test sherlock::everything_greedy             ... bench:   2,062,116 ns/iter (+/- 41,900) = 288 MB/s
+test sherlock::everything_greedy_nl          ... bench:     894,529 ns/iter (+/- 38,723) = 665 MB/s
+test sherlock::holmes_cochar_watson          ... bench:     103,305 ns/iter (+/- 3,798) = 5758 MB/s
+test sherlock::holmes_coword_watson          ... bench:     479,423 ns/iter (+/- 13,924) = 1240 MB/s
+test sherlock::ing_suffix                    ... bench:     318,300 ns/iter (+/- 6,846) = 1869 MB/s
+test sherlock::ing_suffix_limited_space      ... bench:   1,066,300 ns/iter (+/- 19,375) = 557 MB/s
+test sherlock::letters                       ... bench:  21,777,358 ns/iter (+/- 230,478) = 27 MB/s
+test sherlock::letters_lower                 ... bench:  21,152,019 ns/iter (+/- 203,617) = 28 MB/s
+test sherlock::letters_upper                 ... bench:   1,777,626 ns/iter (+/- 26,243) = 334 MB/s
+test sherlock::line_boundary_sherlock_holmes ... bench:     897,509 ns/iter (+/- 24,983) = 662 MB/s
+test sherlock::name_alt1                     ... bench:      32,255 ns/iter (+/- 681) = 18444 MB/s
+test sherlock::name_alt2                     ... bench:      86,369 ns/iter (+/- 2,494) = 6888 MB/s
+test sherlock::name_alt3                     ... bench:      97,618 ns/iter (+/- 564) = 6094 MB/s
+test sherlock::name_alt3_nocase              ... bench:     944,848 ns/iter (+/- 31,039) = 629 MB/s
+test sherlock::name_alt4                     ... bench:     122,029 ns/iter (+/- 2,716) = 4875 MB/s
+test sherlock::name_alt4_nocase              ... bench:     225,544 ns/iter (+/- 5,783) = 2637 MB/s
+test sherlock::name_alt5                     ... bench:      91,897 ns/iter (+/- 3,796) = 6473 MB/s
+test sherlock::name_alt5_nocase              ... bench:     936,420 ns/iter (+/- 15,092) = 635 MB/s
+test sherlock::name_holmes                   ... bench:      33,448 ns/iter (+/- 959) = 17786 MB/s
+test sherlock::name_holmes_nocase            ... bench:     115,864 ns/iter (+/- 1,645) = 5134 MB/s
+test sherlock::name_sherlock                 ... bench:      22,474 ns/iter (+/- 674) = 26472 MB/s
+test sherlock::name_sherlock_holmes          ... bench:      22,184 ns/iter (+/- 497) = 26818 MB/s
+test sherlock::name_sherlock_holmes_nocase   ... bench:      99,629 ns/iter (+/- 2,398) = 5971 MB/s
+test sherlock::name_sherlock_nocase          ... bench:      99,523 ns/iter (+/- 2,674) = 5977 MB/s
+test sherlock::name_whitespace               ... bench:      30,815 ns/iter (+/- 107) = 19306 MB/s
+test sherlock::no_match_common               ... bench:      19,661 ns/iter (+/- 656) = 30259 MB/s
+test sherlock::no_match_really_common        ... bench:      27,544 ns/iter (+/- 527) = 21599 MB/s
+test sherlock::no_match_uncommon             ... bench:      19,553 ns/iter (+/- 31) = 30426 MB/s
+test sherlock::quotes                        ... bench:     369,144 ns/iter (+/- 45,316) = 1611 MB/s
+test sherlock::repeated_class_negation       ... bench:  68,838,857 ns/iter (+/- 330,544) = 8 MB/s
+test sherlock::the_lower                     ... bench:     321,692 ns/iter (+/- 5,418) = 1849 MB/s
+test sherlock::the_nocase                    ... bench:     507,936 ns/iter (+/- 3,080) = 1171 MB/s
+test sherlock::the_upper                     ... bench:      43,705 ns/iter (+/- 788) = 13612 MB/s
+test sherlock::the_whitespace                ... bench:     819,179 ns/iter (+/- 20,071) = 726 MB/s
+test sherlock::word_ending_n                 ... bench:   1,700,300 ns/iter (+/- 36,623) = 349 MB/s
+test sherlock::words                         ... bench:   8,249,767 ns/iter (+/- 75,015) = 72 MB/s
+
+test result: ok. 0 passed; 0 failed; 0 ignored; 119 measured; 0 filtered out; finished in 111.55s
+