Define anchor in regex syntax

[pascalkuthe]

I have a usecase where I am accepting arbitrary regex syntax from the user (helix editor user can type search). In this usecase the user will search a subrange (start..end) within a larger haystack I am using regex-automata (or regex-cursor :P) to achieve that.

However, while this works great for word boundaries and line end/line start anchors there is one feature missing: anchor a search at the start/end of that subrange. I of course know that I could just set regex-automaton::Input::anchored but there are a couple issues with that:

• how would I handle cases where only some patterns are anchored (for example (\A|x)foo using \A as a placeholder for my hypothetical syntax).
• how would I handle anchors on backwards searches (for example foo(x|\z) assuming \z as a hypothetical placeholder syntax again)

I saw a bunch of issues/discussions around something similar but they all boiled down to someone doing this statically or being with using something like find(&haystack[5..]) which won't work in my case since line breaks and other look around assertions must not match.

Some more context: helix-editor/helix#9422 (comment)

[BurntSushi]

Hmmm, I found your description quite confusing here. I didn't really understand things (I think) until I read the linked issue. From what I can tell, you want to be able to execute a search at an arbitrary position in a haystack, but provide a way to optionally anchor the search so that all matches begin at the start of the search and/or end at the end of the search.

The intended way to do this is indeed Input::anchored, but that applies to the entire search. The only possible way this could be done on a pattern-by-pattern basis is if either some new look-around assertion were added or we added an option to change the interpretation of an existing assertion.

I don't think there's anything special about backwards searches here, although there is no "full match" support. That is, this crate has no way of ensuring that a match ends at the end of the search. I believe you'd have to check that after a match. Although I actually think there are cases there that would need support in the search routine itself. For example, if you wanted to do an anchored search at offset 1..7 for foo|foofoo in ZfoofooZ, today's implementation gives you no way to guarantee discovery of the full match at 1..7. Instead, it will just report the leftmost-first match at 1..4. You could... probably enable MatchKind::All to work-around this though.

To be honest, I'm not really sure what to do here. Do other editors support anchors with respect to selections?

[pascalkuthe]

I guess what I am asking is: Is it feasible to hook into regex-syntax somehow to provide that kind of custom syntax. The crate seems very configurable but I couldn't find a place to hook into for that kind of custom syntax (or if its even possible).

For the backward assertion, I was indeed thinking of the same problem you did but forgot about MatchKind::All. It even says that in the docs I guess sorry about that I didn't look in the right place.

I guess I will have to do some postprocessing to recover leftmostFirst semantics since that is what we actually want. I do wonder if that is the most efficient way to go about it but I understand if this is not something you want to support in regex-automaton further.

[BurntSushi]

Yeah there is no way to hook into regex-syntax. It's true that the crate is very configurable, but it isn't extensible or open. It doesn't support ways of arbitrarily injecting new syntax.

I think there are two ways to add this to the regex family. The first and possibly easier way is to add a new config knob that changes the interpretation of existing syntax. For example, anchors_match_at_start_or_end_of_search (terrible name) would make ^ match at the start of a search, regardless of where it is. The second is to add new syntax that isn't ^/$ or \A/\z and means what you want. I find the latter very unattractive, but I think I could potentially stomach the former. The problem is that this will probably require breaking change releases of both regex-syntax and regex-automata. I'm also not 100% certain this approach will work. I haven't thought through all of it.

[pascalkuthe]

I am not particularly keen on introducing a new syntax so in general the first option sounds good. but one thing that would be important for our usecase is that there is both an assertion fort the start/end of line/haystack and for the start/end of the search.

Kakoune handles this by just making \A/\z distinct from ^ and $. So the config option would make \A match the search start but leave ^ untouched. That sounds somewhat similar to the first option you mentioned but I am not sure if that is what you meant.

Would you be open to that. If that is ok I would give it a try to implement that and see how far I get.

[BurntSushi]

Yeah I think the option would impact ^, $, \A and \z, but not (?m:^) or (?m:$). So if you enable the option I mentioned above and multiline mode, then \A and \z will match at the start/end of a search and ^ and $ will then match at the start/end of a search and at start/end of line.

[BurntSushi]

I think I am potentially open to it, yes. I'm pretty certain it will require new Look assertions though, and those are usually a pain to add. It will require breaking change releases of regex-syntax and regex-automata.

[pascalkuthe]

Yeah I think the option would impact ^, $, \A and \z, but not (?m:^) or (?m:$). So if you enable the option I mentioned above and multiline mode, then \A and \z will match at the start/end of a search and ^ and $ will then match at the start/end of a search and at start/end of line.

Thanks for clarifying that sounds good to me!

I think I am potentially open to it, yes. I'm pretty certain it will require new Look assertions though, and those are usually a pain to add. It will require breaking change releases of regex-syntax and regex-automata.

I don't want to nail you on anything here so if you are not happy with my changes that's ok and we will work from there. I don't expect this to land upstream quickly or at all. Just wanted to get a feeling if you may be open to it so it would make sense for me to build a prototype at all (that may or may not get merged). I am not in a hurry and don't expect anything quickly :)