Find component matching regular expression is incorrect

Thank you for reaching out. SyntaxEditor does not currently support "\1" as a backreference match. Any unrecognized escape sequences will match on the character after the "\". In this case, that is why you are seeing the match on the literal value "1". Supporting backreferences is something we can research for a future update.  You can find full documentation on what is supported here:

https://www.actiprosoftware.com/docs/controls/wpf/syntaxeditor/regular-expressions/language-elements

The behavior you are describing in the second scenario is the behavior we would expect. The pattern "(ss?)" should match on both characters. When you perform a "Find Next" it will start after the previous match. This is why the "s" at range [2,2] is not matched because it was already part of the previous match. I tested Visual Studio 2022 and saw similar behavior to what you are describing, but other editors I tested (including VS Code and Notepad++) behave the way SyntaxEditor behaves. Visual Studio's current behavior may be a bug.

How can I implement the search logic by expanding myself? At present, I have not found a way to expand. We have solved the regular expression matching problem by ourselves, but it cannot be applied to the control

The built-in search overlay pane does not currently support any extensions that would allow for custom search logic, but I have made a note about the inquiry. At this time the only option would be to disable the built-in search overlay pane and replace it with one of your own design.