That is likely because routed commands only check for command handlers from the currently-focused element up the visual tree through to the root Window that hosts it. If you have focus in a tool window, then focus is not within the SyntaxEditor in the document window, and thus the command handler there will not be used.
In that kind of situation it might be better to use DelegateCommand (we have an implementation in our Shared Library) where you have programmatic control over everything. Delegate commands don't rely on the visual tree like routed commands do, so they can avoid the issue you are encountering. If you are new to delegate commands, there is a ton of information on them on the web if you search "wpf delegate command". A delegate command execute action could then find the current primary document window, get its SyntaxEditor, and then fire the appropriate Find/Replace command on it programmatically.
I hope that helps!