Hello,
In that scenario with UseHostedPopups="False" and FloatingWindowShowInTaskBarMode="Never", this situation could occur due to how WPF Windows work.
FloatingWindowShowInTaskBarMode="Never" will cause the WPF Window that hosts the floating document to be "owned" by the main WPF Window that hosts the DockSite. When a WPF Window is owned by another, it always appears on top of the owner. Normally only tool windows are set up to be that way. If we didn't make sure the floating document WPF Window was owned by the main WPF Window, and the floating document WPF Window didn't appear in the taskbar, it would be able to fall behind all other windows in z-order and wouldn't be able to be accessed any more.
As you then show the non-hosted auto-hide popup (which is also a WPF Window) on top, the owned floating document WPF Window gets stuck in z-order between the main WPF Window and the auto hide popup WPF Window. There's unfortunately nothing we can really do about that.
It's probably not a good idea to those two options at the same time, since it's not a particularly good outcome in this scenario.