I have 2 store views, one for normal customers (product prices including tax) and one for wholesale customers (product prices excluding tax). See images for reference.
Normal Store View (product prices including tax):

Wholesale Store View (product prices excluding tax):

And additionally, I have special prices for the customer group Wholesale in some products like this:

When I log in with a wholesale account for example in the store view EN (prices including tax), I see the correct special prices.

But when in the same session I change the store view, I lose the special prices and I only see the normal customer prices:

If I log in with a wholesale account but now in the store view Wholesale (prices excluding tax), I see the correct special prices too:

But when in the same session I change the store view, I lose the special prices and I only see the normal customer prices again:

I don't understand why, this happens to me only if I change the store view and it is very strange right? Is this an error for Magento? Any idea why this happens and how to solve it?