That is what you need to do, yes. I'll add this to the to-do list on things to consider.
Ticket #3266 4X - RFE - Known planets removed from map
It is working as it was designed, and the design is still hitting the points it was required to cover for balance and completion.
Perhaps there is another way to approach and address these important concerns relative to planetary information during the post-invasion cycles and I'll look for a way to make that possible within the engine we're using.
No promises but I will find some time to look at it.