I don't mind losing track of enemy units there, but I do agree, you should still be able to see landscape features you've discovered: specific shards, lairs, quest locations, etc. After all, you saw them all, before. They aren't going to change, unless they're removed by the AI. In which case they shouldn't be updated on your befogged map, until you send another unit for later reconnaissance.
