when you switch off the fog of war, it just shows the scouted map and current visible enemy units and buildings. Unfortunatly not the fromer discoverd enemy buildings.
I suppose I experienced the same when watching a replay of one of my single player 1v1 games. The exploration progress (with this I mean the enemy buildings visible in the fog of war) was lost when switching back and forth between the view of the two players.
My game build information: 188.8.131.52 in German
Thank you for the report! The team is looking into it.