I feel that we should have a change in how a player becomes eliminated in a skirmish/multiplayer game. At one point, it was the destruction of all landmarks. Now it is the destruction of non-wall units and buildings.
My suggestion is this:
A player is eliminated if all of their landmarks are destroyed (not rebuilt) and they have no more villagers or unit production buildings standing. Furthermore, once a player is eliminated, all of their destroyed landmarks can never be rebuilt.
You can call this quite a middle ground between how to eliminate a player.