This is just a short meta-post relating to discussions on the forum. It’s also relevant for developers if they read these things.
Balancing a game like AoE2 is all about figuring out relations between game entities, especially units. You want to set up relations such that units can be countered, have effective niches, strategy is not dominated by like 2-3 strats, etc. For example you don’t want strategies involving a unit to be pareto dominated by strategies not involving that unit (i.e. for any strategy where you use unit X, you can find a better strategy that doesn’t use unit X).
Often times these dynamics are so complicated that even if you have data on the aggregate effects of the existing relations, the individual causal effects of each relation can be hard to pin down without training or tons of experience. One way around this is to explicitly go looking for knowledge of individual relations. Things like “Does unit X perform well in situation Y”, if not what are the limiting factors? economic? unit stats? dynamics? etc. This can be done through exploration in the exploration-exploitation sense. Forcing yourself to use a unit is one example of this exploration, as is running unit tests, doing mathematical analysis, or one of many other things. This gets very complicated with multiple changes where you need to take interaction effects into account.
Now some might expect that pros do a lot of this relation-testing or exploration to really dive into these relations. After all knowing them should, in theory, allow them to better optimize their strategies and inform which strategies are worth practicing/perfecting and which are kind of mediocre. But from observation some AoE2 pros are much better at this than others. Even among individuals who are very good at exploring alternatives (e.g. Viper, Daut) there is often evidence that there are serious holes in their knowledge. The cause of these holes can by any one of a variety of things that I won’t go into but the point is they are there.
Below is an example where Viper is doing his standard commentary. He comes across a situation where hes about to use elephants vs farimba camels. Now anyone who has done at least some testing on elephants knows that Burmese elephants don’t do that well vs buffed Camels. Really no matter what you do it’s going to be a pyrrhic victory at best for the elephants unless you start getting pop cap to work for you.
In the video it’s clear Viper has a lot of holes in his knowledge of camels and elephants. Doesn’t know the bonus damage of camels, is very unsure of elephant vs camel outcomes, is trying to draw conclusions from a few samples with a lot of potential confounding variables present, etc. And again this is one of the most experimental AoE2 players who, as the video demonstrates, is open about his lack of knowledge which is probably why hes the GOAT. Many pros give off-the-cuff remarks which can be biased/low confidence statements and referencing these can be problematic.
I only bring this up because people defer to the pros a lot and the devs rely on high-level player input from time to time. When having a discussion about which relations are good/bad for the game, knowing which answers can be trusted and which should be analyzed more critically is critical. Rule of thumb: the less used the unit is the larger the underlying confident interval is. Pro knowledge of meta units is probably pretty on target and high confidence. Pro knowledge of off-meta units or UUs is potentially biased and low confidence. This also goes for strategies, civ bonuses, etc.
