Growing AoE2 Sustainably: A Discussion on Data Structure

Age of Empires II: DE II - Discussion
1

TL;DR

This is not a post against DLC or new civilizations.

I hope Age of Empires II: Definitive Edition continues to grow.

However, as the game expands, maintaining long-term compatibility for data mods becomes increasingly difficult after each major update.

I believe it may be worth discussing whether future development could gradually move toward a more modular data structure that benefits both official development and the modding community.

I’m also interested in hearing how other modders and players feel about this.

Opening

“If you seek his monument, look around you.”
— Sir Christopher Wren (attributed epitaph)

Age of Empires II is one of the most remarkable long-lived RTS games ever made.

Not because of a single expansion.

But because it has continued to evolve for more than twenty-five years.

New civilizations, new campaigns, new mechanics, and continuous balance updates have all contributed to its longevity.

This is something worth appreciating.

However, I believe we are now at a point where another question has become equally important.

Not only:

“What civilization should come next?”

But also:

“Can the current architecture continue supporting another decade of expansion?”

This is not an anti-DLC post

I want to be very clear:

  • I enjoy new civilizations
  • I enjoy new campaigns
  • I want AoE2 to keep growing

This is not a suggestion to slow down development.

It is a suggestion to think about sustainability alongside expansion.

Growth vs Maintenance

From my experience maintaining AoE2 mods over the years, I have noticed a gradual shift.

Creating content is not the hardest part anymore.

Maintaining compatibility after official updates is.

Most mods do not heavily modify the entire game.

They often touch only a small set of:

  • Units
  • Technologies
  • Effects

Yet each major update often requires revisiting compatibility because underlying data continues to evolve.

This is not a criticism of the developers.

It is simply the natural result of a game that has been actively expanded for decades.

Civilization count is not the core issue

Many players hope AoE2 will eventually reach 80 or even 100 civilizations.

Personally, I do not think the number itself is the main problem.

The real challenge is that every new civilization also adds complexity:

  • more units
  • more technologies
  • more effects
  • more interactions
  • more balance relationships

The game becomes richer, but also more interconnected.

At a certain point, maintaining the foundation becomes just as important as expanding the content.

A note on design direction

It also feels like civilization identity has gradually shifted over time.

Instead of “fully complete but different tech paths,” many civilizations are now defined by:

  • missing certain standard technologies
  • compensating with unique mechanics
  • or altering core rules in specific areas

This is not inherently good or bad design.

However, it does contribute to a feeling that differences between civilizations are increasingly defined by what is removed or replaced, rather than entirely parallel development paths.

Perhaps it is time to think about the foundation

I am not suggesting a rewrite of the engine.

Nor am I suggesting that DLC should stop.

Instead, I wonder if future development could gradually explore more modular data structures.

Many gameplay changes revolve around three core categories:

  • Units
  • Technologies
  • Effects

If these systems could become more independently overridable (for example through partial data overrides or layered definitions), it might help reduce long-term compatibility costs.

Not only for modders, but also for future official content.

Why this matters beyond modding

From a community perspective, better long-term compatibility would help with:

  • Mods surviving game updates longer
  • Faster adaptation after patches
  • More experimental community content
  • Less fragmentation over time

From a development perspective, it could also potentially reduce friction as the game continues expanding.

Looking beyond the next DLC

Adding another civilization will always be exciting.

But strengthening the foundation that supports every civilization may be just as important.

AoE2 has already proven it can thrive for more than twenty-five years.

If it is to continue for another twenty-five, then perhaps the discussion should not only be about what we add next—but also about how we ensure that everything we add can coexist sustainably in the long run.

Final note

I’m not claiming the current system is broken. It has clearly worked extremely well for a very long time.

I simply believe that as the game continues to grow, it may be worth having a broader discussion about long-term data structure and mod compatibility.

I’d genuinely be interested to hear how other modders and players handle these issues, and whether they feel similar challenges over time.

1 Like
2

The post is a bit vague, like it’s written in AI language, I don’t quite get what you propose in practice, what does working on the foundation and data structure implies? You repeat many times you are not against more content per se but… And then I don’t get what you propose different.

What they could do tomorrow morning for modding is adding some features to cut corners like you still have to reskin manually every graphical asset which is tedious cause you need to check genie editor and could be simplified with an apposite reskin effect.

Another one could be the possibility to copy paste pieces of triggers instead of having to copy all of it or none.

They could integrate AI commands and slideshows in the editor without you having to continually go back and forth to check the files. In general JSON files are abysmal to use to create slides.

They should separate objects from chronicles and the main game in two bars cause they’re too many right now and it takes time to find them. Some of them even have the same name in some languages which occasionally messes with triggers.

I think they’ve actually been decent regarding retro compatibility. It definitely could be improved but for example when they added centurions they did it by adding a completely new unit as to not break old scenarios using old centurions. However this leads to the problem, as you said, of adding stuff on top of the other, duplicating, creating confusion etc. I’m not sure how you can keep the two together without having any downside if that’s what you’re asking. I’m not knowledgeable enough about technical stuff to tell if there’s a way to make some passages automatic for modders cause as you said it’s kinda annoying to keep mods updated, specially when they’re a lot.

And aside from technicalities but still connected to the work of modders… It’s basically a job so I’m not saying they should pay modders but at least there should be some kind of support since the community doesn’t seem to care much for that. They were modders themselves and they know mods nowadays keep a game alive so right now modders are basically doing some of their job for free. We’re not in 1999 anymore when a game used to come out finished and polished and get an expansion disc and two patches at best. Now virtually all games are basically mods constantly under work in progress, lines between what is official and not start to blur (think of chronicles and filthydelphia’s DLC) but this has to do more with the mindset and business model they aim to.

Right now as a modder I feel like I’m wasting my time cause there’s very little reward and in fact the scene is kinda dead or close to die, compared to the past. Some few members of the community like Kemsyt were willing to revive the scene with nice ideas like interviewing creators and showcasing mods but they can’t do much alone. Of course no one, me first, likes to pay for what they have for free but that’s not a sustainable model in the long run and it’s ethically dubious since art like any job needs to be retributed in some ways, the same way you don’t pirate stuff on the internet or see a concert without paying unless the musician is rich enough to allow you that. Not all mods are art or should be payed but… you payed for chronicles so what’s the difference? I already tried to push this in the past here but I got literally insulted by the community and, given Microsoft is Microsoft, I don’t have much hopes but since you asked I’m gonna have to repeat that.

1 Like
3

Thanks, I think that’s fair feedback. My point wasn’t just “improve modding tools”, but to reduce the need for mods to replace entire datasets. For example, supporting partial data overrides (layered changes instead of replacing whole unit/tech/effect tables), or separating Chronicles from the core dataset so content can evolve more independently. I probably should have made those examples more explicit.

2 Likes
4

So did you “write” with AI to not?

5

How will that improve modding?

6

It would improve modding by reducing cross-dependencies. Mods would no longer need to account for Chronicles content when modifying core gameplay data, which reduces compatibility issues and update overhead after patches or DLC.

7

What about regular monthly patches?they break mods regardless if they add anything new or not.The most recent changes broke ingame campaigns also which has nothing to do with new civis or data related to them.

8

Patches are expected as part of ongoing development, but ideally data modules should remain stable enough that mods don’t require structural changes after each update.

This wouldn’t eliminate mod breakage entirely, but it could reduce unnecessary cross-dependencies and limit how far each patch propagates into mod compatibility issues.

9

Maintaining data mod long term compatibility has always been difficult, it was already in HD version, it’s still now.
But now with XS we can bypass that issue by relying heavily on script integration, either as map script or as randommapscript to modify the content without needing a datamod. That’s the path I choose to avoid the long-term update issue.

That’s why they are improving it by time, the original file format would never have supported the current content in the game. Infact they’re doing clever choices to lower the file size requirment while not messing the game rendering.

Romans ! Wait…That’s the next Chronicle coming this winter already !

As mentionned, with XS scripting we can easily recalibrate things, like stats,for now. I did not dive into the Technology part of XS capability yet, it’s on my todo list of things to be done, but yet I’ve never needed to alter a vanilla tech a way I could not do with trigger using blank techs.

That’s the main reason I picked XS modification rather than Datamod, I let the game balanced be done by the dev teams, and then I just add my XS variation of things ontop of the game. That way I can easily tweak my own changes to stay balanced. The best part in this is that you don’t even have to change the scenarios file, you just change the xs file and if you made no changes in the functions name calling, then it’s just running smooth (if no syntax error, of course)

However, it’s true that XS rely more on programming, but it’s not mandatory for anyone to be a good programmer to use xsEffectAmmount, it’s a code equivalent of Modify Attribute trigger, so it’s only knowing what value equals what, no real programming skil here.

Some civ design are old, some are recent, and what defines a Civ is only known by the dev team. Personnally I feel the opposite, I feel every civ is growing in their “own identity” with time. Of course some elements become regionals, but it’s generally legit for those things to be like that.

They’re doing that already since the hiring of Forgotten Empire long time ago. They’re adding new mechanics features to the engine, new tasks, new behaviors, etc. So basically the game is in good track, as it’s going where you would want it to go.

Cross platform is part of that long-term compatibility they are already doing, but they’re doing not the way you think of it. As a programmer myself I can tell you that they’re improving the game every major update, not just DLC. It’s crazy to see how the engine evolved between 2017 and now, so many features were added, polished, expanded, and not just for modding.

It seems that’s exactly what they’re planning. But it’s not fact so far, it’s only theroy, so stay tuned for more info from the devs soon™

Teuton Knight op, nerf them ! (joking, they’re juste a nightmare to fight, and they should always be !)

1 Like
10

Thanks, that actually makes sense regarding XS — it does help avoid a lot of datamod maintenance by pushing logic into runtime scripting, especially for things that don’t need deep DAT changes.

What I was getting at is slightly different though. It’s more about whether the underlying data structure could reduce cross-dependencies in the first place, so fewer things break or need re-mapping no matter whether people use datamods or XS.

XS feels like a solid workaround at the script level, while my point was more about the data layer itself.

11

XS modding is abit different, as in you have to think of it as a portable trigger-kit packed into a file. As a modder and a programmer, I really enjoy that a LOT ! And I’ve only touched the surface of it yet.

Let me explain. Currently if you open the RMS file for the mission called “Thames”, you’ll see some code that applies BEFORE any ingame triggers, but AFTER the .dat file loading, that strictly changes the Rock Church ingame sprite to a special Wololo redbull event london inspired renaissance building. The Royal Albert Hall sprite is currently in the client, in the graphics tab of the dat file, but is not tied to any object in the dat file at all.
You can see that by your own eyes by just loading a Thames map on your editor, under the map generation settings, and there you’ll see that your Rock Church object is now the Royal Albert Hall.

Further detail about this mechanics is that, the .dat file is loaded first, then the RMS script (which are xs type scripts) are loaded, and after that, triggers inside scenario are loaded.

Triggers and XS files can easily serve for custom scenarios, while the RMS script can definitly be used in a “datamod lite” way. It’s restricted, you can’t do all the thing you can do with a datamod, yes, but the goal of a RMS is not to be a Datamod.
However, from the little I read in that Thames script, I can tell you that there is potential for a total conversion script, multi-player compatible, no-datamod.
And that’s the kind of technic I’m planning to use for my naval project, for the non solo-campaign stuff I’d like to share with the community (aka extended naval combat style, and renaissance era things)

So I’d say the XS is not a replacement at all, it’s just a new powerfull tool available that changes the way we can think about mods.

But think about that too, if it’s hard for us, content creator, to maintain long-term compatibility, it’s also complexe for them. So as long as they can achieve it, we should too be able to. And really it’s incredible the freedom we have today compared to the restrictions of previous versions of the game !

About things to be remaped what so ever, it’s the problem with any type of update, if they are not done properly, they break things.
But so far, the way of updating content is clever, like when they transform an object that was there in the client for long time into a civ feature, for example dragon ship, they don’t remove the original one or breakes it, they just make it hidden in editor and duplicate it into a new unit.

And about data, they’re strongly improving it with time, inclusing jsons, etc. The only real issue is that it’s not very well documented xD

12

Thanks, that clarifies the layering order — especially how RMS/XS sits between DAT and scenario triggers.

I think we’re largely looking at the same system from different perspectives. XS/RMS clearly gives modders a powerful way to work around DAT-level limitations, which already improves practical maintainability a lot.

My original point is more about the data structure side itself — whether reducing cross-dependencies at the DAT level could further reduce the need for these workarounds in the first place.

But I agree that from a modding workflow perspective, the current direction (XS/RMS + triggers) is already very capable.

13

I’ve worked with a lot of vide ogames structuring with the years, even started my own projects, and I’m really stuned by the quality of Genie engine, especially with how old it is.

Yes it’s not perfect, but perfection doesn’t exist.

In the recent years, the AGE tool was improved a lot, many clarifications were done, and XS also got extended a lot, as well as triggers etc.

So my advice would just be to let the time do the job. Because the amount of improvement, and accessibility, we received since Battle for Greece is Insane, and it’s only starting from what I’ve seen.

But you’re right about the amount of data to process, the more the game grow, the more files are in…That also mean that if you’re getting overflown with that amount of data, maybe your way of working with it is obsolete, as in you could learn new ways of handling those things.

In the end, time can only improve the game, as it did since the day I played my first AoK game, and my first TC game on a Day one unpatched CD version featuring lots of bugs I thoght was flaw memories but ended to be very rare witnesses of day one release version. (longship embarking units, turtle ship embarking unit, western european guard tower on atlilla’s first mission)

1 Like