Personally, I rank the games like this: AoE2DE > AoM >= AoE3 (haven’t played DE) > AoE4 > rest I don’t really care about.
For me, it mostly comes down to aesthetics and nostalgia. AoE2, AoE3, AoM are all games I played in childhood and remember enjoying a lot. AoE2 also had aesthetics that I really liked, the East Asian architecture was so cozy. It’s dumb minuscule stuff like these that stuck for ages. 
The isometric style of AoE2DE is also my preferred choice, though the 3D of AoE3 wasn’t a problem for me. I remember AoE3 looked very good on its release. I’m also very partial to fantasy tropes, so AoM was also right up my alley. For AoE3 as well, I really liked the home card system and how each city had its own screen (that also developed as you leveled up or something? I don’t remember much of that system) - since I played Heroes of Might and Magic games, seeing each city develop as you play and progress was very fun. I quickly searched for more about the home cities, and you can apparently customize them? That’d be a major selling point for me personally! (Right now I just don’t think I’d have the time to play it/learn it again, otherwise I’d get it.)
AoE4 just doesn’t do it for me visually at all. It looks cartoonish to the point of it looking goofy. When you make a cartoonish style, it has to land well and still look good. Good cartoonish styles are timeless - see WoW, for example. Bad cartoonish styles end up looking like a badly done realistic-style, which AoE4 feels like for me. The camera view and sounds also were a turn off, on top of having civs that don’t interest me. I also really don’t like the UI style, in other games the UI feels very cozy and welcoming, whereas the new modern sleek styles look very cold and distant. Similarly, I don’t like the UI of Humankind, while Civ 6’s UI looks amazing, even though Civ 6 is a very cartoonish game as well. Another example of a cartoonish style done well!
For me, AoE4 simply looks like a game where the visuals are still in a pre-alpha development phase and you’re expecting a high quality render to happen on the units.
Finally, AoE2 also has so many civs that for me is a bonus. Ideally, you could have representation from all parts of the world, making it feel new and fresh, since so many RTS games that have a historical focus very often end up having the same civs over and over. It’s my hope that the next DLCs in AoE2 could continue exploring other areas of the world, like Africa.
I guess game mechanics are more of an acquired taste, which I’ve grown to like as well. So I guess generally, it’s just mostly the featured civilisations, maybe also time period (I’m less partial to guns and stuff), and mostly just the graphics/visuals.