Analyses of the ratings - Spotting the issues

WoodsierCorn696 · May 10, 2020, 8:04pm

I feel like ratings are a bit off. And i think i am not the only one who thinks there are some weird things into the ratings. So i decided to have a look.

I used the following source:
https://thbrown.github.io/aoe2-de-rating-charts/

This site only shows data for Ranked RM games. I would love to see some statistics about DM (both 1v1 and teamgames) and unranked maps. I dont really have a guess about DM, but i think the results of unranked are even more extreme than the results for RM team games.

1v1:

The distribution of ratings is pretty much symmetrical around 1000. Since this is the starting ELO, this is what I would except. Maybe there is a little bit inflation of the ratings, because you get/loose more points within your first 10 games. So this picture is no surprise at all to me. This is just the example of how ratings should be distributed to me.

I also havent heard much complains about not equally skilled match ups for 1v1. Some games seems pretty much fine.

Teamgames:

The distribution of ratings is centred around 1275. This is already pretty odd. I would not excepted so much inflation of the ratings. At some point ratings starts to inflate or deflate, but i dont really think an inflation of around 175 in just a half year is normal. This is for me a first sign something is off to team ratings.
Second odd thing is also the skewness. It is way too much skewed to the right. The right tail is way too tight. Again this is a sign something isnt right with the team ratings.

I have heard about some complaints about non equally match team ups, even the ELO of the players is pretty much equal. If ratings got inflated and skewed for some reason, it will result in no fair match up, since ELO doesnt reflects your skill level any more. All this together makes me wonder if the team ratings are calculated correcly.

Like i said: I do expect an even more inflated and skewed picture for unranked games.

Comparison 1v1 vs Teamrating:

For me it is really odd almost everyone seems to be better at Teamgames than 1v1.

Open questions

I feel like there is something wrong in the ELO calculations of team games and unranked games. It would be great if we can spot the issue. I have stated some question about this. Is would be great if we can understand what is going on and thus help the devs in fixing the real issue.

Why are team ratings so inflated?
Why are team ratings so skewed?
Is there some smart guy who can make pictures like this for DM 1v1, DM team games and unranked games? I would love to analyse the graph and add this to this post.
Can someone tell us the calculation of ELO in 1v1? I would bet it is the same as HD and Voobly: Just the Elo rating system with K = 32. For the first 10 games, they used a different K value, only for the new player.

Can someone tell us the calculation of ELO in teamgames? First thought would be it will be the same as HD and Voobly (see youtube against for that calculation), but this is not true.
Can someone tell us the calculation of ELO in unranked games. I havent seen any graph, but my bet was is is even more extreme than team games.

CactusSteak2171 · May 10, 2020, 11:08pm

My guess is that since the calculation of ELOs was crap before 2 weeks ago, the system kept feeding noobs to players who were laying regularly together, and thus team of even mediocre player could ladder climb by bashing randoms. Tbh I’m surprised it doesn’t look even worse 11 I guess all we can do is wait for the persons who have inflated ELO to lose games.

Yorok0 · May 11, 2020, 4:08am

They could very easily fix ELO inflation so that average is again 1000 by modifying ELO of everyone accordingly.

WoodsierCorn696 · May 11, 2020, 5:58am

For some degree it can be explained by the mismatch in matchmaking, which now is fixed.

If this guy is true, he just lose 2 points if he lose a game. Good luck for players with a skewed rating to get back to there real rating if they only loose 2 points each game. If he wins, he wil get 19 points. This is quiet strange to me and makes me things something else is wrong too.

You can easily set the average back to 1000, but is this really a solution to the problem? If there is something odd in the calculation the average will still continues to rise again, so after 6 months, ratings will still inflate. Also the skewness will still exists.

I think the first step will be finding the root of the issue. Then we can have a look at how to fix this.

If we want a good working matchmaking for team games, than we need a good Elo calculation.

WoodsierCorn696 · May 12, 2020, 2:28pm

Is there anyone who knows more about the calculation of Elo for DE?

EloquentNine840 · May 12, 2020, 4:12pm

It’s not always -2 but it’s often such a low number. I do consistently gain more rating than I lose though. I’m over 2100 now even though my winrate is under 50% lately. I probably will be 3000+ rating in a week at this rate.

Harooooo1 · May 12, 2020, 5:50pm

Could it be because when u are calibrating into ranked teamgame ladder, it doesn’t start you off at 1000, and instead it starts you off at close to what your 1v1 is. If u are for example 1300 in 1v1, it will start u off on the first game around that number, and then after u are done calibrating, u can easily gain +300-400 if u win a bunch of games.

Also it happens vice versa. My friend was calibrating teamgames before he did 1v1s, he managed to get up to 1700 teamgame, then when he started playing 1v1, it immediately put him to 1600 1v1 rate after the first game, which is like top 500 players

WoodsierCorn696 · May 12, 2020, 8:29pm

Yeah, it seems like there is some connection between both ladders in the first 10 games. This is really part of the explanation. This doesnt explain why @EloquentNine840 just loose some elo if he lose, but get much more elo if he wins. So it is certainly not the full explanation.

XuanZue · May 12, 2020, 9:24pm

There is no MMR for premades. In Starcraft2 they go for a draconian way but I like it.

Basically you have a different MMR for 2v2, 3v3 and 4v4. And if you want to play with a friend or some friends, then the team gets another MMR. compare it with MMR in aoe, where you have the same number for every scenario that is not 1v1.

People like to complain there (in SC2) that being a premade of gold have to play with diamonds, but that is how it works well. A good timed attack in 2v2 will screw 2 skilled and uncoordinated players.

So I’m having games extremely one-sided when playing alone in random teams, meanwhile with my premade we have mostly good games.

I’m pretty sure the inflation comes from the premades. hope microsoft check it.

DM could also be changed or rotated to other game modes like regicide or empire wars

MythicBubble289 · May 13, 2020, 8:21pm

Seeing ratings like this suggest a bug. The guy has been playing both 1v1 and team games, win rate close to 50% in each, yet his rating is 1327 in 1v1 and 2139 in teams. The same win ratio should produce close ratings, right?

MrBnutt93 · May 13, 2020, 9:01pm

Team game Elo is tough to balance out. I’m a fairly good example of this: I’m currently rated around 1350 in 1v1 but only 1450 in TG. Given the data shown here you’d expect my TG rating to be higher. I’d played on-and-off for a few years since HD came out and even a bit before on Gameranger. I’m very familiar with the game, counters, macro and micro, etc. I’m not the best player but I consider myself fairly experienced.

Most of my team games have been with my girlfriend. She noticed me playing DE when it came out. She hardly plays any video games at all and was 100% new to the game. We team up frequently and our results are mixed at our TG Elo. Sometimes I can 1v2, sometimes they try to rush me instead of her, and sometimes she gets rushed down and I face competent enough opponents that I can’t hold off the other team.

What’s a fair rating for me + my girlfriend on a team? Arguably the average of our skill levels might be somewhere in the middle, but since there’s so much variability in the skill of players at that level we get wild swings in how our opponents fare. Ideally some system would be able to determine that I have a disproportionate impact on our team’s wins and reward me with a higher Elo, but this is hard to quantify and make fair.

I do wish the two Elos were not linked when starting out. Since my girlfriend started playing in teams with me, she had an inflated 1v1 score when she first started. She lost her first 10 games in a row and just recently took her first 1v1 victory (with a little coaching from me ), Getting trounced 10 games in a row is no fun.

I can see the argument for taking someone’s established 1v1 Elo and setting a baseline for TG, but not the other way around. In TG it’s easy to be carried to victories, especially in the doldrums of low/mid level players.

MrBnutt93 · May 13, 2020, 10:20pm

You can’t determine Elo from win-rate alone. Let’s say A and B have both played 10 games. A’s games went: [win, win, win, win, win, loss, loss, loss, loss, loss] and B’s games went [loss, loss, loss, loss, loss, win, win, win, win, win]. Both players have a 50% win-rate but (assuming matchmaking was fair) A lost to higher-skill opponents than B, as their rank was higher with their initial wins when they lost.

WoodsierCorn696 · May 14, 2020, 5:46am

There is also a second reason why Elo and winrate arent comparable:
Matchmaking tries to let you play against players with 50% win chance. Therefore almost everyone will have a winrate around 50%. Only the top and bottom will have different winrates. The main part of players will be around 50%.

For example: On 1v1 only around 10% of the players has a winrate above 60%. If you have a look at those people there are two categories:

They havent played many games. If they would play more games, there winrate will probably drop and will be below 60%.
They are pros, the very top of the leaderbord.

This is exactly what i would expect. I havent checked the number of players with rating below 40%, but this will probably kinda the same. Or played few games, or have very low rating.

Maybe this is even kinda the same as your tried to say. So yeah, i fully agree winrate and Elo arent comparable.

MythicBubble289 · May 14, 2020, 5:44pm

But you can notice in the example quoted above, that the player has a streak of +3 wins for the 1v1 low ELO, and a streak of 4 losses for the high team ELO!! I know 50% win rate doesn’t nail the ELO exactly, but it suggests he hasn’t been dominating in one mode vs the other.
I am sure there is a bug here. But no info about their formulae to nail it down.

EloquentNine840 · May 14, 2020, 6:20pm

I don’t know about that. I started TG before 1v1. I’m 1250 in 1v1 yet my TG rating is so inflated that I can hold my own at 2150 rating now. I’ve seen TheViper play against people who were in my games even though I’m merely a humble slightly above average newb!

EloquentNine840 · May 14, 2020, 6:25pm

So if it’s only in the first 10 matches, I should just play a few more 1v1 games before I can compete in an actual 1250 rating game?

See… my last losses were:
-6
-9
-7
-10
Last wins were:
+23
+17
+21
+29

This is insane.
Sure last game I was drushed + 2v1 on water. I gave up water (as Japanese ) and survived on land while my pocket did nothing and resigned as soon as he got attacked. So obviously I lost less rating because they gravely mismatched the teams. It’s just… why? Just take more rating if I lose but for the love of God, why do this kind of matchmaking after every victory?

EloquentNine840 · May 14, 2020, 7:07pm

Where can I see this statistic? This is exactly how it is for me.[quote=“MythicBubble289, post:10, topic:81949, full:true”]

Seeing ratings like this suggest a bug. The guy has been playing both 1v1 and team games, win rate close to 50% in each, yet his rating is 1327 in 1v1 and 2139 in teams. The same win ratio should produce close ratings, right?
[/quote]

WoodsierCorn696 · May 14, 2020, 7:14pm

https://aoe2.net/

If you are logged in with steam log in at that site, you can go to https://aoe2.net/#aoe2de-matches-my-recent-matches and click on you name.

If you arent logged in, go to the leaderboard and search for your name. Than click on it.

Besides this graph, you can also see your match history with the elo of every player at the beginning of the match.

EloquentNine840 · May 16, 2020, 10:45am

Cool! thanks!
These are my stats. For some reason it didn’t record the first 9 games that I won in 1v1 (should be +9 longest streak).
It’s most curious that my TG rating started at 1500 whereas my 1v1 rating started at 1000 rating (which you can’t see on this graph).

Edit: Ahh I see! The first 9 or 10 games are test matches to gauge your rating. That’s probably why it starts me off at 1500 TG and 1300 in 1v1.

Another interesting finding is that I’m matched against 1815 - 2447 rated players (with one 16++ guy being the outlier as he seems to be premade with two 23++ players every game I encounter him in). I also see that everyone consistently has 600-700 TG rating more than their 1v1 rating.

WoodsierCorn696 · May 16, 2020, 11:12am

You get your first data point after the first game. Since you already had some unranked rating, it is used in the test matches. This will puts your starting rating higher than normal (1000). This is why your team rating start and 1v1 rating starts way above 1000.

I have had a look at your profile (i found it on de aoe2 based on this info). To me it doesnt really like Elo calculation is wrong for you. I had a look at some games and your elo before and after, but didnt really found strange things. Sometimes it looks like there was no elo won/lost, but this seems to be an error on aoe2. I have the same at my own profile. It has something to do with getting fast another match, i think.

Also you have a winrate of 60% for team games. This is pretty solid. MM needs to make sure you win around 50% of the games. For you it seems to be around 60%.

–

Just looking at some other profiles, i found the following:

Winning streak of 23 games out of 23. Unranked Elo is still 1000.

Note: I have the explanation: This guy only play multiplayer games with AI involved. Games against AI in unranked games are part of the number of games, streak, … but wont generate any elo. Luckily they dont generate any elo. I thought this can be the reason why Elo is inflated for unranked, but it isnt.