There was a raging debate as to whether the new Rune of Wrath and Rune was overpowered..... it was vigorous on both sides and there were a lot of interesting points made.
In the midst of the debate I made the assertion that if the new RoWR was that OP then we should see the Dwarfs do very good in Pick/Ban tournaments, and end up over represented in the pick rates. I also put up a post asking if CA would post win rates for the Dwarfs in QB, but heard no response.... but heard from others it would be extremely unlikely that they would.
So I decided to get the data myself.... after all, complaining about something without taking action is just whining.
I had hoped to get 750 games in the pick ban format, but only end up getting 661. I also started recording Blind Pick data, and Quick Battle data just because it was fun.
The results: The average number of games played in the pick/ban format was 88, and the Dwarfs played 79 games, which is below average but close enough to average for me to consider them so. The also had a 52% percent win rate over their 79 games, which is solid.
It has been noted that an ability can be overpowered without the faction being overpowered, and there is variance in these numbers for sure.... but based on the data, and personally playing the Dwarfs, talking other Dwarf players and watching a LOT of games, that the following is true:
1) The Dwarfs are not OP (though RoWR may still be)
2) The Dwarfs are competitive in enough matches to be a worthwhile faction to be able to play in pick/ban tournaments.
Some may argue those statements, but I think they are defensible.
That may seem like a lot of effort to understand those two things, but that is simply the nature of data.... there are no shortcuts.
As far as other bits of information that came out of the data, which simply emerged and were not anything I was looking for, is the nature of the types of tournaments being played, and the overall balance of MP in CA.
3) Multiplayer in Warhammer Total War II is incredibly well balanced.
The standard deviation of the win rate of each faction in tournaments was 0.0386, which is amazing. Essentially almost every factions overall win rate landed between 46% and 54%. I have heard some strong players say this game is balanced, and it is "all about the match ups", but I never imagined it would be this good.
KUDOS CA BALANCE TEAM!!!!!!! Seriously, just wow.
4) Pick/Ban tournaments are the most competitive, while blind pick tournaments have the most faction representation.
The standard deviation between win rates in Pick/Ban tournaments is 0.052 compared to the standard deviation in blind pick tournaments of 0.067. This somewhat quantifies the difference skillful faction selections play over pure luck in terms of win rate.
On the other hand the faction selection standard deviation in pick/ban tournaments is 32.3, where as for blind pick it is 14.4, which is a gargantuan differential. To put it another way, in pick/ban tournaments the top 3 factions were selected 418 times with the bottom 3 factions selected a total of 145 for a nearly 2.9:1 ratio..... in blind pick the top three factions were picked 259 times versus 141 times for total of the bottom 3 which is near a 1.8:1 ratio.
From a qualitative point of view, pick/ban tournaments are better at determining the best player, while blind pick tournaments are better at determining the best faction.
As someone who reviewed 1816 games this cycle, I will say that blind pick tournaments are much more interesting to watch, as in pick/ban you often see the same match ups over and over, but that is just my personal preference.
Beyond all this I have not really looked into what is possible with the data. I did put a few feelers out to see if the faction vs faction data could be made sense of beyond a 7 match set, but there is just not enough games to get there I think.
I do hope to see other peoples take on the data, and potential crunching of the numbers, as I did put all the spreadsheets out there for anyone to use.