Welcome

Please register for Total War Access to use the forums. If you're an existing user, your forum details will be merged with Total War Access if you register with the same email or username. For more information please read our FAQ’s here.

Categories

The Warhammer Battles Survey - Results are in!

FredrinFredrin Senior MemberLondonPosts: 3,012Registered Users
edited August 2017 in General Discussion
Thank you to everyone who took a few minutes to respond to the recent Battles Survey. We now have over 1,000 responses and some excellent data, so it's time to share some of the findings.

For those who haven't seen it yet, the survey will be open for a few days longer to capture the opinion of people who don't visit the forums as regularly:

Click here to take the Survey


A quick reminder that this is meant as a fun and constructive exercise. Please don't use the results to undermine the efforts of the developers and bear in mind that every person who took this survey is entitled to the opinion they gave, regardless of how the numbers stack up.


For those who want to delve into the finer detail, here are two formats to view the results (click to download):

Quick Report

Excel Spreadsheet


As it can be a bit of a slog interpreting these, here is a brief written report, drafted as impartially as possible, to help highlight the most obvious trends and stats which I think will be most interesting to everyone. Words are limited in how much they can convey, so I'd encourage checking out the Quick Report/spreadsheet for closer analysis.

I'm open to some discussion of the survey itself in this thread, but I would prefer the focus to be on the results, what they mean in terms of game design and how CA can either build on its successes or fix areas where it looks like people aren't as satisfied. Please keep it constructive!

Summary of Results

Who took the Survey?

[All numbers in brackets are percentages]
  • Most results came from either these forums (34) or Reddit (53), with the rest coming from Steam, Twitch streams and various other sources.

  • The majority of respondents are in the 21-30 age bracket (61). 16% of people are in the 0-20 bracket, while 23% responded as 31 years or over

  • Medieval II and Rome II came in as the games most people had played (76 each). The first Shogun and Medieval games had the fewest players (26 and 33)

  • When asked how dedicated we are as fans, we averaged 7.6 out of 10 for TW and 6.4 for WHFB.

About battles:
  • A full 83% said battles were to some degree too fast, while 15% said they were spot on and only 2% said too slow.

  • On average, people voted for battles to be 34% slower. Those who wanted it faster (50 in total) averaged 2%.

  • 76% of players said that the game would benefit from more tactical options against 1% who said it needs fewer.

  • On average, people gave the battles a 6.6 out of 10, when asked how well "epic" describes them.

  • The siege battles scored a slightly unfortunate 3.9 out of 10 when people were asked for their general feelings on them.

  • There was no great desire for map size to be larger - a 6/10 showing a slight preference from some for larger maps and this is almost exactly the same story for unit sizes.

  • There was a high standard deviation in results when people were asked how "Arcadey" they thought the game was. Overall, it got a 4.6/10 in terms of accuracy.

  • A whopping 89% said they would like to see more depth giving features in future battle design - 68% either agreeing or strongly agreeing, 2% saying they disagreed.

  • People are by and large quite happy with the size and variety of unit rosters, giving that category a 7.5/10, while the overall game score got a healthy 8.1/10.

  • People rated battles an overall 7.9, but were a bit less happy with campaign strategy which came in at 6.9.


  • The marketing team at CA can pat themselves on the back - a huge proportion of people gave their hype levels for Warhammer a perfect 10, with the overall hypometer at 8.6.


Hope you enjoy discussing the results and please share with anyone who hasn't seen it: https://kwiksurveys.com/s/SGbCa6sQ

Cheers,

Fredrin
«1345

Comments

  • ShelenShelen Posts: 11Registered Users
  • KandennKandenn Posts: 454Registered Users
    Nice survey, thanks a lot !
  • mahboi1mahboi1 Member Posts: 771Registered Users
    A very good survey. I am pleased with the results.
    Sigmar wills it!
  • MemnonMemnon Senior Member Posts: 581Registered Users
    Gj @Fredrin .
    Im really hope CA stuff check this and take any conclusion from this survey.
  • krunshkrunsh Senior Member Posts: 3,676Registered Users
    Asking if people want more "tactical options" or "depth" was the weirdest and most biased part of the survey. You might as well as just asked: "Do you want more things or do you want less things?"

    Rest is good.
    i5-3550 CPU @ 3.30GHz
    Geforce gtx 970
    16gb ram

    Team Skaven
  • Commissar_GCommissar_G Senior Member Posts: 10,147Registered Users
    krunsh said:

    Asking if people want more "tactical options" or "depth" was the weirdest and most biased part of the survey. You might as well as just asked: "Do you want more things or do you want less things?"

    I had the same thought.

    As well as the Arcadey and Epic questions.
    "As a sandbox game everyone, without exception, should be able to play the game exactly as they see fit and that means providing the maximum scope possible." - ~UNiOnJaCk~
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users
    krunsh said:

    Asking if people want more "tactical options" or "depth" was the weirdest and most biased part of the survey. You might as well as just asked: "Do you want more things or do you want less things?"

    Rest is good.

    I'll happily admit that specificity is not a strong point of this survey. I might follow up with something more detailed but it would have become quite long and involved if we started drilling down into individual features and options. I might well do a follow-up survey for this though.

    I guess what you might infer from these results is that people don't feel like there are enough depth-giving tactical features in battle design as it stands, so would welcome efforts by CA to include more of these.

    There is an argument to say is it worth quizzing people on specifics if ultimately these are in the hands of the game designers? Perhaps it's more constructive for them to know there's a strong base level of demand for these features and leave it to their creativity to design the actual solution.
  • harngersteinharngerstein Posts: 808Registered Users
    edited August 2017
    Great survey, well done.

    I do have one issue: this study is almost certainly contains a selection bias. The people who responded are people who frequent forums and are pretty passionate about the series (look at how few people had this as their first TW game). While I don't know the actual demographics, I would venture to say that the people who took the survey do not represent well the average TWW player (note how I specify TWW rather than TW). I'd say this probably captures the TW/ Reddit forum populations well, but the people who frequent these sites are on average more passionate and spend more time and money on gaming. Also Likert scales aren't great for consumer research, especially with a product/ IP which people are so passionate about.

    In conclusion I'd say this is probably the best study you could have done given the resources you have available. I just don't think CA cares so much about the opinion of its core fans for this series. This is their first series that has gone into the mainstream and at the end of the day $60 is $60, whether it comes from a passionate TW supporter or a casual gamer on his parent's budget who sees Kroq-Gar screen on a youtube ad and thinks its chill. Although I would love to see CA address this, perhaps I'm wrong
    Post edited by harngerstein on
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users
    edited August 2017

    Great survey, well done.

    I do have one fairly huge critique: this study is almost certainly contains a selection bias. The people who responded are people who frequent forums and are pretty passionate about the series (look at how few people had this as their first TW game). While I don't know the actual demographics, I would venture to say that the people who took the survey do not represent well the average TWW player (note how I specify TWW rather than TW). I'd say this probably captures the TW/ Reddit forum populations well, but the people who frequent these sites are on average more passionate and spend more time and money on gaming. Also Likert scales aren't great for consumer research, especially with a product/ IP which people are so passionate about.

    In conclusion I'd say this is probably the best study you could have done given the resources you have available. I just don't think CA cares so much about the opinion of its core fans for this series. This is their first series that has gone into the mainstream and at the end of the day $60 is $60, whether it comes from a passionate TW supporter or a casual gamer on his parent's budget who sees Kroq-Gar screen on a youtube ad and thinks its chill. Although I would love to see CA address this, perhaps I'm wrong

    Cheers @harngerstein

    It's a fair point - the sample is not perfectly representative of the entire player base. You have to be quite dedicated a fan to visit and then take part in something like this.

    What I will be interested to see is whether the assumption that young or more casual gamers have a widely divergent opinion on gameplay mechanics to older more hardcore fans is indeed a fair one. There is generally an assumption that younger gamers prefer faster battles, but filtering the results down to age group doesn't necessarily support that:


  • harngersteinharngerstein Posts: 808Registered Users
    Fredrin said:

    Great survey, well done.

    I do have one fairly huge critique: this study is almost certainly contains a selection bias. The people who responded are people who frequent forums and are pretty passionate about the series (look at how few people had this as their first TW game). While I don't know the actual demographics, I would venture to say that the people who took the survey do not represent well the average TWW player (note how I specify TWW rather than TW). I'd say this probably captures the TW/ Reddit forum populations well, but the people who frequent these sites are on average more passionate and spend more time and money on gaming. Also Likert scales aren't great for consumer research, especially with a product/ IP which people are so passionate about.

    In conclusion I'd say this is probably the best study you could have done given the resources you have available. I just don't think CA cares so much about the opinion of its core fans for this series. This is their first series that has gone into the mainstream and at the end of the day $60 is $60, whether it comes from a passionate TW supporter or a casual gamer on his parent's budget who sees Kroq-Gar screen on a youtube ad and thinks its chill. Although I would love to see CA address this, perhaps I'm wrong

    Cheers @harngerstein

    It's a fair point - the sample is not perfectly representative of the entire player base. You have to be quite dedicated a fan to visit and then take part in something like this.

    What I will be interested to see is whether the assumption that young or more casual gamers have a widely divergent opinion on gameplay mechanics to older more hardcore fans. There is generally an assumption that younger gamers prefer faster battles, but filtering the results down to age group doesn't necessarily support that:

    Rather than the assumption of younger players wanting shorter battles, I'd love to see the breakdown of battle speed preferences based on whether or not players are casual. I know you didn't have players report on that particular variable, but I'd be interested if there was a significant difference btw speed/depth preferences for players that reported high vs low dedication to TW.
  • Sebor02Sebor02 Posts: 317Registered Users
    Good to see most do want slowdown of battles. Common CA please slowdown battles by about 3 min for the main clash
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users


    Rather than the assumption of younger players wanting shorter battles, I'd love to see the breakdown of battle speed preferences based on whether or not players are casual. I know you didn't have players report on that particular variable, but I'd be interested if there was a significant difference btw speed/depth preferences for players that reported high vs low dedication to TW.

    I did actually quiz people on how dedicated a fan they consider themselves. I'm comparing that data against preference for battle speeds now.
  • krunshkrunsh Senior Member Posts: 3,676Registered Users
    Fredrin said:

    krunsh said:

    Asking if people want more "tactical options" or "depth" was the weirdest and most biased part of the survey. You might as well as just asked: "Do you want more things or do you want less things?"

    Rest is good.

    I'll happily admit that specificity is not a strong point of this survey. I might follow up with something more detailed but it would have become quite long and involved if we started drilling down into individual features and options. I might well do a follow-up survey for this though.

    I guess what you might infer from these results is that people don't feel like there are enough depth-giving tactical features in battle design as it stands, so would welcome efforts by CA to include more of these.

    There is an argument to say is it worth quizzing people on specifics if ultimately these are in the hands of the game designers? Perhaps it's more constructive for them to know there's a strong base level of demand for these features and leave it to their creativity to design the actual solution.
    My commentary was more on the line of: The question should be "are you satisfied with the amount of depth in battles" rather than "Would you like more depth".

    Being specific isn't really the importance here, just the bias. If you asked:
    - Are you satisfied with the graphics?
    or
    - Would you like better graphics?

    Your results would differ greatly. Most people seem satisfied with the graphics, yet almost everyone would always enjoy better graphics.

    Also the meaning of "tactical options" and "depth" is kind of a mystery. One might read tactical options as the amount of clickable ability in an army, another in the variety of units. Again one might read depth as the size of the roster/content, another in how different each faction is.

    I know I had no idea what to make out of these two questions.

    I'm glad to see most forum and reddit folks want slower battles though, shows how much the "Use half-speed!" crowd who are happy with the current speed are wrong in their assumption that most people feel the same way they do.
    i5-3550 CPU @ 3.30GHz
    Geforce gtx 970
    16gb ram

    Team Skaven
  • HisShadowBGHisShadowBG Posts: 2,541Registered Users

    Fredrin said:

    Great survey, well done.

    I do have one fairly huge critique: this study is almost certainly contains a selection bias. The people who responded are people who frequent forums and are pretty passionate about the series (look at how few people had this as their first TW game). While I don't know the actual demographics, I would venture to say that the people who took the survey do not represent well the average TWW player (note how I specify TWW rather than TW). I'd say this probably captures the TW/ Reddit forum populations well, but the people who frequent these sites are on average more passionate and spend more time and money on gaming. Also Likert scales aren't great for consumer research, especially with a product/ IP which people are so passionate about.

    In conclusion I'd say this is probably the best study you could have done given the resources you have available. I just don't think CA cares so much about the opinion of its core fans for this series. This is their first series that has gone into the mainstream and at the end of the day $60 is $60, whether it comes from a passionate TW supporter or a casual gamer on his parent's budget who sees Kroq-Gar screen on a youtube ad and thinks its chill. Although I would love to see CA address this, perhaps I'm wrong

    Cheers @harngerstein

    It's a fair point - the sample is not perfectly representative of the entire player base. You have to be quite dedicated a fan to visit and then take part in something like this.

    What I will be interested to see is whether the assumption that young or more casual gamers have a widely divergent opinion on gameplay mechanics to older more hardcore fans. There is generally an assumption that younger gamers prefer faster battles, but filtering the results down to age group doesn't necessarily support that:

    Rather than the assumption of younger players wanting shorter battles, I'd love to see the breakdown of battle speed preferences based on whether or not players are casual. I know you didn't have players report on that particular variable, but I'd be interested if there was a significant difference btw speed/depth preferences for players that reported high vs low dedication to TW.
    Can't you find this out by the hours played question?
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users


    Rather than the assumption of younger players wanting shorter battles, I'd love to see the breakdown of battle speed preferences based on whether or not players are casual. I know you didn't have players report on that particular variable, but I'd be interested if there was a significant difference btw speed/depth preferences for players that reported high vs low dedication to TW.

    Here you go - the left column is responses filtering out people who answered 0-6 in the "how dedicated a fan so you consider yourself" question. The right column shows little difference in concentration of "slower battles" votes from eyeballing the results. I'll see if this tallies in the numeric spreadsheet.



  • harngersteinharngerstein Posts: 808Registered Users
    Cool stuff. Thanks for putting this stuff together. I'll be perusing plenty after work
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users

    Cool stuff. Thanks for putting this stuff together. I'll be perusing plenty after work

    Nice one, let me know if you find anything interesting :)
  • BrakierBrakier Posts: 1,522Registered Users
    krunsh said:

    Asking if people want more "tactical options" or "depth" was the weirdest and most biased part of the survey. You might as well as just asked: "Do you want more things or do you want less things?"

    Rest is good.

    there is a diffrence, tactical is about the gameplay in battle , while i see deepth as the amount of lore used in spells abilites buildings ect ect. thats how i saw it .

  • bellyboybellyboy Posts: 144Registered Users
    Fredrin,

    You win the forums. Great project! I'm very interested in the results: I thought I was too much in the minority for my opinions and found that was not as much the case as I expected.

    Thanks!
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users
    bellyboy said:

    Fredrin,

    You win the forums. Great project! I'm very interested in the results: I thought I was too much in the minority for my opinions and found that was not as much the case as I expected.

    Thanks!

    Thanks mate.

    Yes, it certainly puts the old "vocal minority" argument to bed doesn't it?





  • Southern_HunterSouthern_Hunter Posts: 164Registered Users
    The earlier poll also showed a significant majority (74%) of respondees (not necessarily players) wanted it slowed down to some degree (from 345 votes).

    Food for thought!
  • Ephraim_DaltonEphraim_Dalton Senior Member Posts: 21,303Registered Users
    I feel you should have put in question on which unit size and battle difficulty people play. I found that battle duration can differ quite a bit based on those parameters.

  • mahboi1mahboi1 Member Posts: 771Registered Users
    And I'd say that 1000+ respondents is representative of the audience as a whole. Meaning that as a whole the audience want's slower (or at least not as fast) paced battles and more tactical depth. Sounds pretty definitive to me.
    Sigmar wills it!
  • mahboi1mahboi1 Member Posts: 771Registered Users
    edited August 2017
    But still I would agree that some of the wording could have been more impartial. Still very good overall.
    Sigmar wills it!
  • Ephraim_DaltonEphraim_Dalton Senior Member Posts: 21,303Registered Users
    edited August 2017
    mahboi1 said:

    And I'd say that 1000+ respondents is representative of the audience as a whole. Meaning that as a whole the audience want's slower (or at least not as fast) paced battles and more tactical depth. Sounds pretty definitive to me.

    No, it's not because it's not a true random sample and even if it was it would only be representative of forum goers, not the playerbase as a whole.

  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users
    mahboi1 said:

    But still I would agree that some of the wording could have been more impartial. Still very good overall.

    I probably let myself down a bit on the "epic, arcadey and depth-giving" questions *hangs head in shame*

    Still, the answers provided have good balance and I think most people know there minds enough not to be too badly led on these things, so the results give quite a good picture of opinion.

    The only exception to that might be "Arcadey" - you can tell from a very erratic spread of numbers that it caused quite a bit of confusion with some people. I think it has many different and contradictory connotations - some some people it's a positive, for other quite the opposite. Without providing context, it's hard for people to know what they're being asked.

    I think that question and the general vagueness of the "tactical options" questions are probably the biggest sore thumbs, but there's still a lot of great data in there if you can be bothered to dig around a bit.
  • FredrinFredrin Senior Member LondonPosts: 3,012Registered Users

    mahboi1 said:

    And I'd say that 1000+ respondents is representative of the audience as a whole. Meaning that as a whole the audience want's slower (or at least not as fast) paced battles and more tactical depth. Sounds pretty definitive to me.

    No, it's not because it's not a true random sample and even if it was it would only be representative of forum goers, not the playerbase as a whole.
    Honest question: do you know where it might be possible to get a sample of the active player base who don't read stuff on forum or discussion pages? I want that sample!

    I might ask a selection of youtubers to publicize the next one and run a version in tandem with one off forums for cross comparison.
  • Ephraim_DaltonEphraim_Dalton Senior Member Posts: 21,303Registered Users
    Fredrin said:

    mahboi1 said:

    And I'd say that 1000+ respondents is representative of the audience as a whole. Meaning that as a whole the audience want's slower (or at least not as fast) paced battles and more tactical depth. Sounds pretty definitive to me.

    No, it's not because it's not a true random sample and even if it was it would only be representative of forum goers, not the playerbase as a whole.
    Honest question: do you know where it might be possible to get a sample of the active player base who don't read stuff on forum or discussion pages? I want that sample!

    I might ask a selection of youtubers to publicize the next one and run a version in tandem with one off forums for cross comparison.
    I'm afraid that's about impossible to accomplish for a single person and you'd probably need at least the cooperation of a major retailer (like Steam or Amazon) for that. Some of them do record the purchasing habits of their customers to sell them to ad companies and such.

    What you can however do to actually at least get a random sample from the playerbases's online presence is to not actually put out polls out in the open but to randomly select forumgoers and send them a PM with a link to the poll, and keep a tally of which people you asked and how many did the survey (just so no one cheats and does the survey more than once). People with gripes are prone to voice their opinions more than those who are content and open polls select for these people.

    I don't know how many members this forum has, but I think 100 randomly selected people should do.

  • hendo1592hendo1592 Posts: 1,586Registered Users
    edited August 2017
    Because of its collection method (convenient and snowball sampling) it can not be generalizable. However, it does tell a story and for the most part a well thought out survey-and I appreciate the time of constructing it and putting out the data.

    I'll put more emphasis on this, just because it is not generalizable to the total population (TWW player base) does not mean it is not useful. In surveys, you can take multiple answers of a question with yes or no. Example: the question concerning how much the respondent wants battles slowed down, could answer the question: do you want slower battles? (Which Fredrin did by merging answers).

    Not generalizable, but it has useful information. And because of the huge gaps between "keep the same" or "slow down" I would argue it is highly likely the majority of players would like the game slowed down. The type of sample methods does hint and point a direction of the overall picture.

    Great job fredrin!!!
  • BeermachineBeermachine Posts: 48Registered Users

    mahboi1 said:

    And I'd say that 1000+ respondents is representative of the audience as a whole. Meaning that as a whole the audience want's slower (or at least not as fast) paced battles and more tactical depth. Sounds pretty definitive to me.

    No, it's not because it's not a true random sample and even if it was it would only be representative of forum goers, not the playerbase as a whole.
    While it's certainly not a true sampling, and only takes into account the most dedicated fans (the ones who use game forums), it definitely gives a good indication that the majority of those want slower battles.

    As for the rest, I'd bet that the vast majority are pretty ambivalent on this issue, and wouldn't be bothered if speeds stayed the same or slowed down (and might not even notice the difference unless it was radical).

    As for getting their opinions on a survey, next to impossible unless they are forced to fill one in. Otherwise your only get a sampling of the most dedicated more casual players, and again not a true sampling of the entire demographic.
Sign In or Register to comment.