Thursday, May 10, 2012

Voice of the Player: the PVP Forums

Mos Ila: Honorable Mention, Hives of Scum and Villainy
The PVP forums may be the most wretched hive of scum and villainy in the galaxy, but there are always a few holdout posts that I find helpful, or even inspiring.

What about the din, though?  Can something be learned from the countless threads of complaints, anecdotes turned into truths, and general hatefest?

I ran a program to download the approximately 32,000 threads posted to the PVP forums, taking the first page only.
Most threads are only active for a day before they no longer get replies (75%).  After a week of activity, only 5% of threads remain active. The median post gets around 5 replies and 240 views. There are a handful of notable outliers - including one post with around 330k views (a dev thread from Jan 18 that gave an update to Ilum's pvp).

Just from this activity, its immediately clear that 1.2 brought a dramatic increase in forum post volume. With the general renewed excitement over SWTOR with 1.2, this is a good indication that the volume of posts matches interest in the game. Posting is something that complements play time, not opposes it.

Class Warfare: calling down the nerfbat
What about the ever present cries for nerfs on the forums? Is the rise-to-power of sent/marauder reflected in forum posts? The stunlocking dominance of ops?

For this, I'm not going to do anything super fancy, just a simple search through the first 10 posts in each thread for mentions of each class (accounting for things like spelling mistakes, or clear references to a class via a popular spec, like 'concealment' matching to operative, or 'galactic overlord of awesomeness' to sorcerer.) I'll also combine the Empire and Republic advanced class mirrors, and reference them by their Empire names.
Looking at the percentage of threads that mention an advanced class, I'm actually astounded just how well that worked. The fights between operatives and sorcs, followed by the overwhelming numbers of sorcerers, until the rise of marauders with 1.2... its all reflected in the volume of mentions of each class on the PVP forums. In contrast: powertechs, snipers, and juggs are all but invisible.  Assassins have made a respectable rise, one that I'd expect to continue, given their impressive on-demand cooldowns and burst.

From this, I'd say forum chatter is a reasonable way to capture community awareness of issues.

Topical Applications
Next up: I sampled some of the other issues that players raised for discussion on the forums, looking mostly at pvp zones, exploits, and some quality of life issues. Again, for each category, I'm looking at word counts, accounting for things like misspellings and including related words.
The first plot nicely captures the rising attention given to Ilum, and its eventual irrelevance. Alderaan - the least innovative of the warzones - also gets the least attention.

Regarding bugs and exploits, mentions of bots and hacks are relatively infrequent - and steadily decreasing over time. Hopefully this is a reflection of the developers getting a better handle on the situation. Lag and bugs, however, remain persistent issues.

I'm glad to see that despite all the attention given to 'unsubbing' threads... they're rare. Like, people spend more time talking about powertechs rare. Premades and stunning, though - they're frequent issues.

But forum goers are weird!
For ages, developers have noted that players who post to the forums are 'different' and don't really represent MMO players in general. I think that makes intuitive sense.

But that only goes so far. Again I'll look at class balance. But this time, I'll divide posters into four groups, based on the number of posts each forum-goer made in my sample.

Post Groups 1 post 2-10 posts 11-100 posts 101 or more
Individuals 10,802 13,009 3,220 218
Posts Made 10,802 51,010 88,336 40,286
% of posts 5.7% 26.8% 43.9% 21.2%

Even though individuals who made 101+ posts are less than 1% of the population, they make over 20% of the posts in my data.
But those who post often are no different from those who post infrequently. Whether looking at those who make one post, or more than 101, the trend is still there. Each group is currently discussing marauders most frequently, was talking about sorcs before 1.2, and largely ignores the powertechs. It is definitely the case that the trend is smaller in the one-post group, but it is still there. 

Class balance concerns are relatively even across each group.

Forums: Windows into MMOs.

I'm pretty impressed with just how rich a source of information the forums can be (particularly given that this took... all of a couple hours to run, once I'd downloaded the data). 

But this was just what struck me while playing with the data. If you've got a pvp-forums related question, hit it up in the comments (or email) and I'll see what I can find.

3 comments:

  1. I'm not sure how this shows that forum posts are representative of real issues... I mean, what is your source and definition for what the current issues are? I would assume forums, blogs and fan sites, but that leads to kind of circular reasoning, doesn't it? /scratches head.

    More generally though, I think posts like these are amazing. So much data... :D

    ReplyDelete
    Replies
    1. I really don't think that forum volume is reflective of "real" issues.

      The post was unclear... because I started the project out of curiosity over what players were posting about on the pvp forums. I did the writeup around a few trends that matched nicely to intuitions I'd seen elsewhere (class balance on sithwarrior, the rise and fall of Ilum via patch notes...).

      But balancing classes based on the volume of complaints in the forums? Do not want.

      Delete