Quality assurance, or "QA", is a team at Jagex that ensures all new content in the game and on the website is of high quality, focusing on the stability and usability of the content. RuneScape QA serves three key games: RuneScape, Old School RuneScape, and Chronicle: RuneScape Legends.
QA assure that the game runs smoothly and is high quality for the players. They test new updates to make sure that there are no bugs, but are also responsible for maintaining all of the content in the live game through investigating player reports, and submitting feedback for post-release changes based on player suggestions.
|“||Of the handful of bugs that do make it into the game, the QA team have already found several hundreds of them before the content is launched.||”|
— Mod Stacey
The Quality Assurance team tests new content after it has been developed and tries to find errors to help make the game better and easier to use.
Playing the game is only part of what the Quality Assurance team goes through during their part of the development process. Checklists make sure that the new content works right and meshes with the content already in the game. Each of these tests has to be done many different times under a number of different conditions.
Even after all of this testing, opening the content up to millions of players brings out problems that the QA team has missed. Players also work to fix the bugs by sending in feedback and bug reports – it would almost be impossible to fix all of the bugs on release, as there are hundreds of thousands of players for every QA team member.
Players can submit bug reports to the QA team in-game; however, more often than not, the QA team receive bugs that do not have enough information to investigate, due to lack of detail. It is also common for the QA team to receive reports that are not actual bugs, due to it often being player confusion.
Live maintenance is the Quality Assurance's job in which they track what the players are saying about new content by reading the forums and checking bugs sent in by the bug tracker. They alert the developers to the problem and the developers work to fix the bug. Good and bad feedback helps the QA team to learn what the players that play RuneScape enjoy.
The Quality Assurance team also investigate all player bug reports, and submit reports to the development team for fixing. They classify bugs as Critical, Major or Minor which the development team use as an indicator for its urgency, so they can shift workload to ensure fixes are released in good time. The QA team confirm the bug fixes in their development and release build, before confirming it can go live to RuneScape players.
Jagex has several Scrum teams that focus on creating individual, unique pieces of content. The QA team are embedded within the development team, meaning that they work directly next to the developers, designers and artists to deliver content as a small focused team. They are involved in the content from beginning to end, contributing toward the design and development, before focusing on the testing and maintenance of content for years to come. Some QA members are skilled enough in code to create their own content, while being supervised by senior developers.
In 2014, Jagex formed a QA team that handle all of the releases before being handed to RuneScape players. Any content that is required for a release is packaged together into a single build, so that the Release QA team are able to confirm that all of the required content has come together safely, and has not conflicted with other areas of the game. They perform both manual, and automated tests to ensure build stability, and also confirm any bug fixes for existing content that are due to go out into the live game. This team plays a vital role within the development process, and it is this team that allows Scrum teams to focus on the next big thing, without having to worry about re-testing everything in a new build.
In 2013 Jagex introduced a career path for the RuneScape QA team, allowing team members to progress from QA Tester, to Senior and Lead roles. This also included the creation of the Junior role, which is used to hire people that have limited, or no QA experience and are trained on the job. It is common for Jagex to place players into Junior roles as they have no previous QA experience.
|Mod Cocoa||Head of QA|
|Mod Stacey||QA Team Leader|
|Mod Dorando||QA Team Leader|
|Mod Grem||Lead QA Tools Developer|
|Mod James H||Lead QA Analyst|
|Mod TomH||Lead QA Analyst|
|Mod Sarah||Senior QA Analyst|
|Mod Kristy||Senior QA Analyst|
|Mod Ramen||Senior QA Analyst|
|Mod Noldor||Technical QA Engineer|
|Mod Kitty||Technical QA Engineer|
|Mod Dolan||Techincal QA Engineer|
|Mod Chaose||QA Analyst|
|Mod Manti||QA Analyst|
|Mod Harrison||QA Analyst|
|Mod Cuppa||QA Analyst|
|Mod Giara||QA Analyst|
|Mod Curse||QA Analyst|
|Mod Wolf||QA Analyst|
|Mod Shogun||QA Analyst|
|Mod Roq||QA Analyst|
|Mod Jam||Junior QA Analyst|
Quality assurance provides a unique opportunity to make the game better, but it can also slow down the development process severely. If QA sees something that they don't like they will send it back to the developers. The developers will fix it, but that isn't the end of the process. The fixed material will go back the Quality assurance team to be looked at again. Depending on how perfect the team wants the material to be, this process can go on and on.
This process can be seen first hand in a case such as Mobilising Armies. Mobilising Armies was revealed in a behind the scenes (Update:Future PvP Updates) on 15 August 2008 along with other future PVP updates including Stealing creation. In November 2008 Jagex confirmed the release date to be in December. In December Jagex revealed that they would release the minigame in early 2009. In the Update:Behind the Scenes - February (2009) Jagex revealed once again that they were postponing the release date to later in the year. It was then revealed in the forums that it would be sent to QA again as of 5 May 2009. After all the delays, it was eventually released in July of that year.
- ^ RuneShark. Mod Stacey + Mod Cocoa Discuss RuneScape QA Testing - RuneShark's Jagex HQ Visit (YouTube video). 1 May 2012.*