As Quality Assurance Manager at Side Montreal, Deanne Curtis has helped bring many games to the world. Over the course of her career, she’s been privy to the workflows of many partners, and a commonality has stood out: how QA is typically treated as the final step in the process, rather than as an essential element throughout. The following content was adapted from Deanne’s presentation at MEGAMIGS 2022, in Montreal, Canada.
QA isn’t just the final step
Often seen as a necessary evil on the road to certification, QA is frequently eschewed in the planning of the production pipeline. Managers understand that QA must happen, but it usually occurs as the final step before release. However, Deanne contends that by actively including QA early in the planning stage, the result is a more efficient development cycle and a higher-quality game upon release.
When examining this phenomenon, Deanne has seen that it starts with the general perception that QA staff aren’t viewed as experts, but rather generalists, because they touch on all aspects of a project. It’s seen as a job that anyone can do, and therefore is staffed by entry-level employees who don’t last long enough to develop real expertise. Even further, QA is dismissed in many cases because it isn’t seen as being creative, only existing to criticize the work of Development.
It's this negative perception that leads to QA being left out of production planning from the start. With proper preparation, QA can be an ongoing guide, exposing errors early so they can be fixed. This saves time in the long run and can avoid missing delivery dates just before the scheduled release.
Including QA early and often
For most game developers and publishers, certification is the goal, as no game gets released without it. Certification is compliance with the target platform’s best practices, and there is a checklist of requirements that must be completed to achieve it. However, without proper planning, there can be a mad rush right at the end of development to make sure all the boxes are checked. This makes for sloppy procedures, stressed teams and reduced quality of the final game.
Treating certification as an afterthought introduces unnecessary risk and instability, leading to poor utilization of the QA team (meaning wasted time and higher costs), less knowledge-sharing across departments, and a missed opportunity for the LiveOps team to be staffed by a QA team who know the entire game’s depth and breadth. This would make Player Support operations stronger too.
Deanne maintains that early and active QA involvement makes certification better. How? The value of true QA inclusion is manifold:
- Best practices enforced at all stages
- Less downtime: more stable builds
- More efficient review processes
- Better live visibility of quality for producers
- Lower risk throughout the development cycle
- Lower staffing costs due to knowledge and talent retention
- Better quality of life for everyone due to less crunch
Best practices
So, what’s the best way to implement QA involvement? It begins with making QA a part of the core team from the beginning of planning. QA ought to have visibility throughout the entire process, as it helps identify small issues before they become large ones and foresees potential bottlenecks farther down the line.
Understanding QA team structure is important. At the start of a project, Deanne meets with the client and strategizes with them. She gives suggestions for the best QA team model based on their needs and selects testers whose individual expertise best suits the client.
QA team structures
There are two basic team structures used. Monolithic teams utilize many testers who are not necessarily specialized in any given area. Often in these cases, projects are run using the “waterfall” method (linear progression throughout a project, where QA is the last step). This kind of team structure favors quantity of testers over quality, which is not always desired.
On the other hand, a smaller number of specialized testers can be deployed based on the individual project’s needs. For example, some testers might specialize in Software QA, others in Dev QA. These projects are run using the “agile” method, which enables testers to be fully embedded into the client’s team, ensuring the best results and utilizing each skilled team member to their fullest. This framework enables Side and the client to collaborate and communicate more efficiently overall.
Often our specialized testers end up fully embedded within the client’s Dev team, working directly with the client’s production team, developers, and internal QA team and leads. These testers range from junior to intermediate, and some essentially work as devs themselves, if they’re familiar with different coding languages.
Our Quality Assurance team at Side Montreal now has four fully embedded teams with clients. It began as a prototype for one client with this embedded team structure; and because they were happy with the results, they onboarded more teams, and this model is now being rolled out to other clients as well.
Once a team structure is decided, the team should be maintained for future projects. Teams will always benefit from ongoing institutional knowledge with the potential for mentorship. Learning from previous development campaigns is highly valuable and can lead to more efficient processes for all involved.
Third-party concerns
Sometimes, hiring an external support vendor is the best solution. These partners can provide a great deal of flexibility, expertise, and reliability. Letting a third party handle elements of development can free up time for other teams to focus on what they do best. Deciding on a partner involves asking the right questions: What are the specific needs of this project? Does my team have the knowledge required, or is a specialist needed? Will we be able to provide the necessary Live Ops support upon release?
Once these attributes are determined, due diligence must be applied when searching for the best service vendor, as not all are created equal. Once hired, it’s important to integrate an external team into your processes with consideration. The points of contact must be clear, so communication flows through the proper channels. Full access to the guidance documents must be given and the scope of the project should be defined from the outset. The Project Manager can always redirect efforts when and if things fluctuate.
The importance of culture fit
Finally, it’s important for the entire project that QA meshes with the rest of the team. How do the team leaders ensure a culture fit? They must speak of QA in ways that prove they aren’t outsiders, but rather valuable components of the overall process. The structure of the project itself should be welcoming to onboarding.
For some, this may seem like a radical change to tried-and-true processes, but once implemented correctly, the value of QA's early inclusion will prove itself in the quality of the final product. Even better, QA involvement throughout the development pipeline can expose inefficiencies and promote streamlining, leading to a consistent, well-regulated process that still allows for ad hoc changes when necessary.