Mock reviews and game evaluations: what are they, and why are they useful for devs?
We speak to industry experts about the insights mock reviews and evaluations can provide ahead of a game launch
Mock reviews are a widespread practice within the games industry, yet due to the secretive nature of the work involved, they remain an under-discussed branch of the feedback process – particularly in comparison to fields such as QA and player testing. With most mock reviews and game evaluations written under heavy NDA, it's perhaps unsurprising that we hear little about this line of work.
With the help of experts in publishing, PR and game consultancy – combined with my own knowledge as a freelance consultant and mock reviewer – this article aims to demystify the mock review process, and explore how game evaluations provide a useful form of feedback at different stages of the development process.
What is a mock review?
In its most basic form, a mock review resembles the sort of review you would see published by games media. It's a written essay that discusses the strengths and weaknesses of a game, intended to mirror the external critique that a game would face at launch. Mock reviews are typically commissioned by publishers or marketing agencies, and often written by people with a background in games journalism.
"The idea is that publishers really want to gauge the potential success of their title and what a critical landscape will look like," says Sterling McGarvey, head of consulting at game consultancy firm Hit Detection. "Oftentimes, the goal is not to be broadsided by surprise reviews."
"Oftentimes, the goal is not to be broadsided by surprise reviews"Sterling McGarvey
Alongside a written article, clients will ask mock reviewers to provide a score, or an estimate of how the game would rank on Metacritic. McGarvey notes that publishers now increasingly value community sentiment as a way of gauging a game's success at launch, but as a forecasting metric, "Metacritic is easier and neater for all parties involved."
Depending on the project, reviewers could be required to give a general overview of their impressions of a game – as if they were publishing a real review – or be asked to focus their feedback on specific areas. This could be something such as a potential sensitivity issue, the game's narrative, or a particular set of mechanics.
"We pull together key messaging points when we're starting a campaign… and a lot of this is [about] how these key messaging points are coming across," says Rebecca Attard, head of global integrated communications at Splash Damage. "If something we really want to lead on is gunplay mechanics, we will focus a lot more on 'How do you find the gunplay mechanics, and what kind of things would you recommend in terms of players creating their build-outs?'"
Mock reviews, game evaluations, and when they happen in development
Mock reviews, game evaluations, and when they happen in development
Although often used interchangeably, the terms 'mock review' and 'game evaluation' describe two slightly different forms of feedback. A mock review is typically a straightforward written review with a score.
"I have done only one pure mock review thus far in my career as a consultant," says PR consultant at Future Friends Games Natalie Flores, who has also conducted game evaluations for consultancy firms Magid and Hit Detection. "I've mostly done large reports spanning over 30 pages, amassed over the course of multiple surveys, roundtables, and many hours of writing; mock reviews have been a very small part of them."
These larger reports – in which mock reviews form only one section – are often termed evaluations. These can contain varying forms of feedback, such as breakout paragraphs or questionnaire results. The other key difference between mock reviews and game evaluations is when they take place during the development process.
"The term mock review is used as a blanket term," says McGarvey. "I look at a mock review as something that comes late in the development process, where perhaps there's not necessarily room to make changes, whereas a game evaluation can come at any stage of the development process. If [Hit Detection] were to evaluate a game, we can look at something as early as concept and provide some feedback just based on the collective expertise of the team. We look at what the landscape [might] look like a couple of years down the line and extrapolate.
"A lot of this is [about] how these key messaging points are coming across"Rebecca Attard
"Other times, we will look at something that's just barely playable, alongside design documents and supporting materials… The idea being that we would look at the vision of the game, what we're playing, the scope of it, the development time they have remaining – and think 'If you're able to hit these elements of your vision, this is where we see you potentially landing with this'. It's just based on when the client feels it's appropriate, and they want to get some feedback on their game."
Though it may sound counterintuitive to commission a mock review close to release – when further changes cannot be made – these late-stage reviews still have their uses.
"There's two things that we're trying to do here: manage the messaging, [and] make sure that we know what to lean into, what to lean away from," says Lewis Denby, founder and managing director of indie marketing agency Game If You Are. "Also, you can use them to manage client expectations."
So in essence, mock reviews provide a final heads-up on what critics will say about a game upon release – allowing for last-minute changes to marketing campaigns – and give publishers and PR agencies the opportunity to manage internal expectations of how a game will perform.
Mock reviews can be carried out internally: sometimes by a publisher's communications department, or even a dedicated in-house evaluations team. Yet alongside these internal reviews, publishers will often look to consultancies or freelance consultants to gain an outside perspective.
- Not just for big publishers
Mock reviews and game evaluations can be commissioned by clients of all sizes. McGarvey says that while his work at Hit Detection used to be "heavily AAA," the consultancy has now evolved to "occupy a variety of niches," also including smaller budget AA titles ("your $30 games," he says) as well as indies.
"There's two things that we're trying to do: manage the messaging, [and] make sure that we know what to lean into, what to lean away from"Lewis Denby
Denby says that for smaller teams, commissioning feedback is a matter of finding "the most valuable thing you can spend your limited budget on to help drive you towards success."
Commissioning a lengthy report might not be a sensible way to spend this budget, but asking evaluators to write about first impressions can be more helpful. This could involve giving the evaluators a trailer, media pack, screenshots, and asking them to play a 30-minute build.
"That can be so informative for indie developers, because a lot of the time these are things that they're so close to, they haven't realised that in the first five minutes of their game, there's a giant difficulty spike right off the bat," says Denby. "Or there might be something about the way they're talking about the game, or presenting the game, that's stopping people from even wanting to load up the demo in the first place. And that's often the biggest hurdle for small studios and solo devs – just getting that foot in the door."
Why are mock reviews and game evaluations useful?
As previously discussed, publishers typically commission mock reviews to gain an understanding of the critical reception a game will receive upon release, while evaluations highlight problem areas that can be addressed before launch. Developers can also become very close to their projects, sometimes making it difficult for them to impartially critique their own work. In these cases, mock reviews and evaluations are able to provide a more 'detached' form of feedback.
"It's very easy to not be able to see the wood from the trees, because you're in it every day," explains Denby. "We sometimes speak to indie developers who say 'We're really confident that we know how people respond to this game because we've taken it to shows and got feedback from people playing it.' And what I always say to that is… people don't want to be mean to your face. And actually, the feedback that's really valuable is that from people who you don't know, and have no vested interest whatsoever in your feelings."
Evaluators can also provide guidance on how to present games to the media at launch. Attard explains that Splash Damage often equips mock reviewers with guides and documentation of the sort that would be given to critics upon release. Mock reviewers can give feedback on these guides, identifying places where journalists and influencers could potentially become confused.
"I've also used mock reviewers early in the cycle when we're doing things like events, or a big press beat, to look at the demo we're putting out and look at the presentation around it to ensure that the way we're talking about this game – based on what they're playing, and what we're saying – is really clear," Attard adds.
Developers are often aware of the problem areas within their own games, but may need further evidence to back up any proposed changes. For this reason, game evaluations can be useful for internal politicking.
"Sometimes, a developer will ask for something from the higher-ups and be unable to get that thing – say, for example, budget for a particular feature or segment that they know will improve the game," says Flores. "A developer can consult outside experts like mock reviewers to see if the experts will agree with them… The developer can then take the expert evaluation to the higher-up, as they will now have a stronger argument they can present when asking for what they need."
- The difference in feedback compared to QA and playtests
Developers receive feedback on their games from a variety of sources: QA, consumer playtests, and influencer playtests being just some examples. All of these forms of feedback are valuable, and in addition to these, game evaluators can bring a different perspective to the table.
"When you want to bring in game evaluators, you're looking for a perspective that understands media, that understands consumers and community to some degree," McGarvey explains. "It's on-the-job training that you bring, there's a certain degree of intangibles from your experience reviewing games."
Compared to QA testers – who do invaluable work highlighting specific bugs and technical problems – the job of a mock reviewer is more about critical analysis.
"What we're really trying to understand from a mock review is – what is going to be the reaction of the media when they play this?" Denby says. "What sort of emotions or thoughts or feelings does it conjure up? What sort of likelihood of coverage is there? All these sorts of things can be really relevant."
Attard notes that mock reviewers will often look at games in a more "impartial" way than consumers do in playtests.
"Users tend to get hooked up on little niggles that they might not necessarily like. Whereas a mock reviewer can give us a much broader view of everything, and in a much more considered way, written up into a report that isn't just raw feedback."
- The importance of using a wide range of reviewers
In most cases, publishers will commission mock reviews from several critics at once. There can also be several rounds of mock reviews or evaluations on a single project, taking place at different stages of the development process.
"It's definitely good to keep your list of mock reviewers fresh, to bring in other perspectives as well"Rebecca Attard
The reason for this is to gather a broad range of perspectives. Attard gives the example of a strategy game: for this, she would likely bring in an experienced strategy gamer, alongside a reviewer who's more of a generalist, in order to gain two different perspectives on the game. She adds that Splash Damage typically brings in both English language speakers and non-English language speakers, as this gives an understanding of different markets.
"We tend to do at least four different mock reviews from different markets and different individual journalists, because it is so important to not just focus on one."
"It's definitely good to keep your list of mock reviewers fresh, to bring in other perspectives as well," she adds. "It's also a generational thing – there's certain things that the older generation might look for that the younger generation don't, and that gives you a fresh take on it with the younger generation."
How do I become a mock reviewer or game evaluator?
Most mock reviewers have a background in games journalism, and transition to evaluation work once they have significant experience writing reviews. The first step towards becoming a game evaluator, therefore, is to build up a portfolio of work that shows you can critically analyse a game and clearly explain your thoughts.
"I would recommend honing your criticism skills," says Flores. "Write your thoughts down after you finish a game, make a coherent argument about your perspective. Know how to relay this verbally, too – you'll be doing plenty of talking and responding to questions from the developer with the more involved mock review assignments out there."
"Networking is, at its best, genuine relationship building, so don't count yourself out if it's something you're still working on"Natalie Flores
Writing reviews for your own blog is a useful way to refine your skills and demonstrate an interest in games journalism but, as Denby notes, you will likely need experience writing for professional outlets before you can take on mock review work.
"We need to see that you have gone through that process – working under an editor, having professionally published content – because we need to understand that you have created content of the sort that we want to understand."
Alongside gaining experience as a professional reviewer, it's useful to build up your knowledge of the game development process. It's important that you are able to look past bugs to understand the developer's vision. Equally, when evaluating a game later in the development process, your suggestions need to be realistic for the project's remaining development timeframe.
- Be a generalist, but have a speciality
Although it sounds paradoxical, being a mock reviewer requires both a general understanding of the gaming landscape, and the ability to specialise. A broad knowledge of gaming news, industry trends, and community sentiment is useful when giving feedback to clients, allowing you to highlight potential marketing opportunities or risks. Being a generalist also allows you to work on a wide range of projects. Yet it's also helpful to be known as an expert in a couple of areas. Evaluating a game in a genre you know well will allow you to make comparisons to similar games in the market, providing clients with useful context.
If there is a specific game design element that you are particularly good at analysing – such as UI/UX design or onboarding – you can also make this your niche. Publishers may seek you out if you are known to be an expert in a specific genre, or if you can provide useful insights into a particular aspect of a game's design.
"If you've got a niche that's outside of the usual 'this game is good or bad,' take that and make it your superpower," advises McGarvey.
Some evaluators specialise in identifying best practices in areas such as accessibility, or diversity and inclusion.
"Accessibility is an area where there's a lot of people doing great advocacy and consultancy work," adds McGarvey. "The work that those organisations are doing helps everybody as a whole… A rising tide raises all ships, and these advancements, in my opinion, are making games better."
- Promote yourself, and get networking
Attracting mock review work is also a matter of making yourself visible, and building a reputation for producing high-quality reviews. As a journalist, you may well have a platform on social media, so it's worth leveraging this to display your work. A portfolio website allows publishers to check your past reviews, and see if you would be a good match for their project.
"Being able to divorce 'what do I think about this game' from 'what will people think about this game' is a really key component"Sterling McGarvey
PR firms and consulting agencies often do outreach to reviewers to ask for their help with mock reviews. Equally, you can be proactive by emailing PR agencies, and letting them know that you're available for work. Over time, opportunities may come your way through word-of-mouth recommendations, and through networking.
"Networking is, at its best, just genuine relationship-building, so don't count yourself out if it's something you're still working on," says Flores. "Just be polite and kind; have some convictions and don't be afraid to voice them constructively on social media… and become someone known for being reliable and doing good work."
- What makes a good mock reviewer or evaluator?
First and foremost, a good evaluator is able to look at an early game build and give useful feedback without being distracted by bugs which are likely to be fixed during the development process. Giving feedback on a game's flow or difficulty curve will be useful for developers; spending three paragraphs describing a bug is unlikely to be helpful.
An ability to detach yourself from your personal tastes and present a wider view of the critical landscape can also be beneficial. McGarvey describes this as the ability to externalise.
"Being able to divorce 'what do I think about this game' from 'what will people think about this game' is a really key component," he adds.
Denby mentions that it's helpful when evaluators simply write a review as they would for a publication, rather than trying to pre-empt what the developers might be looking for: "Just review it, imagine that this is what you were putting out there into the world, because that is ultimately what we're trying to gauge when we distribute this code – what's likely to happen."
Finally, for Attard, one of the most important qualities an evaluator can have is the ability to communicate effectively and forge positive working relationships with publishers.
"Some [evaluators] come back with things that aren't necessarily that positive, and [publishers] have to be able to accept what they're saying and not get frustrated or annoyed at them," Attard says. "That's where having a good relationship – where someone is able to really communicate with you about the process, or the reasons why they're saying these things – is key."