Indies need someone they can trust for usability and QA testing — How about The Behemoth?
Somewhere in San Diego there exists a couple of buildings. They’re buildings with glass windows and a soft feeling, or so I’ve been told. They’re buildings that are “not too intimidating” when compared to other species in their particular building genus. At least, that’s how The Behemoth President John Baez described them to me at the tail end of a roughly 32-minute interview that took place in Boston last weekend during PAX East. Baez and Level Designer Ryan Horn shared their thoughts on several topics: BattleBlock Theater, what the studio would like to see from the next-gen version of Xbox Live Arcade and the developer’s thoughts on working with Microsoft.
Yes, we managed to cover a lot of ground. This despite the fact that we were sitting comfortably in folding chairs set up in a largely unoccupied space behind the booth over which hung a large arrow bearing a single word: “Behold.” What precisely the attention of PAX attendees was being called to may not have been initially palpable to the first-timers among them, but then again, nor was it to the XBLAFans crew when Horn and Baez — the latter fielding an increasing percentage of the questions we asked the two men — began talking about video game prototypes.
Our attention, as it turned out, was being directed towards those two buildings. Or rather, what goes on inside their walls.
They’re not buildings in which the developer makes games, mind you. They are buildings in which the developer tests games to see if they work. Interestingly, one of the games that has been analyzed there isn’t property of The Behemoth — it’s property of fellow successful indie studio Supergiant Games. And at some point this summer, the iOS version of Bastion will lose the distinction of being the only game from another developer to be put through its paces by The Behemoth. The studio will begin using its pair of non-development buildings to investigate whether or not all manner of foreign games work in the manner that their designers intended them to, and whether or not that’s the way they should work.
The Behemoth will task those working inside the friendlier-than-most-of-their-kind buildings with providing quality assurance (QA) and usability lab services to fellow independent game developers. One indie should help another indie. This type of help, however, will come with a price tag — and not a discounted one.
We’re Going to Need More Prototypes
How does The Behemoth, a video game development studio that seemingly exists to…develop video games, today find itself on the precipice of selling quality assurance and usability services to third parties? Did the studio brass wake up one morning with the sudden thought that it was a good day to start doing so? Not exactly.
Today, three years after The Behemoth originally told the world it would release BattleBlock Theater, it’s all but ready to launch the game on April 3. Long before it reached this final staging ground — long before the studio decided it would stage its forces here, even — came the prototypes. Baez told us about the iterating that occurs behind the scenes, out of sight of the thousands who attend game conventions. There are “dozens of prototypes that we throw away,” he explained, “and they don’t even get to the phase where they’re past the prototype phase. But it’s dozens and dozens of ideas.
“If you’re not a game developer, and you’ve got one idea and that’s it, and you’re done, and it’s the only game you’ll ever make the only game you’ll ever have — those are some long odds right there, man. Those are really long odds, you know?” The Behemoth boss believes that his peers in the indie development studio would do well to constantly be working on as many game ideas as they can. It’s hard to argue with his simple reasoning that by diversifying its portfolio of projects a developer increases the chances that a publisher will come calling for one of them.
He described to us a scenario in which a publisher looking for a racing game sits down with an indie studio, but the hypothetical studio isn’t working on any racing games. They should work on one, said Baez, if that’s what they love doing. But if forced induction systems and internal combustion engines aren’t that hypothetical studio’s hypothetical passion, slapping a racing game together just to get a contract may not be in its best interest. Instead, Baez advises said studio to let a different developer, one that does have an infatuation with racing games, know about the opportunity.
Indies should help indies. Even if this type of help doesn’t come with a price tag.
Or so Baez believes. “That’s how the independents are going to prosper and survive, basically. Survive.”
About Those Two Buildings
“I mean, one of the things we’ve done to kind of contribute to that [survivability] is we built a usability lab for Battleblock Theater, which has gone really, really well. It’s about a year old, and it’s only internal, and now we’ve opened it up. Well, there’s that and a QA department — very small, four people — but they’re very, very good at what they do. And now we’re beginning to open that up to other developers. So Bastion for all of iOS, we tested [it] and certified [it] to make sure that [Supergiant’s] game was good.
“So we’re opening that up to independent developers as a resource so they don’t have to go — I mean it’s not any cheaper than going to a big, gigantic test firm — but you’ll get the absolute attention to detail.”
Can The Behemoth really offer that kind of service? It is, after all, only an indie studio itself. One that’s proven astonishingly efficient at selling copies of and merchandise for Castle Crashers, but it’s hardly in a position to set up a usability lab as ostentatious as this one. And how can it promise “absolute attention to detail” to someone else’s games? Won’t its own games always come first?
Baez had an answer to the latter two questions. “Right now it’s under The Behemoth,” he explained, “but it’s going to be broken off into its own little thing because if you’re a game developer you don’t want to send your game to The Behemoth for testing; it needs to go to a separate thing. And literally they’re in two buildings right now, so it’s just that we haven’t done all of the name and documentation.”
While one could still make a case that that would merely be a bit of subterfuge that would attempt to cover up the reality that a game developer is paying to have an entity owned by a competitor test its game, Baez displayed a calm, confident demeanor during the eight or so minutes he discussed the topic.
So then, was the usability lab and QA facility born when The Behemoth decided to answer an otherwise unheard call for these services from less-well-to-do indie studios? “No,” he told me. It was ‘born out of us not getting the testing we wanted.”
“I mean, one of the things we’ve done to kind of contribute to that is we built a usability lab for BattleBlock Theater, which has gone really, really well. It’s about a year old, and it’s only internal, and now we’ve opened it up. Well, there’s that and a QA department — very small, four people — but they’re very, very good at what they do. And now we’re beginning to open that up to other developers. So Bastion for all of iOS, we tested [it] and certified [it] to make sure that [Supergiant’s] game was good.
So we’re opening that up to independent developers as a resource so they don’t’ have to go — I mean it’s not any cheaper than going to a big, gigantic test firm — but you’ll get the absolute attention to detail.”
Can The Behemoth really offer that kind of service? It is, after all, only an indie studio itself. One that’s proven astonishingly efficient at selling copies of and merchandise for Castle Crashers, but it’s hardly in a position to set up a usability lab as ostentatious as this one. And how can it promise “absolute attention to detail” to someone else’s games? Won’t its own games always come first?
Baez had an answer to the latter two questions. “Right now it’s under The Behemoth,” he explained, “but it’s going to be broken off into its own little thing because if you’re a game developer you don’t want to send your game to The Behemoth for testing; it needs to go to a separate thing. And literally they’re in two buildings right now, so it’s just that we haven’t done all of the name and documentation.”
While one could still make a case that that would merely be a bit of subterfuge that would attempt to cover up the reality that a game developer is paying to have an entity owned by a competitor test its game, Baez displayed a calm, confident demeanor during the eight or so minutes he discussed the topic.
So then, was the usability lab and QA facility born when The Behemoth decided to answer an otherwise unheard call for these services from less-well-to-do indie studios? “No,” he told me. It was ‘born out of us not getting the testing we wanted.”
Requirements — Everybody’s Got ‘Em
“Yeah, so at the very beginning with BattleBlock we were required to use a third-party test vendor,” explained Baez.
Required by Microsoft? By the old devils up in Redmond who find themselves ever taking fire from disgruntled indie studios for their supposedly oppressive XBLA policies?
“Well, by pretty much anybody,” said Baez, coolly dismissing the notion that Microsoft had yet another harsh policy not enforced by its competition. “All of the hardware manufacturers want to know there’s somebody else who’s looked at your game. And then, same with Castle. And then during Castle we started our own QA. Then we kind of transformed that a few years later into what it is now. By this summer we should be able to go to have real clients.”
Baez described the endeavor as “fun.” He said it was something no one else had explored, and therefore something that isn’t available to indie studios. The “big houses” of QA are wearing blinders. Ahead, they see only the path to functionality. To their sides are other possibilities, viable possibilities that offer more than mere functionality. As an independent developer itself, The Behemoth is free of the sort of obfuscations that constrain what larger usability and QA service providers are able to offer.
The Behemoth’s president and co-founder described the services his company will soon sell as “very boutique-y. Very boutique-y.”
A Shared Priority
Developers, he said, have already expressed interest in such a boutique service. I asked if he could name a few of them. His response came quickly: “No, I can’t. But the Supergiant guys, they called us. And we’re like, ‘Yeah, we’ve been planning this, strangely. You should call us.'”
Baez’s friends at Supergiant spent the weekend dealing with the omnipresent mob of gamers, game makers and journalists striving to play the developer’s recently announced Transistor. The Behemoth’s had been so busy with its own crowd at PAX that it hadn’t yet had the chance to go see Transistor. But it didn’t sound as though some future work testing and analyzing the isometric RPG is completely out of the question. Baez is rather fond of Supergiant. He sees some of the same qualities The Behemoth his lab’s sole customer to date.
“Yeah, you know, they have the same mentality as we do. Just attention to detail [is their] number-one priority.”
As far as QA tester and usability analyst priorities go, attention to detail would seem a good one to have at the top of the list.
Photographs by Zac “Scratch” Pratt