Automated testing in game coding

March 10, 2014 at 9:38 am (Computer Science, Games, Programming, Rant, Video games) (, , , , , )


So my musings so far today have circled around the idea of automated testing in game engines. Automated tests are a series of unit or integration tests that can automagically verify whether or not you broke your code.Here is a quick explanation of both types of automated tests:

Unit tests are tests that isolate specific areas of your code and rigorously test individual parts. They typically pretend that the rest of your code doesn’t exist (usually accomplished via Mocking frameworks) and when you think about it, isn’t that how you’d debug a problem anyway? An example would be “Does this function in the class return the values I’m expecting when I give it input?”

Integration tests on the other hand function to test your code “as it is” for lack of a better phrase. It tests how the classes and modules of your code work together and whether or not your “business logic” is correct. An example would be “When 3 players connect to the server, does the database correctly log their actions?”.

Now, in game coding the typical idea is to get the game done ASAP and cut corners wherever possible since time is money. By this, I don’t just mean what big companies such as EA, etc. do to pump out new games every five minutes. It functions for all game development to a degree; if your game takes 10 years to be built, you’re probably going to have an out of date game and you’ll have to re-code at least some of it.

Do automated tests truly add value though? If you are a one-man shop, your argument is probably that you know the code intimately and that you stick to your coding guidelines (Thanks to Ruben for the argument). What happens though if you need to modify a really complicated system 6 months after writing it (while not forsaking the codebase in your day job)? Is booting up the game 40 times to do trial and error worth it more than a 30 second run of your automated test suite?

It most likely depends on your perceived usefulness of the tests. If you are just writing unit tests to verify that setPosition(1,1) actually sets your X and Y positions, then there probably isn’t any advantage. You’re likely to waste time building up a nice collection of tests to verify what you know. If however, you are concerned that your A* algorithm is a bit fragile and you’re worried that the AI would try to do something questionable on the game board, perhaps it might be a good idea while you’re working on the code to have some tests.

Keep in mind though, that the idea is to be able to verify without a doubt that the pieces of your code work 100%. If you don’t write your tests correctly or if you don’t even run them, then you’re only fooling yourself. Perhaps you have a good reason for fooling yourself though; perhaps at parties you impress people by telling them you have automated tests in your game engine code…

 

 

Advertisements

2 Comments

  1. Charles Palmer said,

    It’s super useful. In all product development time is money and the point of automated testing is to save you time and money by preventing the growth of technical debt. Having a well tested codebase means you can have confidence in making changes without introducing errors that aren’t caught by manual testing. Further good tests provide documentation for how a piece of code should and shouldn’t be used.

    It’s arguably more important when there are multiple people checking into a codebase but even so it’s useful when you have to come to refactor code you wrote several years ago. It’s even more important when you have to refactor someone elses code written a decade ago in a bunch of systems that are pretty tightly coupled.

    Unit testing setPosition might seem silly upfront but it prevents non-obvious things going wrong (e.g. a regression caused by auto-merging on an integrate). Further the tests are trivial to write… so why not? It’ll be the non-trivial cases that suck up time and therefore money not the trivial ones.

    Unit testing definitely has an upfront cost which you need to be aware of but it amortizes over time particularly in game engine code that will be sticking around.

    Basically unless you are making something small you don’t intend to revisit (e.g. throwing up a prototype) then it pays off.

    • jestermax said,

      I very much agree with all your points. The multi-user checkins aspect was a gaping hole in what I originally wrote; i ended up focusing more on the one-man army that throws out maybe one game on an App store ever 3-5 years.
      Unit testing setPosition is something I would definitely do in a professional workplace and to be honest, with the generally high amount of sketchy hacks/optimizations used in game dev, it would be even more required for that purpose. That being said, I probably wouldn’t put it on equal grounds to unit testing a complex piece of code if I only had X units of time to devote to automated testing. So the overview is “Automated testing is good for everything, but if you can’t use it for everything then pick the areas most prone to breaking”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: