There are many ways to categorize bugs: frequency, severity, module, layer where it occurred, etc. The ones I’ve just mentioned are all useful fields in a bug tracking system. However, it is more interesting to focus on the specific what’s, the kinds of things that can go wrong and should be tested.
Ten types stand out, although they can be sliced up differently, and the list that follows should not be considered to be “a full set”.
- Calculator/math error: This can occur for many reasons. The developer may have misunderstood the calculation or mis-keyed a formula. There could be confusion over the unit of measure or the conversion factor. A set of rules may have been incorrectly interpreted (such as a discount rate at an etailer, or who must pay sales tax). It may also involve localization concerns where commas and periods may have reversed meanings.
- Communications error: A needed endpoint may not be findable or transmitting. This could be a dead link (a URL that used to be good but is no longer), a denial-of-service from a website, database or API that’s up, or just a really slow connection. These are conditional problems that once passed a functional test, but have gone wrong due to age or temporary resource conditions.
- Compilation error: Sometimes, a build does not get built correctly but is being rushed out the door for distribution as a hotfix or patch. Hopefully QA gets a chance to sanity test the build prior to customer delivery!
- Control flow bug: Certain processes must happen in a specific order to function correctly. Imagine the confusion if a user session started without being initialized, or getting logged into a system without having an account. Or to look at a another scenario, imagine that there is a nesting error in the code, or that a BREAK exits the code to the wrong nesting level, as happened with AT&T on January 15, 1990, blocking over 50 million long-distance calls for over 9 hours, losing $60M of service, due to one line of code which caused a cascade effect of reboots through the switches.
- Error-handling bug: Messages should be short, accurate and helpful without exposing data that could be a security risk. “Keyboard not found, Press F1 to continue” is unhelpful.
- Functionality bug: The code doesn’t do what it should (sunny day), or it does do something it shouldn’t (rainy day).
- Load/performance problem: Load is all about behavior as you approach the stress points of specific volumes of activity, which may involve speed, denial of service, or improper or incomplete handling. Performance is more concerned with just the speed, and may involve tracking the times spent on processing (maybe a SQL or API call), transmission, and screen rendering to determine who is taking too long. Perhaps latency is due to a timeout is too long, or a SQL query is poorly constructed.
- Missing commands: Imagine a pop-up warning with the 2 options of Proceed and Delete that you cannot X out of – is it safe to Proceed if you don’t want the deletion to occur? What if you edit a field and “Exit” is the only button (no Save)? How useful is a pop-up with no buttons or X? What about a profile editor with no Save button? These are all missing commands that can easily frustrate users.
- Security bug: Traditionally these are found through penetration testing. For instance, any piece of input user text that may be used in a SQL call should be validated for proper format to ensure that there is no SQL Injection. SQL injection often entails snippets designed to trick the database into erroring or answering with a much wider answer than it should through reserved words or characters or excessive length.
- Syntax/grammar: While often benign, misspelling your company name or making it obvious that work is outsourced to non-native speakers of your language may frighten users about app or website quality in general. This can include not agreeing with the single/plural or specified gender of text that contains a text variable. Even worse, instructional text sometimes becomes risqué or suggests the opposite of what it should (“Press here if you don’t want to save your changes”).
