Context-Driven Testing: iPhone 4S

According to an article by CNET News, users of the iPhone 4S have been rating the battery life of the phone as “subpar”.  Not a good thing, since the phone was just released less than a month ago. Apparently the culprit is a location-based feature that determines when the user has moved to a different time zone and sets the phone’s clock accordingly.

This seems to be a feature that was overlooked during testing of the iPhone 4S. Perhaps it was a feature that was rated as “low risk” and, therefore, due to time constraints, left out of the executed test scenarios all together.  Maybe the feature wasn’t tested at the right point in the development lifecycle.  Whatever the case, this leads us back to questions that have been around for ages.  What do we test?  What don’t we test?  These broad questions lead to more detailed questions. What features have the most visibility to the user and the highest risk of containing defects? What features won’t we test?  Will any features go untested? The answers to these questions are different for each project, client, and product. I believe context must always be taken into consideration.

As a tester, I always ask myself three basic questions on every project I’m on.

  1. What is the estimated timeline for testing on this project?
  2. What features/functionality is part of the critical path of this product?
  3. What tests can I perform to get the most coverage within the given timeline with the tools provided?

Once again, the answers to these questions is always different, but the questions are always good ones to ask.