Well, even if you’re not, have you ever asked a developer how their release deployment was going? You can almost hear a developer’s eyebrows twitching when they mention it. The last challenge. The final gatekeeper. That ‘evil’ bunch of ne’er do wells that seem to take delight in breaking code, and spoiling a developer’s hard work. Brace yourselves, QA is coming!
Such antipathy. Such apprehension. But why? What if I told you QA was here to help?
What most developers fail to understand is that QA may be the MOST important process in your development cycles. Take a moment and read that sentence again. Say it to yourself. Memorise it. Allow yourself to believe in it. Why? Because it doesn’t matter how awesome your idea is, and it doesn’t matter how slick your design is — if you release something that’s functionally broken (or simply doesn’t flow) out into the wild, your customers won’t be impressed, they’ll be pissed, then you’re almost certain to never see them again. AND. Just to be crystal clear: This goes into double-time for the mobile market.
It should be self-evident that mobile testing is a key step in the mobile app development cycle. Fortunately, in many respects mobile testing isn’t all that different from other kinds of testing. Much of what has been learned in non-mobile testing can be easily applied to mobile testing, — but — keep in mind that testing on mobile isn’t exactly the same, and developing for mobile does indeed present some unique challenges. Sure. Developing for Mac & Windows laptops/desktops does have a multitude of different resolutions, browsers and operating systems that you need to cater to, but those are easily dwarfed when compared to the sheer variety of iOS and Android devices; over 24, 000 distinct devices (as of August 2015) in the Android marketplace alone!
Emulators and simulators obviously have a very important place in the overall QA process. They’re simple. They’re straight-forward. They’re great for early-stage testing. They allow you to develop at your desk, and they’re great for proving that the basic functionality is working. However, emulators and simulators aren’t real devices. They don’t perfectly replicate the native environments software and hardware that run on mobile devices. They also really don’t deal with a multitude of real-world scenarios like GPS, carrier differences, connectivity issues, battery drain, storage limitations, call & SMS interruptions, camera functionality, and more.
Given the sheer volume of devices available, it can be a little daunting to test your app on every device and OS combination, not to mention the number of different configurations each device can have. In fact, it’s not really possible to check your app in every combination. In addition, maintaining a device lab in-house can be an expensive, inefficient, distracting and labour-intensive venture. So. The best approach here is to use a cloud-based subscription service like App Insight that allows you to test on real mobile devices housed in as many locations as you need.
So your app works. Great. That’s the first half done. But, do your users actually like using it? Do you know? Have you even asked them? If not, then it’s time for the second half. User Experience Testing has grown beyond its predecessor: User Acceptance Testing. In today’s mobile market it’s no longer sufficient to only check whether a new feature is just ‘acceptable’ to an end user, they have to LOVE it. Why? Simply put: just because a new feature works, doesn’t necessarily mean that it works as well as it could. So, the approach now is to include the end user, which allows you to check whether the user experience that was intended by the designer does in-fact match it’s final implementation.
Whilst performance is certainly a component of User Experience, because its such an important metric, it warrants mentioning on its own. So, the simplest rule in performance testing is this: Make sure your app performs WELL. That’s it. No ifs. No buts. Very simple. But so hard to get right. Because if you’re adding in new functionality, you need to make certain that you haven’t added any delays in the overall performance of the flow that’s being changed. This is especially true in mobile app development, where your core monitor of good performance is the overall speed and responsiveness of the app when a new feature is added. Using App Insight will allow you to find those performance bottlenecks!
So your app works at your desk. Fantastic. But does it work down the road? Does it work overseas? How about offline? As we all know there’s a multitude of various carriers that provide different connectivity options all over the planet. All these carriers have different network technologies, with differing connection speeds and qualities based on your geographical location. In addition, an ever increasing number of mobile apps utility location data and GPS integration, and thus are dependant on their location or their distance to other locations.
Successful mobile testing needs to utilise efficient location simulation by using real devices on real networks. As a bare minimum, you should try to test your mobile app on at least a few of the big carriers in your target markets – get a feel for how they behave – see where you might make it work better. Additionally, you should make sure you test what happens when your connection is lost, or if it suddenly shifts from 4G to 1G.
There’s a simple fact. You’re only human. There are only so many people you know. There are only so many hours in the day. So, how are you going to manage to do any development if you’re having to do all this testing? Well, that’s where automation comes in! The reality is that a lot of QA CAN be automated these days. Unit testing, functional testing, load testing, performance testing, smoke testing, regression testing. All of this can be fully automated without you having to do much at all, — but — not everything.
Unfortunately, there are still some things that computers can’t do. For instance, a computer won’t easily be able to recognise a discrepancy between the designers intended User Experience and the final experience built by a developer. Nor can computers distinguish whether gestures and sounds are synchronised. So, by all means, go ahead and save yourself as much time as you can through automation. Just don’t forget some tests still need to be done manually by a human being.
So you’ve automated a good portion of your testing? Great. Guess what? You can now set up some real-time alerts as an early warning system! Since different devices on different networks behaving differently at different times of the day — the only way to really discover problems is with real-device monitoring. Good mobile monitoring is based on the constant execution of key application functional across devices and networks.
Using App Insight will allow you monitor your app, so, in the case, the performance of your app drops below minimum performance thresholds, your development or operations teams will be able to proactively discover and fix the issue before your end users are impacted.
Still feeling that antipathy and apprehension? No? Good.
As you can see, QA doesn’t just exist to make things difficult. It isn’t here to wreck your day, it’s here to save it! Like two sides of the same coin, QA testers are the yin to a Developers yang. Inseparable by their very nature. What they do makes each other better. So, although QA isn’t particularly glamorous work, it is important. In fact, it’s critical if you want your application to survive out there in the wild.