Mobile App Testing – Best Practices

Mobile App Testing Best Practices

When testing any mobile app, you should always start by assessing the User Interface (UI), as this is the key element that the user will interact with and critical to the overall user experience. But before you start testing the app’s UI, you should first look at some similar apps to get a sense of the key elements you should be looking out for, to use as the benchmarks to judge the app’s UI against, when testing.

These key elements typically include the overall theme of the app – as a rule of thumb it should be similar to the OS’s native apps’ theme, the design of the app’s icons, the progress indicator, the menus and app’s overall responsiveness. If the app is supported on different devices having different screen resolutions, it is best to start the app’s testing with the device having the smallest screen size and then progress to the larger screen sizes, in order to spot any UI layout problems.

Regression testing should also be done after any UI layout changes to ensure that the app looks appealing on all supported screen sizes (both small and large). In case the app supports changes in screen orientation, then UI testing should be done in both portrait and landscape modes, by rotating all the pages of the app to ensure that the display remains consistent and the app does not crash on changing screen orientation.

When testing cross platform apps and/or apps supporting multiple devices and screen resolutions, it is a good practice to test the app on all the targeted OS’ and devices. If however you don’t have access to the different devices needed for testing, you can use an emulator to test the app. Keep in mind though that the overall performance of the app on an emulator might not be the same as its performance on the actual device, so we recommend using a combination of device and emulator based testing in such cases.

The type of generic mobile app testing that you should conduct on all target devices includes:

• Application installation / uninstallation
• Application launching
• Build upgrading
• Receiving text messages, phone calls & social media notifications with the app running in foreground / background

All of the above are practical scenarios from the user’s point of view and must be considered during testing in order to access their impact on the app’s functionality and overall responsiveness during these situations. Depending on the nature of the app developed, you may need to perform additional tests on the app, to ensure that all functionality/features work perfectly. These tests can include:

• Location Based Testing – for apps that offer/use location based services
• Network Connection Switching Impacts (WIFI to EDGE, EDGE to WIFI, WIFI to WIFI) – for apps that utilize a network connection
• Retina / Non-Retina Image Testing – for apps that support both retina & non retina displays
• Memory Leakage Testing – to ensure that the app does not suffer from any programming issues that impact its responsiveness
• Battery Consumption Testing – to ensure that the app is not draining too much of the device’s power
• Security Testing (for enterprise mobile applications) – to ensure that data communication is going through SSL/TLS and data encryption is being done at the application level, etc.

Automated testing can also be conducted on the app if you want to save time, via test automation frameworks such as Robotium. These tools allow you to write automated test cases for function, system and acceptance test scenarios spanning multiple Android activities, and test them on various device emulators.

Mobile Game Testing – Best Practices

For mobile games specifically, the QA and testing involved is slightly different and also more extensive as you need to test a number of additional elements here such as: the UI & functionality, graphics performance, user experience, multiplayer experience (if applicable), cross platform performance, battery consumption and lastly memory usage/leakage.

UI & Functionality Based Testing

For mobile games you need to look at both the UI & functionality a lot more closely, since for mobile games, these 2 elements ARE its key selling points i.e. visual appeal and gameplay. Thus the aspects of the game which should be examined in detail during testing include:

• The Title Screen – is it well designed
• Menu functions – are they working? easy to navigate?
• Game flow & gameplay rules – game flow should be progressive and the gameplay should not be too difficult or too simple
• Character Model & World / levels / scene design – are they well designed & polished
• Opponents in the game – not too easy nor too difficult
• Collectibles – are there enough collectibles in the game?Is there a good variety of collectibles?
• Scoring mechanism – should be accurate and simple
• Social Network Interaction – should be perfectly functional and easy to use, so users can share their experience and/or scores with their friends

User Experience and Graphics Based Testing

Equally important for a mobile game are its User Experience and Graphics, since these 2 elements also hold the key to its commercial success. To test these two elements the following factors must be examined in detail.

• The game’s Graphics & UI – does the game display properly on all target devices & OS (in case of cross platform games). Do the UI layout & its elements (toolbars, buttons, icons, etc.) display correctly on all devices
• Usability of Buttons / Navigational Flow – are all the buttons working correctly. Is the navigation logical?
• Grammatical Text & Fonts – Is the text used within the game grammatically correct throughout? Is the font used legible?
• Special Effects & Animations – Do all animations run smoothly? You need to test for clipping to ensure that two or more polygons do not overlap and cancel each other out and test that all required overlaps (semi-transparent object placed over solid color objects/backgrounds) are appropriate in size, placement and purpose.
• Sound Effects – Do all sound effects play correctly, without distortion & at the right moment? You should to listen closely to all the sound effects to ensure that their start point and length is appropriate.
• Collision between Objects – do all collisions in the game take place correctly? There should be no incorrect or inappropriate collisions, so the gameplay looks realistic. For this you will need to move the character through all objects to observe the consequences (collisions, event triggers or object triggers).
• Screen Orientation – If screen orientation is supported, does the game display correctly in both portrait and landscape modes?
• Screen Resolution – if multiple screen resolutions are supported, the game should be tested on all the supported resolutions to ensure that the graphics do not get altered

Online Multiplayer Games Testing

If the game supports online multiplayer gameplay, you also need to test this capability, to ensure that: the game allows internet connectivity on both devices, the game state is synchronized on both devices, both players have a similar distribution of game resources and there is no lag in game response time for both players. You should also check to see if the game continues to run if a player exits from the server (if applicable). All of these elements are vital for multiplayer gameplay and any issues in any of these areas will severely impact the game’s experience, resulting in negative feedback and uninstalls by users.

Cross Platform Testing

If your mobile app or game is cross platform you should test it on all its supported platforms and on as many different physical devices as possible (from the list of targeted devices). However given the investment involved in physical hardware it may not be possible to test the game on all supported devices physically. So you can use a workaround solution like DeviceAnywhere, Testdroid and Perfecto Mobile, which are cloud based services that provide the ability to remotely control real devices in order to test mobile apps &games, without incurring any hardware costs.

Memory Leakage Testing

Since all devices and operating systems have varying memory limits ( for e.g.the iPhone 3G has 128 MB, the Samsung GalaxyS4 has 8 GB, etc.), thus it is vital that you run a memory leakage test on the app or game (which tests for memory allocation and de-allocation on different instances in the game), as incorrect management of memory allocations in the game will severely impact the its performance, by reducing the amount of system memory available on the device. In the worst case, it can result in the app/game crashing or causing the device to slow down.

Battery Consumption Testing

Certain apps/games also drain the mobile device of an excessive amount of battery. This can be because the app continues to run even when not in active use or because it starts automatically when the device is powered on. This sort of battery drain is negatively viewed by users and results in users’ uninstalling the app and giving poor ratings. So it’s essential you conduct a battery drain test on the app/game as well, to ensure that it works optimally and does not drain the user’s device battery.

Final Words

Given the stiff competition in the mobile apps market and variety of apps & games available for users to download, it is vital that you thoroughly test each and every aspect of the app / game before releasing it into the market, to ensure that the product meets the quality standard demanded by users. For if the app / game is released in the market with potential bugs, the users who download it will simply uninstall it when they find the app is not up to their standard. They may also leave negative feedback for the app on the app store, which will drive other potential downloaders away.

As a premier software development company, Folio3 offers Software Testing and Quality Assurance as an independent service to our clients. We specialize in manual & automated mobile app testing, enterprise application testing and website and web applications testing. Our software quality & services include Functionality Testing, Performance Testing, Unit Testing, Load testing, Bug Tracking & Reporting services (among others).

For more details about our Software Testing and Quality Assurance services please contact us.