Optimize your games with Compatibility Testing
What is Compatibility Testing?
With the huge variety of games available and all the different versions of the same game, there are bound to be compatibility issues. Compatibility Testing is a game testing method to ensure that the game runs successfully on different hardware, operating systems, or mobile devices.
Compatibility issues examples:
Android Fragmentation
Nowadays, compatibility testing for mobile games faces big challenges as there are a massive number of different Android OS versions and models in the digital world. There are now over 1,300 Android brands that have produced over 24,000 distinct Android devices. This is called Android Hardware fragmentation.
The image below is taken from OpenSignal’s July 2013 report on Android fragmentation, which indicates the number of different Android devices that downloaded their app in the first half of 2013. We can see that there were already a large number of devices in 2013. Now in 2021, the number can only be much larger.
One of the biggest positives to hardware fragmentation is that it gives consumers more choice when purchasing mobile phones. But for game testers, it is not very good news. Android Fragmentation indicates the need to test an increasing number of devices and versions during the compatibility testing for mobile games.
In this case, efficient and scientific selection of test device models becomes very important as the basis for compatibility testing.
Efficient selection of device models
Although it’s always better to have as many phone models tested as possible, it is not cost-effective to test all the available models in the world. To boost efficiency, we need to determine the test models more objectively and accurately based on user insight and big data.
In WeTest, for example, several different solutions are differentiated with the number of phone models covered.
The TOP100 models cover almost all common Android versions, resolutions ratio, memory, manufacturers, CPUs, and GPUs. The TOP300 models cover about 87% of game users in China. For PUBG mobile, WeTest used the TOP500 models, while the TOP500 models cover more than 90% of game users nationwide. Lastly, the TOP1000 models, powered by the top 1,000 popular phone models worldwide, can represent on average 80% of the global users. This model is often used to effectively test games that aim to expand their target market internationally.
Efficiently test the game scenarios & Find the issues
After determining the test models, what we need to do next is to efficiently test the game scenarios and automatically find the issues.
Automated testing & AI testing are often used to execute the testing and identify issues in specific game scenarios. The method uses game AI training flow, and the weight file will be generated.
First, start the game.
Then, enter the game scenarios automatically through UI-Automator.
The weight file will be used to traverse more game scenarios. During the process, issues can be automatically found.
Finally, the depth test report will be generated. The potential issues that could be found include crash, ANR (Application Not Responding), process exception, black and white screens, blurred screens, etc. Performance data, logs, and screenshots are also used to conduct a more comprehensive analysis.
Real Devices in cloud lab
For the issues found during the testing, real devices are needed to locate the issues more accurately and efficiently. WeTest, for instance, uses thousands of real devices in its cloud lab. Websites and phones are used to easily monitor and control these cloud devices, with low latency (less than 50 milliseconds). It also has real-time log and sound output and uses ADB commands for debugging. With its intelligent scheduling strategy, data center self-recovery, efficient remote operations, and automated testing framework, real device testing can ensure higher stability, availability, confidentiality for the overall solution.
Compatibility optimization effect
After the compatibility testing and optimization, the crash rate can be largely reduced and most of the bugs related to compatibility issues are expected to be fixed.