We’ve been discussing different aspects of developing mobile games and testing various required architectures and systems to build more durable mobile games. The feedback we’ve received has been overwhelmingly positive and we urge you to connect with us since we’re determined to continue providing our knowledge, most effective practices, tips and tricks , and other pertinent information in the making of more mobile-friendly games. Today, let’s examine the way that testing enhances the development process of mobile games in general – and also which are the top crucial indicators to think about.
Moving across the mobile game development chasm
The majority of testing for mobile games by UK game studios is performed manually, and does not provide a reliable method to ensure that everything is covered. Effective mobile game testing must stem from a structured and methodical approach, the using automated testing framework(s) along with seamless integration into your agile processes.
In addition there are many test automation frameworks that were originally designed for native applications (or elements of web apps) which is one reason why the biggest myths for mobile game developers is that testing automation tools do not conform to the requirements of game testing which means that native UI elements such as their IDs, features, such as. can’t be immediately identified. This isn’t the case, and many of these frameworks provide great methods to test games for mobile. For instance, they can be tested using features for image recognition.
When testing is concerned it is easy to assume that the process is to have bugs discovered and documented to ensure that developers can fix these bugs. So, the metrics that are traditionally used to measure software testing such as the amount of bugs discovered and the amount of test cases developed – are excellent, but they don’t show the value to testing. As a result of this, testing isn’t considered to be a successful and efficient effort for companies. But, the instant feedback from gamers via App Markets, and the concern regarding user retention, it has altered the way that serious mobile game developers approach testing. Poor ratings and negative feedback lead to a lower number of downloads, and ultimately leads to the game to become a risky investment.
Can crash reporting aid you? Absolutely however it only gives you with the knowledge that your game on mobile doesn’t function and crashes or users are experiencing another issue or experience other issues. It’s too for you to fix it. Your game is now open to the public, the negative feedback has been made public and fixing the problem won’t necessarily help. In certain cases, poor reviews or reputations can make it impossible to get the next game accepted, or at least not being able to get the game out in front of millions of gamers.
3 Tips for Increasing value in Mobile Game Testing
The added value of testing into the process of development can be additionally seen in its efficiency in speeding up the development process (improved efficiency) and the potential to publish your work earlier (time-to-market). These ‘improvements’ can be directly measured by worth:
Your effectiveness in your QA in identifying and developing to fix the flaws. This is the. top expense driver for mobile application as well as game design. According to the report, direct costs of defects can have an immediate impact on your bottom line including the actual costs of fixing the defect through testing, the cost for fixing or verifying the crash report data from the marketplace, and the loss of customer acquisition costs (CPI) as well as the lost value over the life of the customer (LTV that affects top-line).
It is often easily overlooked as a cost-driver, however, if you put your numbers honestly in the open you’ll see that the late bug-fixing process is consuming many developers’ time and could delay the release. This amounts to loss of revenue and customers.
Costs associated with time-to-market directly affect your bottom line. Time is an essential attribute not just to beat your competition, but also for generating revenues as quickly as is possible. When you begin your game, each day counts because you begin earning revenue sooner. The most important measurements are monthly/daily acquired players (MAU/DAU) and the average income for each (daily actively) user (ARPU ARPDAU, ARPU). Keep in mind that delays in publishing aren’t due to testing if your development process allows for agile testing and development.
Many game designers do not consider the effects of delays. However, honestly it’s quite simple to determine the impact on each day of delays to your bottom line. Manual testing, even for mobile games can slow the launch further. The solution is flexible, automated test-driven development that includes continuous integration with continuous testing, and quality assurance for real-world devices every day during development.
The effectiveness of your Quality Assurance. This is the evident way to affect you bottom line. The most effective way to evaluate the efficiency is to measure the value of manual testing to costs associated with test automation tools and automation costs. While this only provides the cost estimates for financial purposes however, it is essential to look at other factors of the typical approach to test-based mobile gaming, and in the present the majority of the crucial testing and QA work is completed at the end in the process of development. We have discussed the concept of using an agile method for mobile game development and ways to maximize the advantages this approach could bring – and also increase the effectiveness of your QA.
How can you add value to the Games Development Process?
Effectiveness. The easiest method to overcome the problems related to efficiency is to integrate the game development and testing processes into a unified agile process of development. This enables you to automate a huge number of different types of tests, using real mobile devices (all sorts of those) and check code after every build (daily/nightly/weekly builds). The efficacy has three attributes and benefits:
1. Instant feedback on the presence of defects. Mobile development provides an added aspect of automation using real devices that your end users use. From a developer’s perspective the immediate feedback on any defects can boost the productivity by a significant amount, as you have the code you last wrote fresh in your head. Who is able to remember what you had for lunch two weeks ago, vs. today?
2. The effectiveness of defect detection prior to the release can lead to lower failures in the field. The gaming apps are most vulnerable with the most crash rates of all app types. Take an example: A mobile game that has 50k monthly active players submits 2200 crash reports per month. If you’ve got 3 games in the marketplace this is 6,600 crashes each month. If the analysis-fix-verification takes around 1h, the cost is huge. This is not because of the engineering cost however, but due to the expense to the reputation, lost customers and the possibility of losing revenue.
Conducting testing on real mobile devices prior to release could have enhanced the game and helped to make the game more stable. The efficiency of development could have improved by a factor of 2 to 4. Monitoring post-launch is vital, but it is important to not be in a reactive mode when it comes to finding defects as it can cost you lots.
3. Loss of CPI as well as LTV. You are aware of how much each installation will cost you, and you know your LTV that your customers use. If your game is not working after the release, it’s likely that you’ll be unable to recover both CPI as well as the LTV of the user or all users that use the model of the device. It is recommended that you connect your CPI plus LTV for each failure that happens in the field in order to comprehend the consequences of crashes.
Being proactive and using a the true Agile development methodology, with a continuous gaming testing methods using real mobile devices will result in an increase in top line and a significantly increased bottom line aswell happy developers. developer would like to work on the latest features, not fixing existing ones in firefighting mode, based on the crash reports that are a result of the market. Remember! Manual testing isn’t flexible, and is not the answer to these previously mentioned issues. It is not possible to get immediate feedback. If you’re planning to test your game by hand using top 50-100 devices for example, the expense of doing it bi-weekly and even monthly can be expensive and nearly impossible.
Time-to-Market. The shorter time to market can increase your profit margin. The sooner you launch your game, the earlier you begin making money. If you are able to release your brand new game or an old game that has new features monetized two weeks earlier than previously , using agile and automatic mobile tests, you will have 14 days cut off from each cycle of release. If you release 6 times each calendar year (every for 2 months) that’s an average of 84 days you save annually.
The math behind lost revenue is actually quite simple:
Imagine 100,000 average daily users (DAU)
Let’s say $0.05 average daily income per Active User (ARPDAU)
14 Days (x) 6 releases/year (x) 100 000 (x) $0.05 = $420 000/year/game to your total revenue.
If you own five games in your portfolio the total amount would be approximately $2 million from faster time-to-market.
It is possible to achieve this by slashing two weeks from each release cycle. Some games are not directly revenue-generating, therefore substitute ARPDAU with the appropriate value for every mobile user.
Time-to-market can be improved through a unified development and testing. This is the case with every software development process, but for mobile app development it’s the only way to conduct development in a timely manner. There are a lot of different platforms for end-users (e.g. various devices that have different software and hardware configurations) which means that your games need to support an ever-growing amount of complexity in games that have to be tested on the devices for each new release.
You can automate a massive amount of testing and run it on actual devices following every code update or nightly builds. This can reduce your development testing process, speed up defect fixing and lead to a better overall performance.
The speedier time to market helps keep your competition in check. Your competitors won’t have the chance to advertise their products over yours, which is an opportunity when your releases begin to fall off. It is possible to release more often with greater confidence in how good your products.
Productivity. Quality control (QA) is a crucial element of making and maintaining games that are successful on mobile for millions of players. One of the measures during the process of creating games is the QA’s effectiveness and the ability to effectively identify and eliminate issues in games before they reach the hands of gamers. This kind of efficiency of quality assurance is assessed using various metrics like:
Bugs discovered rate – how quickly the QA team discovers bugs in the testing process, per hour spent on testing
Post launch, issue rate is high – how many issues are identified following the go-live release
Rate of satisfaction with the customer For example, complaints to customer support, problems that are reported to app stores, overall rating of the game
The quality of the deliverables and the documenting issues – how easy you can make it for your developers comprehend the issue and implement an appropriate fix based on documents
The number of test cases that are added per day of time
The main challenges that mobile game development and quality assurance are the diversity of devices as well as different operating systems as well as hardware configurations, carrier and OEM customizations. These aspects add to the difficulty of QA and can lead to longer development time (time-to-market) and higher field crash rate (lower levels of customer satisfaction, and consequently which can impact your bottom performance) and more costly customer support costs (more complaints from customers) and a sloppy development teams (dealing with issues with customers, as opposed to. creating innovative features). To improve QA efficiency, teams have to be able to access real devices – exactly the same ones that the end-users utilize.
The developers and QA teams require access to the devices when they have a problem in development. They fix issues after launch or are receiving complaints from customers. The earlier the issue is discovered more affordable is the cost of fixing the issues. To make sure that bugs are found efficiently You can automate the testing and run it on actual devices following each new code check or nightly builds. This will reduce the development testing process, speed up defect fixing and lead to a higher bottom line.
Utilizing a cloud-based platform to develop mobile apps and for QA performance for your entire team is likely to increase significantly: The cost is a fraction of manual testing process, developers are more satisfied (instant feedback on the latest codes and no need to deal buying when purchasing new devices) The product owners are more satisfied (higher customer satisfaction, higher LTV, speedier time-to-market) Customer service is more pleasant (fewer complaints from customers) and, most importantly of all your customers are happier because they don’t get angry because of “not operating on my gadget”.
Conclusion
There are many methods to improve the current processes for developing games on mobile. If you concentrate on the above three points and adhere to the flexible process that includes constant testing, you’ll quickly see advantages and create more durable games. Here’s a checklist for you to take the next step with higher quality:
You might want to automate the majority of testing, and choose one of the top open-source testing frameworks for the most widely used mobile platforms.
Do not be a victim of a lock-in to a vendor by using standard language (Java dotnet, perl, dotnet PHP, php, Python and ruby) There’s a huge pool of developers who use these languages to tap into.
Automate as much as is possible of the testing process to increase the agility of the organization (even assign developers the responsibility of developing automated tests)
Utilize continuous Integration and Continuous Testing. testing is a continuous procedure in agile , and will give you results 24 hours a day, 7 days a week.
Integrate your test management system to increase communication and transparency within the company. Tools should facilitate a smooth flow of collaboration
Instant accessibility. The teams working on development do not have the time to deal with lengthy sales cycles, procurement processes for every device or tool. Select a solution that offers rapid provisioning that does not require waiting for the hardware or procurement, or internal processes.
Last but not least, the most important tool: Tools are important however how you utilize them is what’s more important. Tools can make a difference however, how you utilize them is what will determine the level of success you will achieve.
Happy Mobile Game Testing!
