Contact Us

Tag: testing

Why Does Performance Testing Matter for the Web or App Success?

Posted on by Pavel Tantsiura
In this issue:

Previously we have covered the topic of Quality Assurance in general. Since we are definitely not fans of the thing commonly referred to as “bliss” — now it is time to get a little deeper into the specifics. This time — we are going to explain the importance and certain nuances of performance testing and also explore certain challenges that it helps to solve.

There is nothing more important in the application than its smooth, stable functional operation and ability to maintain big and bigger than big workloads. These are the foundational parts of any website or application essential to its success. However, they are rather sensible and need a lot of care in order to carry on. That is what performance testing is for.

What is it? PT (short for “performance testing”) is a type of testing that measures, validates, and verifies the operational capabilities of an application or a website. It consists of a wide variety of methods designed to monitor and determine the quality and capabilities of certain aspects of the system’s operation. This reveals how the system acts in various situations.

Performance testing is commonly considered as one of the central parts of software testing routine as it directly deals with the software capabilities to do what it supposed to do.

Why do you need performance testing?

While it might seem like no big deal but PT is rather complicated in its nature. The pieces of it are relatively simple — but the overall procedure needs to be thought-through step by step in order to gain maximum effectiveness. The first and foremost thing the tester needs to do is to define a strategy for testing routines. If not — then the info will be simply a jumbled mess with no particular use.

Basic performance testing strategy includes:

  • Defining which tests are required;
  • Writing test scenarios;
  • Selecting time in-between of test cycles
  • Selecting the number of iterations the test will be performed;
  • Comparing the results of various iterations with one another and with general requirements;

Usual suspects of PT are well-known and rather notorious. It is a cesspool of processing flaws in the system regarding speed, stability, and scalability. I.E.:

  • if the site runs slow with several users on board;
  • If the program presents inconsistent results;
  • if the thing stumbles on a different operating system;
  • If the system misbehaves significantly because of internal updates;

In other words — PT keeps performance smooth and sweet while routinely trying out its various aspects. It helps to calculate how many parallel users the site handle, shows how implemented changes affect the performance behavior. The endgame is fine-tuning of the entire system.

Without it — the thing will likely mess itself up at some point and subsequently fall apart and you will have once in a lifetime chance to learn how to pronounce “mistakes were made” in as solemn and somber manner as possible.

Basically, Performance testing serves as a riot squad watchdogs of well-rounded operation according to stated expected goals and documented requirements.

Another important aspect of PT is gathering data on the system’s activity in a certain scenario with a workload of varied extent. The results of performance tests serve as a basis for future feature specific tests. This gives a clear understanding of systems limitations and provides direction on what to refine and improve.

The goal of performance testing

The primary goal of PT is rather obvious — to define how much workload the system can take before breaking down or stalling in terms of user activity and expose weak points of the system with detailed specifics regarding the source of the problem before the damage is done.

Aside from helping to locate problems but it also provides directions for possible solutions via results and comparative tests. It makes clear when and why the problem occurred and what causes it to happen.

A basic set of performance testing requirements

  • Assessing the system’s workload capabilities against standard criteria;
    • Under normal circumstances;
    • Under peak conditions;
    • Also includes the system’s ability to restore to normal operation;
  • Estimating the response time
    • Average;
    • Under max load;
  • Finding problematic points in the system’s operation;
  • Defining breaking points and bottlenecks in operation;
  • Comparing test results on multiple systems;
  • Defining limits of the system’s operation from the user’s standpoint;
  • Estimating optimal hardware configuration required for adequate system maintenance;

All that allows seeing the heatmap of the application.

Performance testing routines are usually broken down into several types:

  • Stress — studies the system’s behavior and checks its stability in situations when the hardware is unable to maintain software. I.e. if CPU, memory, disk space are lacking;
  • Spike — aimed at studying specific segments of performance upon increasing load way beyond the projected scope for short periods of time;
  • Scalability — regarding an ability to adapt to changing workload. Specifically — testing user load, a number of possible actions, data volume.
  • Volume — used to monitor the efficiency of the operation by subjecting the program to large amounts of data.
  • Endurance — used to study the system’s behavior over long periods of time. Tests the system with the expected amount of load to check for memory leaks, processual fails, or scrambled acting;
  • Load — involves testing the system with increasing load until it reaches the breaking point in order to define threshold value;
  • Isolation — repetition of a test in order to check whether detected error or issue was fixed.

Every test is measured by certain metrics. The most common parameters are:

  • Response time (average & peak)
  • Amount of errors that occurred over the course of the test;
  • Throughput ability;
  • CPU / Memory load;

Want to Learn More About The APP Solutions Approaches In Project Development?

Download Free Ebook

In conclusion

Every application is a tight knot of various functions connected with one another. This means every element of the program has to be durable enough to carry the excessive and intense workload and don’t fail miserably in the process. But that is not the thing that happens on its own.

The smooth and stable operation is achieved by thorough and through and through the testing process. Test after test the process is polished to utter perfection.

It takes time and a lot of effort but it is definitely worth trying. It is a kind of guarantee that any significant issue will be solved before it gets badly out of control.

That is what performance testing is all about.

Read also:

Quality assurance and testing processes at The APP Solutions

Want to receive reading suggestions once a month?

Subscribe to our newsletters

Mobile App Testing

Posted on by Pavel Tantsiura
In this issue:

App developers know that to test an app is expensive and rather time-consuming. At the same time, this process is very important as it ensures your customers have a positive experience while they use your mobile application. Failing to do proper mobile app testing you’ll have your customers do that for you.

But unlike your testing team, the new users don’t have all those tools and time to report back problems. Those people wouldn’t also be happy to feel treated like guinea pigs. If they don’t like your product, most probably you won’t hear a word from them because they’ll just remove your app without coming back.

A perfect situation is when your developer did his best and made no mistakes. But finally, your goal is not to find the bugs, but to check if the app works as expected, does it meet the need of your users, so they can come back.

Mobile app testing may present some unique challenges. The mix of methods and techniques used during your app testing requires different choices and tradeoffs to consider. Each testing method has its own pros and cons and you’ll probably find out that there is no single super testing method that will completely satisfy you. Instead, you might consider a unique testing strategy that includes several testing options that in common offer the best overall testing result, quality, balancing the tradeoffs between cost and time to market.

Below we discover different testing options for mobile apps and explain the factors you need to keep in mind while determining your testing strategy.

Native Apps

For most people, mobile apps are similar to native and hybrid applications. These are usually downloaded from an app store and promise the end-user an exclusive experience that boosts the capabilities of the operating system and the device, for which these apps are developed. The apps’ downloads are controlled by the gate-keeping app-store with certain mechanisms to charge potential customers. This proven and rather a simple model of app monetization has built solid popularity among the developers.

Native apps can offer a stunning experience for the consumer and maybe a lucrative experience for a developer but these can also add certain complexity to the lives of guys from the testing team. After all the updates are made, you have to be 100% sure that your application can be released and valued by the end-user. There’s a common failure to consider that an app that is successfully tested on 1 device, will have the same positive results on the others with the same operating system.

Note that native apps are tied to the operating systems and hardware for which these are written. That is why you have to ensure compatibility with both new and older versions of the devices you’re expecting to support. And you’ll also have to remember about the regular updates for OS, specifically well-known “major” releases that users will apply soon after availability.

Web Apps

Like everything related to the Web, a mobile web application can be viewed by any user all around the world. Even if your target was to approach users from a single network or country, you’ll get an overview of the global dynamic.
When testing both web and native apps there appear a couple of challenges.

Each challenge means the necessity to explore and manage issues as well as decrease the possible risks. The right solution here is a proper understanding of the advantages and disadvantages of each testing option and deciding which technology better suits your app testing requirements.

Devices: A Huge Mobile Testing Challenge

Mobile app testing techniques

The biggest challenge for mobile app testing is the mobile devices used by customers. Generally, there can be thousands of various client devices used to access your app and these all should be considered when testing your mobile app.

Moreover, don’t forget about different types of operating systems and finally the combination of the system and the device. Ignoring all these specifications you have big chances that your mobile app will not work for a number of potential users. To handle the challenge you may consider 3 options: you can test using exclusively real devices, test with emulated devices or combine both of the options.

Real devices are the first option to consider as these will show all the quirks and limitations of the current operating system, firmware, and hardware set used by your target customers. But on the other hand testing real devices can appear very expensive.

You may also ask the network operator or manufacturer to loan you some devices for testing, but be prepared to “stay” in a long queue of the waiting list, continuing to convince hundreds of manufacturers and network operators that you are their priority. You’ll also have to pay for subscription and airtime.

Emulated devices are much easier to manage. You can quickly switch the devices by simply loading the new device profile that will imitate the real device. The emulators were specifically designed to run on more powerful servers and PCs with testing in mind. That is why these are fully prepared for diagnostics of protocols that go back and forth between server and client at the various levels.

Speaking about the disadvantages of emulated devices it is necessary to say that they lack the faults, quirks, and some other characteristics that just real devices can provide. The emulator is also not sensitive to ambient conditions that can influence the behavior of the device. So, if for example, you’d like to see how well a device performs in a certain location, such as a crowded stadium, a real device will be the best choice.

Fortunately, there is the third option- to choose a mix of both real and emulated device testing. We recommend you to start with an emulator to benefit from the diversity of devices and speed. IN the early development cycle you may achieve the goals at a relatively low cost. Adding real devices will help you to make sure that the apps are functioning as expected and meet all those development aims and requirements.

Network: A regional Challenge

Today there are more than 400 mobile networks available in the world. Each of the operators can support multiple network technologies, such as CDMA, LTE, GSM, and some less popular networking standards such as FOMA, iDEN and TDSCDMA. Every network features an exclusive combination of network infrastructure that tunnels the packet-based protocols used by mobile network operators into TCP-IP protocols originally used by the mobile web.

At the same time, each network operator uses systems, which are different from vendor to vendor. Finally, the majority of network operators started to use mobile web proxy servers/ gateway to dictate when, how and if you are actually able to connect to the particular website. Mobile web proxy can restrict the flow of information that is between the test client and the server. There are proxies that can limit the sites that can be accessed via phone. So the user can visit only the websites approved by their operator. So, the network challenge can be quite complex.

Talking about the network challenges it is also important to discuss the location. It’s obvious that to completely test the full network, you must be connected to the target network. But here’s another challenge to be accepted: the radio signals are not very strong on cellular networks, so you have to be adjusted to a cell connected to the operator’s core network to perform your test. That means to make a test against China Mobile you have to be in China and if you want to test against SFR you have to be in France.

And of course, traveling to every network operator that you’d like to support may become very expensive and there are also tradeoffs that should be considered.

Recommendations

Now after you understand much more about the main challenges related to mobile testing of both native and web apps, what are you going to do with this information? What strategy are you going to apply for mobile app testing?

Actually, there is no matter of selecting one technique or one tool just because of there is a lot of compromises to be made. Most likely you’d prefer to use a mix of testing tools and techniques to meet all your quality requirements. Naturally, you can narrow your choices down by following the below recommendations: get the advantage of using a device emulator, invest in a real device cloud, and automate wherever possible.