Evaluating response time results

When you initiate a test run from the test workbench, the Mobile and Web UI Statistical report displays graphs in the Step Performance view by default. This graph shows the average response time for each test step. However, there is an option which, when enabled, will collect data on the response time within the application and outside the application (in the network and server) for each selected step. With this additional data, you can identify the contribution of the application against the contribution of the network and server to an unexpected response time or breakdown for a test step.

Before you begin

To collect response time data for Android native applications, the instrumented version of the application must be installed on the device. Verify that the Playback on Instrumented app option is enabled in the mobile test client settings. If the playback version of the application is installed (this scenario will happen if you have already run the test with the same application either from the mobile test client or from the workbench), make sure that the instrumented version of the application has been re-installed. To do this, start a recording so that the instrumented application is installed on the device again. This can be a dummy recording. For example, you can exit the recording as soon as the application is launched. You do not have to record the test again. This step is just to install the instrumented app.

About this task

Important: This topic is applicable only for version 8.7.1.3 and later.

Note: To capture Response Time and Resource Monitoring data about mobile application and device's resources, you must have Android 2.2 or newer.

This feature applies to all mobile applications: native, hybrid, and web (for Android or iOS). To collect response time contribution data, the test must be started from the test workbench, command line, Rational® Quality Manager, or Rational® Performance Tester.

Procedure

Click Run Test in the test workbench.
In the Run Configuration wizard, in the Performance Measurement page, click Next, and select Collect Response Time Contribution Data. When the test is completed, the Mobile and Web UI Statistical Report displays graphs that represent the response time for each step in the test. By default, only the 10 highest response times are collected during playback, but the filter can be changed as follows:
1. Right-click anywhere on the Step performance report, and select Modify Filters in the context menu. In Modify Filters, select Count Filter: 10 highest (see figure 1) and click Edit.
2. In the Performance Counter Filter wizard, select Filter by count. You can enter a new value in Number to Display, and select Show highest values or Show lowest values. For example, if you enter 15, the report will show the 15 highest response times or the 15 response times with the lowest values.
Figure 1: Modify Filters
Right-click any response time graph and, from the pop-up menu, select Display Step Response Time Contributions (see figure 2). A graph displays the time spent in the application On App Time, and another graph displays the time spent outside the application (in the network and server) Off App Time (see figure 3). If you do not select Collect Response Time Contribution Data when you run the test, no response time contribution data is collected for the steps in the test, and the Display Step Response Time Contributions option is unavailable.
Response time is also measured for the first step, that is, the start step.
Restriction: Off App Time is not measured for the start step of Web and Hybrid applications, for Android versions prior to 4.0.

Figure 2: Response time contribution bar graphs for the 10 highest steps of the test

Figure 3: On App and Off App Time for one selected step
Click the Step Performance link at the beginning of the Step Performance tab to return to the main page.