Analyzer

Analyzer file Extension is “.lra”

1. Cross results option: Allow you to compare two “.lrr” files as part of benchmarking test.

2. Section explorer: Contains “.lrr” path, period, duration, average throughput hits per second, total throughput, hits per second, transaction response time and status code.

3. Graphs: Allow you to add and delete the graphs.

4. Properties: Allow you to exclude/ include think time and generate percentage response time.

5. Controller output message: Controller error message will be displayed which will be help full to analysis.

6. User data: Allow you to write something.

7. Raw data: Based on the request we can pull the raw data and send to architecture people to analysis purpose.

8. Graph data: Will give raw data for graph.

9. Legend: To make you understand which color is indicating which measurement.

 Scale: Indicating number of measurements in graph.

10. Granularity: Time difference b/w two saturation points.

    NOTE: minimum Granularity for throughput and Hits per 5 second.

    For all remaining graphs 1 second

11. 90th percentile:

90 percent of the transactions are completing with in this limit.

        Step1: Write all the response times in ascending order.

        Step2: Take out 10% of values from below.

        Step3: Which will be the highest value consider as 90% response time.

Note1: We have to report only 90% response time to client.

Note 2: Based on client requirement we can generate 80%, 85%, 90% …etc.

12. Reports: By default we can generate doc report, Html report, crystal report, PDF report.

• Processing Result
• Rich Text Document
• HTML Result 
• Analysis report (Excel Report (With Base line comparison or with baseline comparison)

      Reporting: Once the test got completed, I will export the response times to excel and I will prepare a comparison report.

Summary Report :-

1. Analysis Summary

2. Static Summary

3. Transaction Summary

4. HTTP Response Summary

1. Analysis Summary : 

   a.  Execution Period(from to To time)

   b.  Project Name

   c.  Test Name

   d.  Test Description

   e.  Run Time

   f.  Total Duration

2. Static Summary :

    a. Max Running VUsers

    b. Total Throughput

    c. Avg Throughput/sec

    d. Total HITs

    e. Avg HITs/sec

3. Transaction Summary :-

    a. Transactions Total Passed, Failed, Stopped

    b. Transaction Name

    c. Min Response Time

    d. Max Response Time

    e. Avg Response Time

    f.. Std Deviation

    g. 90 percentile

    h. Transaction Pass

    i.  Transaction Fail

    j.  Transaction Stopped

4. HTTP Summary :

    HTTP 200

    HTTP 300

    HTTP 400

    HTTP 500

Comparison report:

It compares 90th percentile response times with baseline response times of previous test results and I will maintain a RAG (Red Amber Green) status.In some other tab, I will copy merged graph to understand the test results.Apart from comparison report, we will prepare a quick analysis summary which contains objective, scope, how we designed the scenario, test environment, observations in terms of resource utilization, high response times and controller, web server logs.

I will send a mail to get AWR, NMON reports for future analysis.

I will prepare a PPT by analyzing all the supporting files (AWR, NMON …) by mentioning objective, observations, environment comparison, high response transactions, root causes to present to the stake holder.

What is your approach to analyze the statistics?

(or)

What is the process you are following to identify the bottleneck?

A:

Once the test got finished, I am going to compare derived statistics with expected statistics. If both are not comparable, then I will start the process to find out the root cause.

• Client side statistics analysis (analyzer, throughput, Hits per second, Response time).
• Server side statistics analysis (Hardware and OS level statistics).
• Application side statistics (Methods, I/O operations, DB, EJB, Packages etc..).
• Configuration setting analysis (Current limit, Connection limit, Thread limits..etc..).

Result Analysis : 

The primary goal of any performance testing is to provide a clear status on application performance and to identify the performance issues (if any). Performance Test Result Analysis is crucial because wrong prediction or a decision to go live with risk may impact revenue, brand, market perception and the user experience. Hence it is important to learn:

How to read the graphs?

How to merge the graphs with other graphs and metrics?

How to conclude the result by analysing the graphs and numbers?

How to make a Go/No-Go decision of an application?

So, let’s start with some basic graphs 

1. User Graph

A User Graph provides complete information about load pattern during the test. This graph helps to identify:

        When did the user load start?

        What were the user ramp-up and ramp-down pattern?

        When did the steady-state start?

        How many users were active at a particular time?

        When were the users exited from the test?

        Every performance testing tool has its own term to represent user graph:

2. Response Time Graph

Response time graph gives a clear picture of overall time including requesting a page, processing the data and getting the response back to the client. Generally, the response time of a website at a normal user load could be in between 1 to 5 seconds although for a web-service it could be less than 1 second or even in milliseconds.

Types:

Request response time: One shows the response time of a particular request on a webpage.

Transaction response time: It shows the time to complete a particular transaction. Just to remind you a transaction may have multiple requests within it.

A Response Time graph provides information about:

        Average response time of request/transaction

        Min and Max response time of request/transaction

        Percentile response time of request/transaction. Example: 90th, 95th, 99th etc.

3. Throughput Graph

This graph represents the amount of data transferred from the server to the client per unit of time. The transferred data is measured in terms of bytes per second, KB per second, MB per second etc.

Throughput graph is used to find out the network-related issue like low bandwidth. A drop in throughput graph also denotes the issue at the server end especially queuing issue, connection pool issue etc. Throughput graph needs keen investigation with other graphs to get understand whether the issue is with network bandwidth or at the server end.

4. Hits per second Graph

Hits per second Graph refers to the number of HTTP requests sent by the user(s) to the Web server in a second. Always remember, there is a difference between Transactions per second and Hits per second in terms of performance testing. To provide more detail, a transaction is a group of requests which creates multiple hits on the server. Hence you may see multiple requests (hits) against one transaction. Hits per second graph help to identify the request rate sent by the testing tool. High response time may cause less HPS.

Example: A login operation which involves 10 HTTP requests can be group together into a single transaction, so when you see 1 dot in ‘Transaction per second’ graph there will be 10 dots in ‘Hits per second’ graph.

5. Transactions per second Graph

TPS aka Transactions per second graph represents the number of transaction executed per second. During the test, while doing live monitoring Transactions per second (TPS) graph helps to keep the eyes on transaction rate. If you have TPS SLA then you need to prepare the workload model in such a manner so that it can maintain the defined TPS; neither more nor less. TPS can be controlled by setting proper pacing in the test scenario. High response time may lead to less TPS.

6. Error Graph

Error graph shows the number of errors identified during the test. Error Graphs could be quantitative as well as descriptive. The quantitative error graph shows the error count and occurrence at a particular time and descriptive error graph shows the error message in detail which helps to find out the root cause. The merging of Error graph with other graphs like response time, throughput graph etc. gives a clear picture and helps to identify the bottleneck.

7. CPU Utilization Graph

Moving towards server-side, the first metric is ‘CPU utilization graph’ which shows the percentage of utilized CPU. The stats of server-side metrics are noted for pre-test, during the test and post-test period, so that utilization percentage can be compared.

Server Monitoring tool captures the resource utilization. CPU is one of the resources among them. These monitoring tools can be agentless, agent-based or in-built with the server.

8. Memory Utilization Graph

The purpose of the memory utilization graph is as similar as the CPU utilization graph. The only difference is that we check the memory status during the test in this graph. The pre-test, during the test and post-test period, helps to determine the memory leakage in the system. Memory utilization graphs are captured by monitors which can be agentless, agent-based or in-built with the server.