Test Automation Dashboard
1. Introduction
Software test automation is a process of performing tests on an application to validate its behaviour with the help of test automation tools. Whenever an automation test suite is executed, a test report data is generated and analysed. The test results data is a valuable source of information as it has the summary of all test activities performed along with its execution results. This data can be highly useful to know how the application under test performed over a period.
Software test automation helps to perform regression tests, receive a faster feedback cycle, reduce time to market launch and to save on the business expenses. With so many benefits listed, it had few drawbacks too. The one which is of concern is test automation lacks a systematic approach to manage the test execution reports and analysis. If the analysis is not carried out efficiently, it can leave faults unattended and cause wastage of time, resources, and efforts.
All the test execution reports are of value individually. But they are far more valuable when looked at together. They can offer greater insights into application behaviour and its performance over a period. These insights can then help make decisions for product development and help make product more robust. A healthy automated test environment in the long term has more important aspects than just ensuring that the tests have passed.
Business Need
We have multiple test automation runs over any week and we also have runs that go one for weeks, in loop. These executions generate lot of logs and test execution reports. This situation leads to a challenge for customer, to go through these logs and make sense of the data.
As we know, data is one of the most important assets of businesses in today’s world, as decisions regarding business strategies and approaches are mostly data-driven. However, data is an asset only if it is organized and structured to analyse the information for growth.
A systematic data will help the organization and the software automation team to
1. Make complex information accessible and easy to understand
2. Provide analysis to determine trends and exception conditions
3. Know the applications real-time performance
4. Organize relevant data for decision making
5. Set more actionable goals
2. Solution Implemented
- The solution implemented is to consider all the test report data and present it visually in the form of charts, graphs, and tables in a single view.
- Instead of having to spend hours on looking through different reporting features or various spreadsheets, viewers can get information easily with help of dashboard. Dashboard helps users to track, analyse and report on key performance indicators and metrics.
- Our test automation suite can be run for regression tests or longer period of times in loop for reliability tests. The data of tests success, additional data relevant for the scenario and performance parameters are collected during the test executions.
- Considering this data and evaluation of multiple dashboard tools available in market, we found Microsoft Power BI tool as best suited for our requirements.
- Power BI is a cloud-based, business Intelligence service by Microsoft which provides non-technical business users with tools for reporting, analytics, and visualisation.
- As part of the solution, we created a dashboard for the automation test suite reports. The dashboard is not limited to test success or skipped but also includes additional insights.
- It includes visuals to display application response time across various versions and over time. With the help of the dashboard, we were able to dig deeper and draw a correlation between the metrics and KPIs to understand what’s working and what’s not.
- User can drill down on visuals or charts as required to get more insights. This enabled us to have a clear picture of the application regression, performance and reliability among different versions of the application which otherwise was not possible.
- The dashboard has made it possible to make the test results and insights available for greater audience. It is helping in decision making and increasing coverage by test automation.
3. Benefits Derived
Here are the benefits derived by implementing a data dashboard in the organization.
1. Improved product deliverables The accurate analysis of data through a visual layout enabled better decision making, thus helping the customer to set measurable goals and deliverables for greater results.
2. Increased profits Dashboards were incredibly valuable as it made possible to bridge the gap between information and insight. It helped to present vast amount of information in a clear and concise manner using advanced data visualization and presentation techniques. It thus helped them to save dozens of human hours and millions of dollars from unnecessary spending.
3. Easy to read Use of dashboard reduced complexity and provided clarity. Dashboard made the complex information accessible and easy to read.
4. Easy performance checks and balances Case Study – Test Automation Dashboard 3 of 3 This enabled early identification of performance bottlenecks thus enabling faster resolutions. With help of dashboard, we were able to track progress, meet specific goals and even improve individual performance rates.
5. Monitor and track Key Performance Indicators As users do not have access to multiple different tools and services to track their progress, dashboard helped them to view and analyse all the metrics in one place which also helped in team engagement and accelerated growth.
Information is a key part of decision making. However, your business’ information is your asset only if you leverage it well. The major consequences of poor-quality data are poor decision making, business inefficiencies, mistrust, missed opportunities, and lost revenue.
A dashboard helps to show monitored data as abstractions or summaries to help users identify the metrics. Showing data with the context and offers users at-a-glance insights that simple measures by themselves cannot provide. Therefore, a dashboard, allow users to easily access the detailed information in a one-page canvas.
Let's transform your testing insights into actionable strategies for enhanced product quality and accelerated growth.
Discover more
Cloud <-> Intranet Gateway Design
Perforce to Git migration
