One of the biggest club management software providers in USA collaborated with Qentelli to up their game of Performance Engineering.
Business Vertical:
Club Management Software Provider
Region:
USA
Size of the firm:
Partnered with 16,000+ health clubs and 150,000+ fitness professionals
Main Challenge:
Application Performance
Challenge
Contemplating the data history, we noticed there are maximum number of transactions during November to February due to increased yearly subscription during that period every year. As the users increased, the client started getting feedback from their stakeholders about slow response rates and frequent application crashing incidents during these peak business hours. The in-house application team did not have a choice but to restart the entire farm in sequential order every week to restore the system and it became a habit over time.
Slow response, unavailability and crashing of the application in production environment are nightmares, especially when it happens during peak business hours. It not only impacted end user/business user but also hit the revenue very hard.
Solutions Proposed
The client’s club management system was configured to be highly customizable to an intricate detail such as the button colour, text, design, etc. for every client. Besides, having inventory management system, order management system and many other features in one place makes it a great choice for the club owners. To help the client retain their top-player position in the market, it was high time to fix the performance issues in their applications. Here’s what the team of quality experts suggested the client in a nutshell.
- Improve the logging and alert methods
- Continuous Load, Stress and Endurance tests on all critical business scenario
- Setting up end-to-end automated Load, Stress and Endurance tests on every major build
What Qentelli Did
During the initial investigation and discovery phases, there was a lot of uncertainty about the key bottleneck. After carefully studying the server logs, we suspected memory leak on the code. As It was a legacy application, the logging and alerting system was not intact.
We teamed up with the in-house application team to understand all the critical business scenarios and started creating a common configuration for the business flows. Because every club that’s using the application had a different configuration and it was difficult to build the scripts to test each.
Strong and dynamic scripts were created:
As we learned that the application is highly configurable, we decided to make the performance test scripts more robust to ensure there are no scripts issues going further. This initiation of Script enhancement helped us add custom logic, enable basic parameters and dynamic value handling functions etc.
Fully automated test data pipeline setup:
Manging test data was one of the critical aspects as a lot of test data was not reusable. The application data was stored in Oracle and two other databases. So, before every test team has to request for the fresh data which was not ideal. Qentelli’s Performance Engineering team proposed to remove manual and data dependency completely and decided to automate this process without compromising the users’ access and data security.
Our Performance team streamlined the entire data pipeline so that all the test data from multiple DBs would go through PostgreSQL (Performance test DB) and Load testing tool (JMeter) shall test the data from PostgreSQL DB without disturbing the main application DB. The benefit of having dedicated performance DB is that we can perform Read and Write operations more conveniently.
Tuning the system to improve Response time:
Performance Engineering team has simulated multiple load & stress tests and tuned many transactions which were designed to breach Service Level Agreement (SLA). The team focused on:
- Optimizing the DB query and Indexes
- Re-designing the web pages that are fetching lot of data at each request from end-user
- Focusing on JVM utilization, upgrading the Java scripts, and using optimized Garbage collection algorithms
- Improving load balancer algorithm
- Making sure Garbage collection remove the large objects form the heap memory
- Optimizing the Thread & Connection pool configuration
Investigating the memory leak:
Our Performance Engineering team executed multiple rounds of endurance tests, spending over 72 hours for each test. It helped the engineering team drill down and get to the root cause of memory leak. Application has multiple functionalities, it’s performing just fine when the user request required less data however, it was becoming a logjam when the user request needed a lot of data to display on the screen. The application was consuming a lot of memory not only from the server but from the end-user as well. Qentelli’s team suggested Pagination to reduce the load on JVM. The code was also identified as one of the reasons for the memory leak in the system apart from user enrolment business flow.
Tech Stack
Outcomes
60%
increased sign-ups
60%
reduced risk usage
75%
improved SQL query performance
60-98%
better Transaction response time
%20(1).png){kind=small}
%20(1).jpg){kind=icon}