Posts Tagged :

TestAutomation

Test Automation in the age of microservices – Strategies and Challenges

Microservices have been gaining traction across industries and are poised to see a stronger adoption rate in the years to come. Across sectors, many companies are aiming to achieve better enterprise agility and bring in a system more efficient than the traditional monolithic architecture. With organizations such as Amazon, Apple, Google, and Netflix scaling their microservices, the acceptance and implementation of it has grown and has compelled other players to emulate the model.

As companies look to decouple domain-level problems, API gateways see strong adoption across sectors. An API Gateway is a reverse proxy that exposes microservices as APIs. As the name implies, it acts as a “gatekeeper” between clients and microservices. Typical features of an API Gateway include the ability to authenticate requests, enforce security policies, load balance between backend services and throttle them if necessary.

While it may seem like a perfect answer to do away with monolithic systems, IT solutions need to be more modular and independent. As a result, microservices will need to co-exist with the traditional architectures and interact with the existing processes. Moreover, it will also need to be in sync with compliance imperatives for best results. To simply put, with organizations having numerous other architectural patterns already deployed, doing away with the traditional system completely may lead to a new set of challenges. In order to tame the complexities and manage the speed and flexibility of microservices, the API strategy makes for the best solution.

Test Automation in microservices and its challenges

With microservices becoming a critical part of Enterprise Architects, applications implementing them will need to be tested to ensure that the services are fully functional and are orchestrated as per business requirements.

As per the “Automation Testing Market by Technology, Testing Type, Service, Endpoint Interface and Region – Global Forecast to 2023” report published in 2018, the global automation testing market size is expected to grow from USD 8.52 billion in 2018 to USD 19.27 billion by 2023, at CAGR of 17.7% during the forecast period.

The increasing adoption of mobile devices and technologies, increasing adoption of the DevOps methodology, and transforming testing by digital transformation, are some of the factors driving the automation testing market. Moreover, manual tests are time-consuming and not full-proof. With most teams favoring automated tests in a CI-CD pipeline, the Test Strategy must use an Automation-first approach. However, microservices bring certain unique challenges to the Testing team – we articulate some of them from our real-life experiences:

Skilled resources: The primary challenge for automation testing is the lack of skilled resource. Most organizations struggle to set up QA teams that have the right skills to write automation scripts. The test automation frameworks that enterprises employ require testers with the skills to compose test scripts using various scripting languages and frameworks.

Tracing the problem: Automated tests are tools that are used to identify errors. Being able to trace back the to where the applications business logic failed is a mammoth task. using behavior-driven development could be a possible solution to refer to business-readable language that tells the business which requirement the automated script is testing.

Scaling test environments: QA teams usually don’t keep into consideration, the possibility of scalability issues while introducing test automation. The challenge arises when the need to rapidly make provision for the differing test environments that automated testing requires, scale them up, run the tests, tear them down, and do it all again just as fast if you’re not testing in the cloud. On premises, teams typically have a limited number of environments they can use, which means fewer tests they can deploy at any given time. As a result, testing takes much longer.

Too many UI tests: CSS and XPath locations in the UI change often. If QA teams target attributes like these in automated tests, it can lead to false positives and continued maintenance when the changes weaken or break the tests. Hence, the bottom-up strategy is vital that demands a unit level testing of APIs, which makes the testing more consistent.

Lack of transparency: Test automation can lack visibility when different teams are using different, disconnected automation strategies. For teams that work remotely, using different test automation frameworks, getting insights on the total testing quality can become challenging.

Adapting to the culture shifting: Adopting test automation requires a culture shift, an evolution of behavior and thinking. Often, team members and stakeholders look at throwing tools at a problem, which doesn’t fix the underlying mindset.

Strategies for successful test automation

These were just a list of some challenges that come in the way of a successful test automation of microservices. But to be able to overcome the challenges and to ensure that the results meet expectation, the most recommended is Mike Cohn’s Testing Pyramid, which takes a bottom-up approach for a quantitative analysis of how much automation effort is required at each stage of the testing lifecycle.

GU TESTS 2@2x-8

Let’s look at some approaches for how to approach automated testing.

Unit Testing: This is an internal testing service and is the largest in number and size. It is also the fastest and cheapest to automate.

Component Testing: Contract testing should treat each service as a black box and all the services must be called independently and their responses must be verified. When assured of the results, the longevity of the accuracy increases, also making way for seamless new additions to the existing systems.

Integration Testing: It is a must to perform verification of all independently tested services. Service calls must be made with integration to external services, including error and success cases. Integration testing ensures that the system is working seamlessly and that the dependencies between the services are as expected.

End-To-End Testing/API Testing: As the name suggests, end-to-end testing verifies that the entire process flows work correctly, including all service and database integration.

Additionally, there are several non-functional tests that requires equal attention. While functional testing helps in ensuring smooth performance of all the major functions, nonfunctional testing helps in assuring the reliability and security of the application.

Performance Testing: This not only evaluates the performance of the software, but it also ensures that the response time aligns with the desired time. Performance testing is carried out as a part of integration testing as well.

Load Testing: Load testing is carried out to check whether the system can sustain the pressure or load of many users accessing the application at a time. The production load is replicated in the test environment to get the accurate results by load testing.

Stress Testing: Stress testing is conducted to push the application beyond its capabilities to observe how it reacts. Contrary to load testing where maximum capacity of load is generated, stress testing is conducted where the load which is generated is more than the application can manage.

Improvement in software architecture has led to fundamental changes in the way applications are designed and tested. Teams working on testing applications need to constantly educate themselves and stay informed on the latest tools and strategies. Here are the most popular test automation tools that teams across organizations are using to get best results.

Hoverfly: Simulate API latency and failures.

Vagrant: Build and maintain portable virtual software development environments.

Pact: Frameworks consumer-driven contracts testing.

Apiary: An API documentation tool.

API Blueprint: Design and prototype APIs.

Swagger: Design and prototype APIs.

While there is a gamut of tools to choose from, there is no single-stop solution for automated testing. Testers must evaluate all the available options to make sure that the tool in consideration meets all the criteria for testing. Microservices are the next big thing in the IT solutions’ market but to be able to adopt the technology and implement the changes is more challenging than it appears to the naked eye. A skilled team equipped with the right tools is critical to ensure that a complex system architecture built on microservices can deliver the functionality, scalability and performance that business needs.

The Evolution of ‘Testing’

Over the last few decades, the art and craft of Software Development have followed the Darwinian principles of Evolution. From being an activity to ‘check if the software works’, to ‘find out when it doesn’t work’ to the current thinking of ‘anticipate failures and prevent them’, Testing has grown in importance within the lifecycle of software application development. Testers are no longer people who simply click buttons on a screen, but write frameworks, test scaffolds, code automated tasks, setup and manage environments and do everything that is traditionally associated with Developers and Technical staff.

Quality Engineering is the new mantra, where QA teams work side-by-side with developers in the trenches, engineering quality into the product from the start, preventing many defects from occurring and being detected. If you would like to move to this level of maturity, get in touch with us and we’ll set up some time for you with our experts to see how we can help you.

Reimagining Test Automation in the age of Digital Transformation

In the digital age, applications are the primary mode of consumption for nearly all products and services and brand differentiation lies in providing seamless, omnichannel customer service experience. Organizations have already adopted lean and DevOps ways of development and they have to afford new approaches to test new ecosystem of complex, highly interconnected, APIs and cloud-driven applications. IT leaders cannot pick the two out of Speed, Quality and Cost. They need all three–Speed, Quality and Cost for creating all-inclusive digital journeys.

Companies like Netflix are already changing how media consumption works and all credits to their engineering practices, right from development to testing. The company has gone from a manual mode to continuous, fully automated and high-volume testing. We are not talking here to replicate their engineering practices.

It is impractical to replicate the automation framework of any organization as there are stark differences in applications, technology stack, leadership style, team structure and size. Testing automation should happen in incremental ways to achieve required maturity. Every IT leader has to develop a unique blueprint for testing their applications to ensure a fully functioning digital customer experience. This blog talks how teams should re-imagine their test automation approach to complement their digital transformation journey.

The path to test automation in the age of Digital Transformation

Test automation is a buzzword but still not a household practice in organizations. Digital tsunami requires pro-responsive test strategies focused to deliver differentiated and high-touch services to reinforce brand identity in the digital era. The change in testing strategies need to touch myriad of processes, from challenging the status quo of established testing model to cater to the need for speed in dealing with changing development course with customer and partner feedback. Some of these changes that are required for test automation in the age of Digital Transformation are –

Focus on business tests – Teams are required to zoom out of the code and testing details; and take a closer look at the important business-level problems and write tests to solve these business problems vis-à-vis deliver seamless digital experience. One way of doing this is looking beyond the requirements and scope documents and build tests by looking at the real-time and operational data on how users are interacting with the application. Yes, it is possible. There are tools available in the market that gives idea metrics about end-user interaction. Teams need to understand these and develop tests accordingly.

Digital transformation revolves around customer-centricity and thus test strategies should focus on writing the test cases from a customer’s point of view. Behavior Driven Development (BDD) encourages use of simple language to blur the lines between engineering and business teams. BDD holds relevance for a digital future as it focuses on the outcomes and not the product important for the success of digital transformation.

Introduce Continuous Testing into your development – Digital transformation is beyond responsiveness and agility, its pro-adaptation to the future and related outcomes. Continuous testing ensures that engineering team are proactively testing every new feature in the development stage.

To introduce continuous testing in development, engineering heads have to introduce automation and leverage tools available for environment provisioning to test continuously at developer-machine level. Organizations often term test data management and generation as challenges because they are highly manual and time-consuming. These are the areas where organizations should use automation heavily to have testing running in parallel with the development. Test automation is required to test new codes continuously and this ensures a timely feedback about the bugs and issues to fix them early in the cycle.

Improving processes with Automation – Human-driven processes are prone to error, forgetfulness and skipping when they are redundant and highly manual. But with machines programmed for a specific function, there are zero possibilities of skipping or forgetting any test to run or data to generate.

Automating tests is pivotal to achieve continuous testing and make them apt for Digital age. Achieving 100% automation is an ambitious target because of constant changing requirements but achieving 85 to 90% test automation is doable even for the complex applications. Some areas of automation are test data generation, test data and environment management, running test suites and generating reports.

Test automation is a time and effort taking task. It’s very important to identify right tests to automate at the first attempt so that efforts and time do not go waste. Testing teams must create mini regression suite covering critical user journeys of high business value and run them first. Once teams have confidence about mini regression suite, the complete test suite is run, and results are collected to act upon among teams. There needs to be a defined acceptance criterion for every user story to ensure story is completed and functioning as expected.

Allocate budget for test automation – Testing requires same treatment and importance as given to the development activities. Testing is development cum testing as testers are developing code to test application under test. Businesses have to invest in right tools and technologies to make every-step automated and trouble-free. Teams devoting separate budgets have clear goals to be achieved out of testing and how to contribute to the software development lifecycle which makes it as a critical function.

Testing smartly with AI –Newer development methodologies around user interaction are uncovering the limitations of test automation. Testers write test scripts with guesses of how end users are interacting with the application. Though it’s a good way as testers are thinking from end-user perspective but successful test automation scripts must cover the end to end user journeys accurately. This is the larger issue with the current test automation practice.

Enters AI that can speed up the test automation practices by applying algorithms to large amounts of data produced by testing activities. Moving from manual and partial tests to matured CI integrated end-to-end functional suite includes a lot of manual and repetitive tasks. Organizations can use human capabilities to explore the areas of Automation, AI and Big Data in application usability, feature and integrations and test data analysis. Further, AI-powered test automation creates a knowledge base for self-learning and taking proactive actions.

A critical function, driving Digital Transformation

Organizations using AI for testing are surely creating a niche advantage for themselves rather than their counterparts by using data instead approximates and automation–two imperatives of digital transformation.

Qentelli’s test automation strategies are well-suited for DevOps and Agile environments to provide ROI to clients. Our test automation services cover web and mobile applications to get new digital services of client faster to the market. Do you feel manual testing is holding you back from right time releases? It’s time to engage deeper with us to address existential challenges in software deliveries at info@qentelli.com.

Automate your way to success – the power of all things automated

These days organizations are grabbing headlines about how fast they are evolving their businesses for radically changing customer behavior. Most of the times this evolution is software-driven. This draws the immediate attention even from businesses where core business is not software-driven to change their approach towards software deliveries.

While this gave front seat to engineering teams, and they are visualized as innovation squads, they are still responsible for application deliveries. Delivering both (Innovation and Applications) with old ways didn’t work. With this, the first to fail was the traditional “waterfall” approach where planning, analysis, designing, development, testing and deployment follow sequential steps.

As a result, they embraced the new age practices like SCRUM, Agile, SAFe, CI/CD for better software deliveries. Still there were unidentified silos, things are breaking in production, ops lack the vision of development timelines, etc because of various issues.

Organizations are now quality obsessed, and they want to think ahead of consumer. This gave the realization that despite of Agile or other development practices, there are silos in how different teams are working (Development, Operations, Infrastructure etc.).

This introduced the concept of DevOps which talks about functioning in a highly collaborative manner to avoid any communication barriers and ability to move fast without breaking things.

Development practices without Automation

DevOps as much as it gives the speed to engineering teams, the pressure to match the speed was enormous and this led to human error that costs companies millions of dollars. Few classic cautionary tales subjected to human errors are –

Automation is not a silver bullet

Automation is not a silver bullet to solve all your  business problems. There are some aspects of business that cannot be automated such as People, Culture and Leadership style.

Automation can never substitute for the creative mindset that teams bring to solve business problems.

So what can be automated in DevOps?

  • Your Builds
  • Your Tests
  • Your Infrastructure
  • Your Reports and Alerting
  • Your Security and Compliance

Whatever is not creative enough for your team’s attention!

AWS S3 Outage, Feb 28th, 2017: An authorized team member executed one command wrongly and removed a large set of servers than intended.

Knight Capital Group’s glitch worth $440 million: One technician forgot to copy the new code of Retail Liquidity Program and the whole incident happened. This is because of the manual deployment process.

While these human errors cost millions of dollars to organizations, they are invaluable lessons to others on the path of embracing software as business enablers. These infamous incidents of human error in updating servers or maintaining them, create a need of automating manual processes that require none of human creativity to accelerate engineering lifecycle.

How Automation entered DevOps practices

Can DevOps function without Automation? Yes!

Does DevOps function with the fullest efficiency without automation? Pretty much, No!

DevOps has evolved into successful and widely adopted development approach, so just practicing it will not give companies the true benefits of DevOps. True DevOps is not possible without automating manual tasks. Organizations are holding vis-à-vis generating lots of data enough to put manual and redundant tasks under automation. Automation gives the speed and feedback to resolve issues early in the software development lifecycle.

Code check-in, Test, Deploy, Repeat

With the convergence of multiple systems, IT services and multiple applications talking to each other, identify and automating processes can be tricky. Qentelli approaches DevOps automation with the focus on the areas of Planning, Development and Testing, Build and Deploy and Information & Reporting. With each of these areas, organizations need to optimize people, process and technology to achieve complete DevOps automation. Our automation approach upgrades day-to-day business operations in terms of speed and feedback.

Developers are checking in their code several times a day in the central repository (Git, SVN etc). These builds should go through the required tests with the defined acceptance criteria. Further to this, builds should go through the automated unit and functional tests to move further in the pipeline. Several organizations are running manual tests on newly added builds decelerating the pipeline. The first step towards DevOps maturity model is having automated code check-in and build with automated test suite kicking off as soon as developers check in new code. Automate your CI process with quality check gates based on the results expected. If they perform as expected they move forward, or developers receive notifications about the broken build.

Automated deployment to non-production environments require self-service infrastructure and automation of tests completely. While deploying in live is still practiced by a smaller number of organizations, it is achievable. Environment Management is the key element for automated deployment. The idea is not just to automate build and test, but also the underlying application environment with required configuration and dependencies.

Enabling automation configures feedback at the different stages in the engineering lifecycle to support various stakeholders. Integrating automation tools with the real-time dashboard can give teams real-time feedback on Engineering lifecycle. This reduces the resolution time of issues and businesses can deliver seamless experience to customers.

Looking ahead

Automation is the way towards success in DevOps model. DevOps automation has many moving parts for organizations to consider. To the core of DevOps automation lies automated provisioning, testing, build and deployment. Parallel to this, DevOps teams need to maintain continuous feedback with insights and action items moving back and forth continuously. While organizations have already achieved a basic level of automation in the business processes, extreme automation is the new target. Extreme automation in development lifecycle ensures the seamless communication between all the phases of development lifecycle.

We believe the future of automation is Extreme automation ensures that all the processes are being tracked into a single window collecting constant stream of data from connected development and operation systems to adapt development strategies to customer expectations.

Qentelli has partnered with clients to help them with their DevOps implementation and automation initiatives. Our team of experts carry out an extensive assessment to understand the current state and provide expertise in areas where improvements can be made. This assessment also involves areas of recommendation where automation can help. Get in touch with us today at info@qentelli.com for assessment of your DevOps initiatives.

Leveraging AI to automate QA and testing

The technology industry is stronger and dynamic than ever, thanks to Amazon, Uber, Facebook, Google. Price wars do not drive success for them but the convenience they bring for millions of lives. Software is driving organizational success with AI enhancing overall development practices. AI is the ‘not-so’ secret ingredient for smart software development practices. Agile and DevOps are speeding, organizations are using AI to automate the development process. QA and testing–the two areas where companies are experimenting and implementing use cases to automate manual processes.

2018-19, World Quality Report, “57% of the respondents said they had projects involving the use of AI for QA and testing, already in place or planned for the next 12 months.” The numbers show that organizations are looking to adopt AI in creating self-operating test systems.

One-size doesn’t fit all

The very first challenge for organizations is to achieve a desired level of automation and speed with the existing legacy applications and new application structures. The second challenge is “one-size doesn’t fit all”. Enterprises require unique testing frameworks and architectures for each application to test them optimally.  The third challenge is an ever-changing UI for applications because of the flexibility provided by Agile and DevOps. This requires a continuous change of automated testing scripts.

AI-driven testing platforms have potential to create smart testing landscape for a dynamic, modern and ever-changing applications. Such AI tools support business objectives to deliver quality applications at a faster rate.

Translating the potential into reality 

Organizations surpassed experimentation phase for AI testing use cases. They realized AI has the potential to transform slow, manual and error prone testing processes. 2018-19 World Quality Report, states that 36% respondents were using AI for predictive analytics in testing and 35% said they were using AI for descriptive analytics in testing. As organizations continue to mature in predictive and descriptive AI tools, we will see more use cases with the advanced AI tools and technologies.

The future testing platforms will have solutions such as script less test automation, test data and environment management, security and performance testing to achieve a complete end to end AI-driven testing. They will provide intelligence, analytics and availability of dashboards to understand how testing life cycle is improving and gaps to improve it further. Soon, it will be possible to analyze log files and test even before code is ready for testing.

Such AI testing tools will help in correcting code during the development phase and suggesting course corrections in development stage to avoid potential pitfalls. These course correction solutions will create a knowledge repository for organizations, for a quick fix. This helps in parallel execution of tests and code development to make deliveries even faster.

Leveraging AI will be the new normal 

With AI tools and accelerators available in the market, organizations leveraging AI will be the new normal. The debate around identifying right processes for AI will shift to leveraging AI for maximum manual processes. This requires enterprises to go under a rapid cultural change.

Organizations need to consider Systems under test (SUT) and Applications under test (AUT) requirements by going beyond the requirement documents. Teams leading the effort need to look at the business proposals, product case studies, high-level business problems and detailed system and design requirements for selecting right tools and technologies. This includes reviewing data requirements, process and environment requirements and framework and tool requirements. Organizations need to tie their AI testing tools to gain desired outcomes for measurable results. Features to look out for when organizations choose AI-driven QA and testing tools–

21638185-045a-4bc6-a0b0-7e01e46cd3be

The path towards AI 

As AI finds its way into the development activities, organizations need to look for various ways to consume AI in different engineering practices. The AI driven testing platforms will save time and efforts towards the redundant and manual tasks, allowing teams to explore other areas of software testing. The best possible way of leveraging AI to automate QA and testing is adoption of right tools and re-skilling to work in collaboration with AI systems. In the long run, AI is not about investments in tools and technologies but adopting it to sustain, scale and deliver quality products to the customers.

Intelligent Test Automation – The Qentelli way towards test automation

Businesses have realized that seamless digital experiences are less of a choice and more of a necessity in the world of ‘being connected’ always. Businesses have already started using agile and DevOps methodologies to create omnichannel and seamless experiences for customers. This means that every single code or apps and software releases need to be rolled out immaculately coupled with billions of devices connected to the environment. The advancements in the technology and need for speed requires second thought on: Is traditional way of testing will be able to cope up with the advancements in the digital world? What is the way out to survive in a digital world that requires real-time and high-quality digital experiences?

This article talks about how intelligent test automation can transform the software testing services to match up with the digital world software deliveries.

The challenges of test automation

Companies are using automation to put large amounts of test data into action, otherwise lying idle. Automation based AI algorithms can help in generating and optimizing test cases, prioritizing testing, monitor test results, and capture data.

The challenges of test automation cannot be generalized as they are dependent on where the companies lie in their overall journey of DevOps. For companies, those who have just started their DevOps journey, even capturing test data can be a challenge. The second challenge can be the time taken to automate test cases is long adding delay to overall development lifecycle. Whereas DevOps mature companies might be facing a challenge to plug in testing automation in the overall DevOps chain or moving from automated testing to continuous testing. Usually, companies keep test automation behind the actual development.

For instance, if a developer is adding a feature in sprint x, the automation of that feature is done in sprint x+1 or x+2 but in DevOps, the feature needs to be tested with automation in sprint x itself i.e. in-sprint automation.

The approach towards test automation should change from taking it after the development to integrate it with the development process. The test automation should be integrated with the Continuous Integration/Continuous Delivery (CI/CD) pipeline and act as a quality gate.

The future of test automation and how to start building futuristic test automation practices

Test automation is going to move beyond testing applications, environments, creating redundant tests for creating end-to-end automated user journeys. Owing to the fact that testing or development is no more working in silos under DevOps and every single function is accountable for accelerating application performance without compromising on quality, creating seamless experiences for users.

The future of test automation is about building business specific processes and strategies for test automation solutions based on the technology and type of application businesses are using. Futuristic businesses are dealing with advanced technologies and at the same time, they cannot do away with legacy solutions and thus, custom solutions are required. The future of test automation solutions will change based on app landscape and user journeys for every business.

Some of the ways to get started on building test automation strategies that will take you far –

  • Be tool and technology agnostic – Companies have a vast tool, technology, and application landscape and this keeps on evolving as per business needs. This advocate for multiple options of test automation tools and platforms. Today’s best automation tools will not align with the technologies of tomorrow. This requires businesses to build tool and technology agnostic framework with industry practices such as behavioral driven testing.
  • Build a solution for scalability and performance – The new test automation practices need to be scalable as per the business scalability. The key characteristics of scalable test automation solution is easy to develop for better adoption and full test coverage, quick to deploy for faster feedback cycle, completely automated for repeat and reliable results and cost-efficient even with growing business.
  • Work in sprint with the development team – Testing sprint needs to be integrated with the development sprint, whereas in most of the scenarios, testing is lagging. Application development and testing must be in the same sprint for quality and achieve maximum coverage during the development phase itself.
  • Automate test data required for testing – Test data automation is usually a challenging, time-taking and a very crucial step towards achieving test automation. Test data automation requires articulating a long-term vision, thinking about the context in terms of application, development standards, user testing etc. Read complete test automation best practices here.
  • Automate test environment creation – The manual provisioning of the environment will not be able to sustain for companies moving towards DevOps and automation, and companies must get rid of it quickly. Automating test environment can improve the test cycle times and ensures quality application or code being rolled out every single time, without impacting compliance and security standards.
The Qentelli Way

We at Qentelli have built many custom solutions for legacy, modern, web, mobile and other types of app landscapes. They have become like reusable utilities which can be plugged in together to make everything work seamlessly with all the devices connected with the application.

Qentelli has a suite of accelerators such as MoBe (Mobile and Beyond) for all the device access and automation, FAST (Framework for Automated Software Testing), AiR (Artificial Intelligence for remediation across the DevOps value chain) and others that provide advantage to test automation initiatives by integrating test automation within the complete DevOps value chain.

Our automation tools provide enterprise level test automation with DevOps first approach helping organizations to expedite their digital journey by testing fast, increase their test automation coverage and improves return on investment. Facebook Twitter LinkedIn

About Qentelli

Headquartered in Dallas, TX with global delivery teams in India, Qentelli is an Industry Thought Leader in Quality Engineering, Automated Testing, and Continuous Delivery. With high performing engineering teams working in the dedicated Innovation Group, Qentelli brings design thinking to address complex business problems and enables Continuous Delivery across Enterprise IT through automation for its global customers.

IT IS AUTOMATION, NOT AUTOMAGIC: AVOIDING FAILURES IN TEST AUTOMATION PROJECTS

Test Automation today has become an integral part in any QA teams’ arsenal. A decade ago, the agile delivery paradigm necessitated the need for automated test cases, while the current trend towards CI-CD and DevOps is fueling faster adoption of Automation across the Development Lifecycle.

There are multiple tools, frameworks, and solutions that claim to solve your Automation problems overnight. However, multiple challenges still exist, and IT Managers are not smiling.

The top reasons for Test automation projects failing, as found in the World Quality Report (a premium survey covering CIOs and top executives from 100s of companies).
As can be seen from the snapshot above, the challenges in automation have increased YoY in almost all categories..

Only about 18% of all Test Automation Projects continued after 6 months, which means that stakeholders get disillusioned with the results. But because of the significant time and cost already invested, the automation team continues with very low value.
The most common reason for such failures, however, is the expectation that Automation is a silver bullet – you get a team, install a product/solution and lo, your tests are automated end-to-end and can do everything, but brew a cup of coffee too!

Unfortunately, the road to sustainable Test Automation is long and hard.
Here are some steps that you can take to avoid failures in your test Automation initiatives.

1. Articulate a long-term vision:

This is the most important, yet oft-neglected phase. It is common to think, “hey, we need these tests automated, so that we can get through Testing faster/get feedback frequently”, but that is not the only aspect. Think about things like:

  • What are the data points that show your current problems? (For example, manual tests just to regression test the changes takes 2 weeks with 4 people)
  • How will that data point look like once automation is in place?
2. Think about Context:
  • What kind of applications exist?
  • What is the tech stack and how many layers exist?
  • At what levels do we need tests?
  • How frequently do we make changes to the source code and then release?
  • What Development standards do we use currently?
  • Does the application have significant integration components with other upstream/downstream applications?
  • How many defects slip into production or even User testing?
  • Consider Test Data and Test Environment availability, which can be major blockers for QA and not just for automation – you may have to invest in Containers/mocking frameworks additionally
  • Work with stakeholders to determine what reports would be needed and how they would be used
3. Consider your Application and Technology Roadmap:
  • How will the product evolve in the next 1-2 years? Is the product release model going to change soon (speed of release or # of items in the release)?
  • Is there a plan to rework the application in terms of the technologies used?
  • What new Technologies/paradigms is the organization planning to adopt (such as DevOps, Daily releases etc)?
4. Evaluate Tools:
  • Only after you have thought through the above, should you look at potential solutions
  • Evaluate multiple tools, using a pre-defined checklist to provide an objective view of capabilities
  • Consider running multiple proof-of-concepts using the most complex scenarios, not the simplest. This would help you uncover potential problems before you go down a specific path
5. Set Expectations:
  • Even with the latest tools that can generate some level of tests automatically, tests are going to detect only those defects that they have been coded/configured for. Automated tests are a safety net only and cannot replace humans for some time to come
  • When creating a business case or presenting the benefits, keep expectations low and provide a best-case/worst-case/realistic level of benefits
  • Communicate that the initial cost of setting up Test Automation vis-à-vis the ROI may be high, and the benefits kick in over time
  • Draft a plan with the most valuable tests first and then add more tests as needed, instead of a big-bang approach

In summary, Test Automation, especially for Continuous Testing in a CI-CD world, requires the right effort and investment to make it work. Planning for the Long-term and using an experienced partner to jumpstart your Test Automation can help you be well on your way to the holy grail – “Continuous delivery of value”.
To learn and explore more in detail about Qentelli’s AI-driven automated testing solutions and DevOps implementations, please write to us at info@qentelli.com. Our experts will be delighted to engage with you. Also, you can visit Qentelli’s social links for more details– Facebook Twitter LinkedIn

About Qentelli

Headquartered in Dallas, TX with global delivery teams in India, Qentelli is an Industry Thought Leader in Quality Engineering, Automated Testing and Continuous Delivery. With high performing engineering teams working in the dedicated Innovation Group, Qentelli brings design thinking to address complex business problems and enables Continuous Delivery across Enterprise IT through automation for its global customers.