Category Archives: Quality Intelligence

Home »  Quality Intelligence

Give eyes to your SDLC

On June 18, 2019, Posted by ,

Devops-blog-postWith the ever-increasing complexity of the modern applications, it is now important to keep an eye on all aspects of the Software and Infrastructure to achieve greater agility, availability, and quality to take immediate action based on exceptions. In other words, Continuous Monitoring using an Integrated Dashboard is becoming a critical aspect of DevOps. But what to monitor and how? Can Automation be any help? Build or Buy? If so, how to find the right SaaS vendor? Read on to find out.

“DevOps is around only for about a decade but Monitoring SDLC goes back a longer time than that. We don’t have to reinvent the wheel and learn how to monitor the life cycle. We just need to find the right workflow that fits your business so you don’t overlook monitoring the important features which will often lead to unforeseen and unpleasant surprises down the lane.”

What To Monitor?

Today, nearly everything across the SDLC can be monitored and reported with the help of a Monitoring strategy, a little perspective and a lot of tools that are already available in the market. Let’s look at the major aspects that need to be monitored to stay informed well enough to stay alert, plan well, perform better, react faster and fix issues proactively.

Planning : Continuous Planning is where it all starts – after all, this isn’t a one-time task anymore when Continuous Delivery is the agenda. So, just like every other activity and asset of a project, Planning must be monitored too. Today, planning inputs come from user opinions, complaints and requests, competitive analysis, product vision and even operational insights. The ultimate goal of DevOps or any other IT methodology is to deliver Business Value.

What to measure?

Planned Value (PV) – Estimated cost of project activities planned/scheduled as of reporting date.

Sprint Goal Success Rate – Measuring Average number of sprints that met the goal in a defined time

Agile Velocity – Number of user stories completed by the team, on average, in previous sprints

Sprint Burndown – Number of hours remaining to complete the stories planned for the current sprint

Available Tools

01

Development Milestones : Development is a phase that involves the actual built of the software, writing code, designing infrastructure, implying possible automation efforts, defining test process, security implementation and preparation for deployment. Evidence is the most important part of this phase and adopting the right strategy and tools would make it achievable. As frequent code changing became the new normal, an efficient Code Manager can help developer store the code to make it re-usable, versioning, managing environments, and modules. Detailed development and defects statuses can be tracked through an Application Lifecycle Management (ALM) tool. There are various Continuous Integration (CI) tools available to monitor the build job and pipeline.

What to measure?

Cycle Time – The time taken for a task to go from ‘started’ / ‘in progress’ to ‘done’

Code Coverage – The percentage of code covered by unit tests

Cumulative Flow – The status of tasks in a sprint or release to visualize bottlenecks in the process

Time to Market – The time a project takes to start serving or providing value to the users

Release Frequency – The rate of official releases being deployed to production

Available Tools

02

Infrastructure : Every IT department thrives on a reliable, highly-performance and secured infra set-up for smoother operations. Considering the need of having 99%+ System Availability, businesses need to invest in real-time as well as proactive infrastructure monitoring solutions. As the organization grows, and the operations are spread over Virtual, On-premise and Cloud Infrastructure; there is stress on the system availability, maximizing uptime and reducing errors.

What to measure?

MTTR & MTTF – To estimate the uptime of systems

Infrastructure Stability – Percentage of reduction in the number of major incidents

Velocity – To evaluate Throughput and Bandwidth

Available Tools

03 (1)

Application Log Output : Having a system to monitor this fundamental part of the application can help administrators and security professionals to collect, analyze, and correlate the log data and provide actionable insights to the management teams. Application logs are Informational events which can form data that can help identifying abnormal user activities, troubleshoot abrupt application crunches, and detect security risks.

What to measure?

Average Response Time – the amount of time Application Server takes to provide the results requested by the user.

Error Rates – Identifying how often an application is exposed to Bugs and Production issues.

Count of Application Instances – To analyse the in-demand and off-peak times.

Request Rate – Understanding the traffic of the application

Available Tools

04

Application Performance : All the greatest code, tools and frameworks in the world are not going to help if the user is unable to use the application as expected. It’s important to have a system that can quickly identify when a problem arises and find out the root cause of the glitch and fix it immediately. That can be done only when the response time for various requests, CPU, network, memory usage etc. are monitored regularly.

What to measure?

Availability – Operational and Functional usability of an application to fulfill user requirement.

Requests per second – The throughput handled by a system

Response Time – The time taken by a system to react to a given input.

Available Tools

05

Quality Assurance : customers value bug-free software with fewer or no post-deployment dependencies after being delivered. Plus, with DevOps practices, there is really no need to sacrifice quality for speed anymore. Measuring the QA efforts through result as well as predictive metrics is important not only to improve the software testing frameworks but also to have a deeper and better understanding of the end product quality.

What to measure?

Performance (Response Time) – measuring the time application takes to respond to a given request

Automation Test failure rate – frequency of failures during automated testing

Application Quality Index – calculating and reporting the stability of an application under testing

Available Tools

Asset 9

Vulnerabilities : The list of dependencies for modern application is increasing day by day, so is the list of potential security threats. The Vulnerabilities that come as part of the package with third-party systems needs to be monitored. But the ones that evolve due to poor coding and development practices need to be identified and prevented as quick as possible. Mapping the complete application dependency tree, constantly monitoring the source code, test, production and pre-production environments, having a proactive monitoring system with alerts enabled can help.

What to measure?

CVSS Score – Capturing the principal characteristics of vulnerabilities and their severity.

Coverage – A qualitative view in culling the asset’s data and scope of scanning practices.

Dwell time – Time that a known vulnerability exists in the user environment

MTTD – Time taken to detect a vulnerability in the system.

Error rates – The fact of life for applications to track Bugs and production issues.

Available Tools

Asset 10

Now that you have all the major aspects under monitoring with the help of these tools, how to make sense of all this monitoring data? Whenever there are multiple tools at work, the only way to collaborate them is to have an aggregator. But,

Dashboards – Build or Buy?

Well, that’s always a tough decision to take, isn’t it? Well, it’s not impossible to build the ultimate DevOps Monitoring Dashboard. Whether to purchase a pre-built tool or to build your own solution, these are what you need to look for:

  • The range of tools supported – Through Plug-ins and Data Collectors
  • Ability to customize data fields for data collection
  • Powerful UI for data representation and customization of fields & views
  • Range of metrics supported to represent the business KPIs
  • Ease of installation and configuration A.K.A Usability
  • Ongoing Maintenance

MONITOR YOUR DEVOPS PIPELINE – THE QENTELLI WAY

Make a list of the Tools and Activities to monitor and identify the Metrics for each tool. Not just from development but cover the tools from all the areas of DevOps like code building, repositories, quality assurance, testing, deployment, and feedback to get the full picture. Now, correlate these metrics with the desired business value to define KPIs. This way, the monitoring console shall report all the aspects and demonstrate the current status and gaps. Well, having a well-designed dashboard is nearly useless if there is no data flowing into it, isn’t it? As the metrics are defined, the next step is to identify the right API or command-line interface to fetch the data.

Wait, we’ve already done all the messy work have a solution that is ready-to-implement (well… right after defining metrics to suit your business needs) and we call it TED – The Engineering Dashboard. TED is a machine-learning engine driven by Artificial Intelligence built on Big Data architecture and that gives you the much-needed single source of truth. TED is built to aggregate the data from all the tools through a custom-built API set that integrates into nearly all the tools within DevOps pipeline. Once the data is gathered and ready-to-use, the correlation algorithm starts its job, bringing the UI charts and dashboards to life for the consumption by Engineers, Chief Executives or anyone in between.

Any specific question about DevOps Monitoring, the tool or TED? Write us: info@qentelli.com