How Automation Is Giving Product Engineering A New Lease On Life

In a nutshell, Product Engineering can be described as the process of designing, developing, and deploying a product. With time, the traditional Waterfall Model of product development became ubiquitous. The sequential design approach quickly captured the imagination of engineers, due to its inherent simplicity and ease of use. Each step is linear and clearly defined, where each phase is completed before the next begins.

Why the Waterfall Model Has Run Dry

While the Waterfall Model changed course and was modified several times over the years to suit changing needs of the customers and costs, it still inherently had a number of challenges. Chief among them being extremely long release cycles, lack of flexibility, lack of prototyping, higher risk, uncertainty, and integration possible, only at the end.

With the passage of time, in a generation of almost instant gratification, customer needs evolved and they were expecting bigger and better things in a matter of days and weeks, no longer satisfied with long waiting times of months and/or years.  This was against the very core of the Waterfall Model. This ship had sailed. The need for a model that is on-demand, faster, cheaper, and better, that could be modified in a shorter amount of time, with lower overhead costs became the wave that product companies felt the need to ride.

Zinnov conducted an extensive study where we found 50% of the product engineering spend and resources was deployed after the code developed in the implementation phase (ref image 1) had been checked in. A far from optimal way of developing a product, as a result of vast time being spent and a lot of redundancies being created. Additionally, any modifications, alterations to the product after deployment, was expensive as well, because it meant going back to step one of the entire process. As market dynamics evolved, Product Engineering organizations have adopted strategies to overcome the challenges of the Waterfall Model, the chief among them being the Agile Product Development methodology, and now Product Engineering Automation.

Reducing costs, building near-complete products, and faster release cycles need to be paramount for product engineers. These can be achieved through an agile product development approach, where cross-functional teams (that use Scrum or Kanban frameworks) work parallelly. It’s a collaborative approach where adaptive planning, evolutionary development, early delivery, continuous integration, continuous testing, continuous deployment, and continuous improvement/modifications are possible.

While the agile method of product development is a faster, cheaper, better way, there is still room for improvement. By automating certain processes after the code has been checked in, the efficiency of the agile method can further be improved. Testing automation has already gained substantial traction, and now Test Design, Installation, and even Staging Release are being automated for precision-led outcomes and shorter development cycles.

When sub-processes are automated, it leads to increased organization agility and speed, while increasing product reliability, sales, customer satisfaction, innovation, employee efficiency to name a few. In turn, it helps reduce costs, time to market, lead times, and failure rates, thus increasing organizational agility and speed.

What Throws A Spanner In The Works

Although there are marked benefits of Product Engineering Automation, our research shows that there are significant challenges to automate processes within the agile framework. Some of them include –

1. Legacy Applications: They may not support integration with newer DevOps and Test Automation tools. Earlier releases of the products or end-of-life products which run on obsolete platforms are hard to integrate into a new age engineering automation setup.
2. Varying rhythms of Business Lines: Different business lines may have different security needs, user scenarios, and mode of access restrictions. These situations prevent an effective engineering automation strategy from being implemented.
3. Varying Vendor Maturity Levels: In the current outsourcing world, different vendors may have varying maturity levels, where some vendors may not have the capability to work in a centralized engineering automation scenario.
4. Mindset Shift: To be able to effectively implement successful engineering automation, most developers may need to think like testers, rendering individual testers obsolete. This may require a mindset shift, which is bound to come up against resistance.
5. Different Technology Stack Requirements: Increasingly, there is a business need for the products to support multiple platforms like MS Stack, Open Source, Mainframes, Cloud vs On-premise, may hinder in an effective implementation of engineering automation.
6. Security Concerns: Since Open Source tools are used extensively, many security concerns are raised. Using Open Source tools and the need to make data available to many more people raises security concerns.
7. Deployment Mechanism: Making the choice between Cloud vs On-premise for deployment of the product.

While Product Engineering Automation infuses speed and agility into the process of product development, organizations need to deliberate both in planning the rollout and also in the selection of a partner who will be implementing the same. We recommend that companies aiming to automate should find a strategic partner who has prior experience in product engineering automation, to maximize chances of success.