#28 - How Technical Tradeoffs Shape Outcomes
How technical decisions like environments and deployment piplines impact the product experience, and why we should care
I remember I worked in a place with five different development environments. When new code was introduced, it had to go through various stages:
Dev - local development
QA - where the team could test the new code
Integration - where the broader organization tests the new code
Pre-production - a prod-like environment for running automation and performance tests
Production - where real users interact with the software
While decisions on environments and deployments are technical, they significantly impact the product life cycle. Let’s see an example by imagining two teams: Team Alpha, with only two environments (Dev and Production), and Team Beta, which runs with five, like the above example.
In our example scenario, a Product Manager discusses hidden friction after a call with a strategic customer. The team then brainstormed and converted it into actionable tasks. Let’s see how each team handled the tasks.
Team Alpha (running with two environments) started writing and testing the code locally. Within two days, the work was ready for production. Utilizing continuous deployment, the code was in production by the end of the second day. They used a feature flag to control when the new code was active to mitigate risks. They encountered a bug, closed the feature flag, and spent another day fixing the issue. By the end of the third day, they reopened the feature flag, allowing the strategic customer to experience the new feature.
In summary, Team Alpha delivered the change within three days, although they experienced unexpected behavior in Production. Let’s examine some pros and cons of this approach:
Pros -
Faster time to market
Less overhead
Cons -
Prone to errors
Increased risk on Production
Accumulated technical debt (feature flags)
Team Beta (working with five environments) followed a different process. Like Team Alpha, the code was ready for deployment within two days. It then entered the QA environment, where a bug was discovered and fixed within a few hours. The code was redeployed and awaited entry into the Integration environment. There, another team faced breaking changes, resulting in a delay until the issues were resolved. By the fourth day, the code was ready and waiting to reach the Pre-Production environment. The automated pipeline ran on the fifth day and eventually deployed the code to Production. Without additional configuration, the strategic customer experienced the new feature by the end of the fifth day.
Team Beta delivered the change within five days without any regressions in Production. Here are some pros and cons:
Pros -
Enables early bug detection
Reduces technical debt (fully integrated code is tested early)
Cons -
Increased coordination with other teams and infrastructure
Higher costs (for the multiple environments)
Time-consuming
The structure of deployment pipelines and development environments exists on a spectrum of trade-offs. There isn’t a one-size-fits-all solution but a solution tailored to your organization. Trade-offs involving time, quality, costs, and technical debt also impact Product Managers.
This reminds me of other areas where we face similar dilemmas:
Product discovery and impact analysis - how many validations should we conduct before building something?
Product planning cycles - how many steps should we go through between deciding to build something and building it?
Product rollout - how should we gradually release a new feature versus a Big Bang release?
Sales cycles - the time, meetings, and personnel transitioning from a prospect's initial interest to deal negotiation.
Organizational decisions - communicating decisions through mid-level managers versus an all-hands company meeting.
Inspired by the trade-offs in deployment environments, I suggest viewing these questions through three lenses:
Speed - how fast do I want to reach the endpoint
Feedback - how much feedback do I want before reaching the endpoint
Alignment - what is the expected level of alignment before reaching the endpoint
For instance, a longer sales cycle offers earlier feedback (a BDR might quickly disqualify a lead) and a more substantial alignment (deal negotiation starts from a better point). Conversely, a faster product rollout reaches the endpoint sooner but comes with higher risks and potential misalignment.
To conclude, trade-offs and technical decisions, such as environments or deployment pipelines, can unexpectedly impact your work. So, what would you choose? Do you prioritize speed and accept some risk or increased coordination and alignment at the cost of efficiency?
Image by vectorjuice on Freepik