#48 - Testing Culture
Explore how software testing strategies can reveal valuable insights into your organization
Intro
Someone I appreciate used to say that the only constant is change. To ensure a team or organization achieves its goals, it needs to be dynamic and willing to adjust structures and processes to the changing reality.
Software faces a similar challenge. Features keep being added, libraries are updated, technology changes, and throughout all of this, you need to make sure that everything works as expected.
I thought about how different testing approaches could shed light on how organizations and teams can test their structure, processes, and culture, and make sure they’re on the right track.
Unit testing
A unit test is a way to test a specific component in a specific scenario. It doesn’t mean the entire software works or that it will interact with other components, but this specific component in this specific scenario works.
Yesterday, I installed a bell on my kid’s bicycle. Unit testing meant I first tried the bell and saw it works—it rings. It doesn’t mean it connects to the bicycle yet; it doesn’t mean it’s positioned in a way my son can ring it, but I know this specific bell is working.
Similarly, thinking of product teams, a unit test could be whether a specific PM can write a compelling user story. It doesn’t mean it will get implemented, or the developers will understand it, or that it works for the entire product team, but this specific PM can write a specific user story.
Integration testing
Integration testing starts checking the communication between different components. Each component on its own passed the unit test, but that doesn’t mean they communicate well.
As an example, you could build a roof with Lego bricks and a house with the same bricks. Each on its own looks great, but they don’t connect because their sizes are not exactly the same.
Following the thread on product teams, an integration test could be to take the user story that the PM wrote and run it with the development team. See if it meets the other end and if the developers can collaborate on it and implement it.
E2E testing
End-to-end testing mimics a real flow that the user could potentially experience. Following the examples above, it could be riding a bicycle while ringing the bell or building a full house using Lego bricks.
To draw on the product example, end-to-end tests could be the full product life cycle: discovering a new idea, prioritizing and planning it, building the user story around it, implementing it successfully, and launching it.
We tested each component on its own and tested different components’ integrations with one another, but now we are simulating a full end-to-end flow of the process.
Acceptance test
An acceptance test usually means that someone outside of the team who built the software tests it on their own terms and accepts it. Common examples include a PM testing a feature or a customer participating in an alpha group or usability tests.
The difference between end-to-end and acceptance testing is that end-to-end runs from the point of view of those who built the software, while acceptance testing is from those who use it.
Going back to the product team example, an acceptance test could be seen as a peer review effort, a kick-off meeting with stakeholders, or a general review of the product work process with the organization.
Acceptance tests are important because, at the end of the day, we build software for our users, just as we build processes and culture to meet our goals.
Load test
Load testing is conducted when everything works as expected, but we don’t know to what scale. It could mean running millions of requests to a service or looping through a process in parallel on a scale we haven’t seen yet. This type of test is crucial to make sure you meet the demand and have a resilient product.
In an organizational context, load testing could mean ideating and thinking about whether your current process and structure could handle being 2x, 5x, or 10x larger.
For example, if we have a team of five engineers and one QA, where everything goes through the QA, what would it look like if we had ten teams or if we had ten engineers in a team?
Alternatively, assume as a VP you’re running a planning meeting every two weeks with each team, and you have three teams. Would it work with ten teams?
Load testing your structure and processes is a way to make sure you’re setting up your organization for success in the future.
Smoke test
Finally, a smoke test is a type of test to see if a certain process or service is up and running. For example, every 15 minutes you might want to check if your signup process works. You don’t check its logic, its scale, or its end-to-end flow; you just want to make sure it’s available for users.
Similarly, a smoke test could be a heartbeat or a retro where you assess whether your current process, structure, or task is available and working as expected.
Some examples include checking a Jira board once a day to see tasks in different statuses, randomly picking a user story to see if it has valid content, or choosing a team once a week to join their daily stand-up.
These are ways you can check the pulse of the organization and its processes to see if everything works as expected. If it doesn’t, additional testing methods are required.
Conclusion
To conclude, software and products in general have similar concerns to companies and teams. They need to make sure they work efficiently, at scale, adapt to changes, and serve the end-user needs or goals.
I went over some popular testing strategies, and hopefully you got an idea of how to improve your visibility into your organization or team.