The Power of Continuous Experimentation

Continuous experimentation is a method of conducting experiments in a continuous and ongoing manner, rather than one time when a product is starting off.

It allows for quick feedback and the ability to make rapid changes based on the results of the experiments.

Continuous experimentation is a valuable tool for organizations like Irembo looking to optimize and improve their products or processes in a fast-paced and constantly changing environment

One example of the importance of continuous experimentation is the 2018 Snapchat app redesign. Snapchat shipped a redesign that was not well-received and hurt key business metrics, including active users. CEO Evan Spiegel later admitted that they “rushed the redesign” and did not adequately test it with a smaller percentage of their community before launching. This lack of testing led to the need to continue making changes after the redesign was launched, causing frustration for users. The Snapchat story re-iterates the need for iteration and gathering user feedback before go-live.

Guidelines for Continuous Experimentation

There are several principles and guidelines to follow when implementing continuous experimentation in an organization.

• Collaboration

Experimentation exercises give us a chance to collaborate across different roles that include product managers, software engineers,, and designers. This inspires creativity and ownership.

In a waterfall approach, the product manager decides what to build, designers decide how it looks or acts and engineers decide how it is built. However, the agile approach dictates that the whole team should be included in the ideation. Morale is higher when everyone is involved in decision-making and solution-crafting.

• Visualize your Experiments

Add experiments being done in the to-do column, such as customer interviews, surveys, landing pages, and so on.

With this method, the researcher can easily ask other team members to participate in the process if they choose to do so, while also achieving flow.

• Limit Experiments in Progress

It can be tempting to try to complete all experiments at once, but it is more beneficial to take time to carefully analyze the results of one experiment before moving on to the next. This helps to avoid repeating results and allows for the use of more precise questions that can provide new insights. Additionally, taking time to reflect on the effectiveness of different experimental methods allows for the identification of any weaknesses that can be addressed before continuing with the next experiment.

Conclusion

Continuous experimentation is a valuable tool for organizations seeking to optimize and improve their products or processes in a fast-paced and constantly changing environment. By following guidelines such as collaborating with various team members, visualizing experiments, and limiting the number of experiments in progress, teams can effectively gather data and make informed decisions based on that data. The Snapchat redesign serves as a cautionary tale, highlighting the importance of gathering user feedback and testing before launching a product. By implementing continuous experimentation and following these guidelines, organizations can avoid similar pitfalls and improve their products in a more efficient and effective manner.

Examples

• Experimentation at Spotify

• Airbnb’s experimentation program.