Benchmarking
👋 Introduction
Benchmarking is a way to rapidly build context about the domain we’re designing for. It helps us understand the needs of users and features that are table-stakes. Benchmarking also helps us build on top of well resolved solutions for validated user needs.
1️⃣ Understand the domain
Identify competitors. Identify 5-8 competitors. Ensure a variety in selection from immediate competitors in your geography and adjacent geographies, to competitors that build for a similar user persona and the best products in this category worldwide.
Identify target audience for each competitor. They may or may not be the same across the products you’ve chosen. Understand the objectives of the audience of each competitor. Identify their attributes and qualities.
Create domain framework. Turn attributes and qualities into axes of a 2x2 matrix to create a framework. Find your product’s placement or positioning in the domain. This clarity will help tremendously in making product decisions.
2️⃣ Understand features
Start with the product architecture of each competitor. The architecture will reveal key features. These are typically areas such as onboarding, support, home page, and any features in the bottom navigation of your app. This will give you a good overview of all kinds of capabilities being provided to users in your domain.
Identify the goal of each feature and qualities of an ideal experience. As you look at the same features being provided in each of the competitors, look for attributes of the flows and designs that make for good and poor experiences for users. You are likely to discover a lot of interesting ideas this way.
3️⃣ Understand JTBD
Identify superset of JTBD for each feature across competitors. Create a matrix of availability. This matrix will help you quickly separate the feature leaders and feature laggards in the domain. It will also reveal if there is a agreed set of features that are table-stakes.
Collect screens for each JTBD for each app and make note of what is and isn’t working well. The best way to do this is to use the app yourself and take screenshots or screen record flows end to end.
Sometimes the app may not be available in your region or you may not be eligible to use it. Youtube walkthroughs are a reliable back up.
4️⃣ Evaluate apps
Create a feature rating matrix for the persona in focus based on notes, comparison and ideal experience. This will help you form an opinion of how well key problems in your domain have been solved by competitors.
Give apps an average overall rating as a simple average of the rating of all the individual features rounded up or down as needed. Make sure that each of the features are defined and grouped in a way so that they are equally important to the product. Doing so will help the average rating be an accurate representation of the overall product.
Conclusion
Through the benchmarking exercise, you will have built a deep understanding of the product space, broadened your idea of various solutions for the same problem, understood tradeoffs that competitors have made, and identified what constitutes as table stakes in your domain.
This learning should help you with more informed usefulness research, defining product strategy for an MVP feature or Product, identifying the USP for your product, and area for innovation in your product.
Last updated
