Abstractly, startups are a huge experiment.
The hypothesis is that this business can discover the right model to yield success.
Founders, early employees, and investors all believe that the experiment will work – but it’s yet unproven over the long term.
Simultaneously, for the company to succeed, it will run many experiments to learn what works with products, clients, and processes.
"𝑰 𝒉𝒂𝒗𝒆 𝒏𝒐𝒕 𝒇𝒂𝒊𝒍𝒆𝒅. 𝑰'𝒗𝒆 𝒋𝒖𝒔𝒕 𝒇𝒐𝒖𝒏𝒅 𝒕𝒆𝒏 𝒕𝒉𝒐𝒖𝒔𝒂𝒏𝒅 𝒘𝒂𝒚𝒔 𝒕𝒉𝒂𝒕 𝒘𝒐𝒏'𝒕 𝒘𝒐𝒓𝒌."
– Thomas Edison
Foster the Right Mindset
Experimentation allows you to explore incredible opportunities which could substantially benefit your business with minimal risk.
Let’s break down the elements.
Growth Insights. Most people believe that only huge, complex initiatives can cause large results. However, a simple product feature can heighten user engagement and a small benefit can drastically raise employee satisfaction.
Experiments do not have to be complex. Sometimes a small change can have an outsized impact.
Valuable Time. Experimentation allows you to evaluate new markets and invest in existing clients faster to jump ahead of your competitors.
These short-term tests save you time because higher-stakes decisions require more deliberation and resources than lower-stakes decisions.
Optimized Margins. Among your current customers, some may be willing to spend more if given the chance.
Imagine a coffee shop with hundreds of customers. Almost everyone wants a cup of coffee. A few people may even be willing to purchase an expensive latte machine given the audience.
By exploring options that complement your value proposition, potentially you will increase your profit margins.
You also can test operational changes such as increasing efficiency (e.g. automation) and offering incentives (e.g. referrals) to boost profits.
Minimal Risks. Another benefit of short-term experiments is the minimal downside when done correctly.
They often require less time and expense, and in the process, you may make some amazing discoveries.
For instance, by offering valued customers the chance to try a new product early, you may get constructive feedback before going public.
How to Run an Experiment
𝘚𝘵𝘦𝘱 1: 𝘊𝘭𝘦𝘢𝘳 𝘖𝘣𝘫𝘦𝘤𝘵𝘪𝘷𝘦
The more focused your intention, the more likely you are to achieve it. Determine success based on specific outcomes and clear metrics.
"𝑾𝒉𝒂𝒕 𝒚𝒐𝒖 𝒇𝒐𝒄𝒖𝒔 𝒐𝒏 𝒃𝒆𝒄𝒐𝒎𝒆𝒔 𝒑𝒐𝒘𝒆𝒓𝒇𝒖𝒍." – Karen Young
Decide and define the ultimate outcome e.g. seeking new customers, generating more engagement, improving profit margins, etc.
𝘚𝘵𝘦𝘱 2: 𝘚𝘪𝘮𝘱𝘭𝘦 𝘏𝘺𝘱𝘰𝘵𝘩𝘦𝘴𝘪𝘴
Ideally, dictate a single dimension to measure.
An easy example – will changing my headline cause more clicks?
Tests may be run across different teams: Product UX, Marketing outreach, Sales demonstrations, Operations software, etc.
𝘚𝘵𝘦𝘱 3: 𝘊𝘢𝘯𝘥𝘪𝘥 𝘊𝘰𝘮𝘮𝘶𝘯𝘪𝘤𝘢𝘵𝘪𝘰𝘯
Talk with different employees and stakeholders about your objectives and process to get valuable feedback.
As the impact increases, talk with more people via samples, focus groups, and interviews. Verify you are asking the correct target audience.*
𝘚𝘵𝘦𝘱 4: 𝘋𝘦𝘴𝘪𝘨𝘯𝘢𝘵𝘦𝘥 𝘖𝘸𝘯𝘦𝘳
You want one person to ultimately own an experiment.
That person may be managing a team directly, facilitating a task group, or directing both internal and external resources.
𝘚𝘵𝘦𝘱 5: 𝘜𝘴𝘦𝘧𝘶𝘭 𝘋𝘢𝘵𝘢
Distinguish sentiment scores from actual user behaviors.
For example, asking people if they might buy a new product is far less meaningful than seeing verified purchases.
𝘚𝘵𝘦𝘱 6: 𝘔𝘦𝘢𝘴𝘶𝘳𝘢𝘣𝘭𝘦 𝘙𝘦𝘴𝘶𝘭𝘵𝘴
Pair the original objective with a successful outcome. Once underway, use metrics to confirm which behaviors lead to the desired outcome.
Ideally, you can compare metrics over multiple periods to eliminate seasonal trends which may impact your findings unrelated to the test.
𝘚𝘵𝘦𝘱 7: 𝘕𝘦𝘹𝘵 𝘚𝘵𝘦𝘱𝘴
With your results, you may expand the experiment, test another element, or concentrate on something else.
Even if the result is not what you expected, you still have learned where not to waste your resources, getting you closer to the right formula.
Too Much of a Good Thing
Following success, you may get so excited about experimentation that you generate a huge list of additional tests.
That’s great you’re excited - but you need to prioritize that list.
Decide which experiments to run based on impact (growth and efficiency), certainty (expected success), and cost (time and expenses).
Experimentation opens a world of possibilities to increase revenues, reach, and efficiency. Utilize testing to move forward quickly.
--------------------------
* When McDonald’s rolled out the Arch Deluxe burger in 1996 to target “sophisticated urbanites”, it spent $200M in marketing (around $360M in today’s dollars). Volunteers recruited to test the new burger were already fans of McDonald’s and were asked if they wanted to try an upscale new item. See the bias here? The target audience was the average consumer but the experiment was offered to existing champions offered a free taste.