Prototyping and Experimentation

Most of the cost of a bad idea comes from committing to it fully before finding out it was bad. Building the whole thing, then discovering nobody wants it, is the most expensive way possible to learn a lesson you could have learned in a week. Prototyping and experimentation invert the order: learn first, commit later.

The point is not to avoid failure - it is to make failure small, fast, and informative. A prototype that flops has cost you days; a launched product that flops has cost you the year. Failing small is not a consolation prize; it is the entire strategy.

Prototypes: making an idea tangible

A prototype is a rough, deliberately incomplete version of an idea built to answer a specific question. A paper sketch, a clickable mockup, a cardboard model. Its value is not in what it can do but in what it lets you learn before the expensive work starts. The best prototype is the cheapest one that still answers the question you actually have.

Minimum viable product

An MVP is the smallest version of a product that lets you start the learning loop with real users. It is not simply a smaller product - it is a learning tool. The question an MVP answers is "will anyone actually want this?", and it answers it with behaviour rather than opinion. The discipline is resisting the urge to add features before you have evidence anyone wants the core.

A/B testing

An A/B test shows one version to one group of users and a different version to another, then compares what they actually do. Because the only difference is the change you made, the difference in behaviour can be attributed to it. A/B testing replaces "I think the blue button is better" with a measured answer, and it is how large web products settle disputes that opinion alone cannot.

The build-measure-learn loop

Popularised by Eric Ries in "The Lean Startup" (2011), the loop runs: build the smallest thing that produces evidence, measure what real users do with it, learn from that, and feed the learning into the next build. The aim is to get through the loop as many times as possible - the team that learns fastest, not the one that plans best, tends to win.

Suppose you want to launch a weekly newsletter for local restaurant owners. The expensive path: spend two months producing four polished issues, build a website, then launch and hope. The experiment path: write one issue, offer it on a one-page signup, and measure how many of fifty owners you email actually subscribe and open it. If two do, you have learned - for the cost of an afternoon - that the idea as framed does not land, and you can change it before you have sunk two months into it. If thirty do, you build with confidence. Either way the loop paid for itself.

Not everything should be prototyped, and "just test it" can be an excuse to avoid thinking. Some decisions are one-way doors where a cheap experiment cannot capture the real risk - you cannot A/B test whether to perform a surgery. Experiments can also mislead when the test conditions differ from reality, when the sample is too small to mean anything, or when you measure a convenient number (clicks) instead of the one that matters (paying customers). A fast, wrong answer is still wrong.

Eric Ries's "The Lean Startup" (2011) recounts how Dropbox validated demand before building the hard parts of its product. The underlying file-syncing technology was difficult and expensive to build, and the founders faced a familiar risk: spend a year building it and then discover nobody wanted it. Instead, co-founder Drew Houston made a short demonstration video showing how the product would work - a prototype of the experience, not the product itself - and shared it with an audience of early adopters.

By Ries's account, the video drove a large jump in signups for the beta waiting list, giving the team evidence of real demand before the heavy engineering was done. Ries uses the example to make a specific point about the MVP: it does not have to be a working product, only the smallest thing that produces a genuine signal about whether people want what you intend to build. The video turned an expensive assumption into measured evidence - the build-measure-learn loop in miniature.

• An MVP is just a smaller product. It is a tool for learning, not a stripped-down launch.
• Prototyping means building a rough version of the final thing. Often the best prototype tests the experience without building the product at all.
• A/B testing is only for cosmetic tweaks. Used well it settles substantive choices; used badly on tiny samples it produces noise.
• Experimentation is the opposite of planning. It is a way to plan under uncertainty - you plan what to learn next.

Given an untested idea, design the smallest experiment that would tell you whether to continue - name the question you are answering, the thing you would build, and the single number you would measure.

Think Like a Maester: Ask what is the cheapest thing you could build that would still change your mind - then build that first.

1. What question is a minimum viable product designed to answer?
2. How does an A/B test establish that a change caused a difference in behaviour?
3. What are the four stages of the build-measure-learn loop?
4. Why is failing small an advantage rather than a consolation?
5. In the Dropbox example, why was a demo video enough to count as an MVP?

Testing an idea cheaply before committing to it is the core discipline of problem solving under uncertainty. Prototypes make ideas tangible, MVPs turn assumptions into evidence, and A/B tests let real behaviour decide between options. Run the build-measure-learn loop quickly and often, and design each experiment so that failure is small, fast, and informative.

• Evaluating Possible Solutions
• Problem-Solving with AI and Digital Tools