Review: The Lean Startup by Eric Ries

the-lean-startup
  • Sharebar

I picked up my copy of The Lean Startup with great enthusiasm. It was one of those books that was generating a lot of buzz around the startup community. The last time I’d heard the tubes of the interwebs filing up with all that commotion was when 37signals’ book Rework was published March of last year.

Does the book match its hype? I would say Yes. In fact, if I’d had a copy made out of dead trees instead of digital bits (courtesy of iBooks and iPad), it would have had about a hundred sticky notes threatening to fall out of the pages. Honestly, I need to go back and reread the book and make copious notes. My usual method of reading is more akin to speed reading.

But enough of my preamble. What’s so great about the book and why should you read it?

This will not be an exhaustive review of every main point of The Lean Startup. I will basically spell out a few highlights that caught my attention and then offer my thoughts which may differ from yours. In fact, if you disagree with me (or Eric Ries) on any points, please let me know in our comments!

A big Thank you to our December sponsor who helps make book reviews like this possible: . If you need A/V setup and event planning for your seminar, sales meeting, presentation, or other corporate event, contact DJ Dax.

1. Develop a Minimum Viable Product, or, Find Out What People Actually Want

A major theme of the book, and possibly the biggest takeaway for most entrepreneurs getting ready to launch a new product, is to avoid waste. Avoid wasting time, money, expectations, and code. Eric’s main example from his own venture, IMVU, was of spending a huge amount of time and effort coding a huge “selling point” of their 3D chat product — interoperability with existing IM networks — only to find out that once the product launched, hardly anyone actually wanted that feature. In fact, the way most people were interested in using IMVU was to find new friends not connect with existing ones. And some people, with their lack of understanding of technical jargon, were unable to grasp why it was a useful feature after it was explained to them in great detail.

So how do you avoid confusing users, generating features nobody wants, and wasting developer time and precious startup resources? The answer is two-fold: Develop an MVP (Minimum Viable Product) and then quickly iterate using a Build-Measure-Learn feedback loop. (The latter is called Validated Learning, I will get to that soon.) An MVP is essentially the core element of your product minus any other frills or add ons in its simplest form, launched to beta testers or the world at large, with the goal being to see if anyone actually would want the product. Now “want” can mean different things. For paid products, the question is, will people pay for it? For “free” or “freemium” products that are supposed to grow virally, the question is, will it in fact grow virally? The MVP gets you testing and answering your major business assumptions as early on in the startup cycle as possible.

Jared’s Take: There seems to be confusion about what an MVP actually is, and while Eric Ries seems to be OK with that, it bothers me. In some cases, companies develop a real product that can be used in the real world for their MVP — it just doesn’t have many bells and whistles. That’s fine, I think that’s probably a good thing! Simpler is usually better. But other companies are basically coming up with cute demos or barely-functioning prototypes and pushing those out as MVPs. And still others are actually putting up landing pages promising that their product will do XYZ once it launches, and they consider those MVPs! I think this train of thought can get a little out of hand. I don’t want to start finding product pitches all over the Internet when it’s really smoke and mirrors and the products don’t exist yet. I also don’t like the idea of users being treated like guinea pigs in a social experiment. I remember when Google was made fun of for perpetually labeling Gmail a “beta”. Now it seems like every product out there is a beta. When do we get a finished product that is relatively bug free, high quality, and reliable? I’m not sure.

Conclusion: Done correctly, I think the idea of an MVP is in fact quite excellent. Instead of spending years and major $$$ building something nobody will actually want, get something simple onto the market, find out what works and what doesn’t, and then make improvements accordingly. Avoid hype and splashy launches and listen to real-world customers as early in the development cycle as possible. Be ready to grow the product in directions you may not have thought of when you began developing it. Which brings me to the second major theme of The Lean Startup.

2. Use “Build-Measure-Learn”, or, Keep Your Feedback Loop as Short as Possible

A core component of the lean startup methodology is to introduce “validated learning” into the process. What that means is that you want to make sure that your assumptions about your product are in fact correct, whether it’s on day 1 or several versions along. For example, what if you think Feature A would be great to add to the product, but another team member thinks Feature B would be a better idea? Instead of it becoming a leadership tug-of-war, you would rather find out which feature can be tested in the real-world as quickly as possible to see if users want it. But how would you know? What does success look like?

The Lean Startup is a major advocate for split testing, also known as A/B testing. This is where you present a new feature (or a redesign of an existing feature) to only a subset of your users. You then test engagement of that feature and compare it with the “baseline” (which would be either the product minus the new feature, or the previous version of the feature before its redesign). If engagement (clicks, usage, completion of a task, etc.) is a positive win, the new feature is considered to be “validated”. If engagement is negligible or actually goes down, it’s a losing proposition and probably should be axed (or entirely rethought).

The cycle of building a feature, measuring it, and then learning from it is considered to be a feedback loop that is critical to the long-term success of your business. Therefore, The Lean Startup strongly encourages that you keep that feedback loop as short as possible so you can learn from your successes (or mistakes) as early as possible before you start burning through cash and missing your window of opportunity for the startup to take off.

Jared’s Take: I think this is one of the most compelling and hard-to-argue-with points of The Lean Startup. As a developer, I understand the Build-Measure-Learn loop on a miniature scale — it’s called make a change to my website code, refresh the browser, and verify my change works as I expected. You’d think that loop is important for development efficiency, no? Well, I’ve actually worked on projects in the past (not led by me!) where that feedback loop was much longer. You would have to make changes blind, submit your code into a repository, wait for a build process to complete, and then test if your changes did in fact work as expected. It was a nightmare. But the crazy thing is, even hip developer-friendly startups do this on a macro level! It doesn’t make sense to spent months working on a major new component of your product, only to find out it doesn’t work as expected in the marketplace once it’s in users’ hands.

Conclusion: It can be a challenge to introduce validated learning, split testing, and a short feedback loop into the process of development and decision making at your startup, but the payoff is worth it. You learn what works and what doesn’t more quickly, you glean valuable insight from user behavior, and you also have an audit trail to show off for why features get built into or taken out of a product. It’s less about the whims of head honchos and more about discovery of what the market actually wants.

3. Keep Your Options Open, or, Don’t Be Afraid to Pivot

We’ve all heard the term “pivot” if we’ve been around the startup or tech communities long enough. This is where a company takes the core technology or team they have built up and reorients around a new market, new platform, or something else of a substantial nature. It’s not throwing away all the hard work, but rather reapplying it perhaps to solve a bit of a different problem.

The Lean Startup gives a good overview of how to discover when you should pivot and how to do it. I won’t go into all the details here, but the main takeaway is that once you’ve built an MVP, and iterated on it a number of times using validated learning, you are in a position to find out if your continuous improvements have reaped rewards in terms of engagement or sales, or if you’re stuck in a hole you can’t get out of.

Jared’s Take: As a consumer, I personally have a bad taste in my mouth from some of the pivots of companies in the past that I deeply admired. One example is Be, Inc. They had developed an amazing operating system called BeOS which I ran on my PC in the late 90′s. It was poised to revolutionize the multimedia industry at a time when Windows was mired in the doldrums of Windows 98 (or NT which was nice for enterprises but hardly suitable for creatives). This was also before Mac OS X, of course. The problem Be faced was that OEMs would not preinstall BeOS on their PCs and were tied at the hip to Microsoft. So in an act of desperation, Be did a “focus shift” as they called it (it was a pivot), abandoned BeOS on the PC, and instead turned their attention towards “Internet appliances” which pundits said was going to be a major market. BeOS became BeIA (IA stands for, well, you guess), and in fact Be and Compaq jointly developed a prototype of what might have eventually been the iPad, years prior. I was ready to be first in line to buy one of these amazing new Internet tablets. And then Be got bought by Palm, which in their staggering stupidity killed both BeIA and BeOS products and that was that. What was the point of “pivoting” in that case?

I also remember the day that lives on in infamy when MetaCreations, makers of the most innovative creative graphics software ever invented (and I would add “to date”, personally), announced that it was laying off hundreds of workers and selling off most of its products to “significantly restructure its business to focus on its e-commerce visualization solutions”. Yeah, seriously. Ever hear of MetaStream 3D files? No? What a surprise! Obviously MetaCreations’ “focus shift” did not achieve the success they expected. (OK, so today’s company Viewpoint is the direct successor of the original MetaStream technology, but the company looks nothing like it did then and in my opinion is a sad shell of its former self.)

Conclusion: I know Eric Ries would come back at me saying that a pivot when you already have users that love your current products may not be wise. The whole point of doing a pivot is that you haven’t achieved market traction. My fear is that not all companies, even startups, may not what recognize what market traction looks like. What if you develop a product that attracts a cult following of loyal customers that simply LOVE what you’re doing, but you can’t figure out how to make enough money to keep that going? Is that success, or failure? I would say it’s success. Find another way to make money but keep supporting your customers. Never, ever, underestimate or underappreciate your loyal customers. A good example of this is the MMO game Uru in the Myst series made by Cyan Worlds. Cyan Worlds tried several times to bring Uru to market under the umbrella of a commercial game publisher, but they just couldn’t find a way to sustain the huge resources it takes to build new content for an MMO to the degree of quality that is a hallmark of Cyan Worlds’ products. But when Uru shut down, the clamor of loyal fans was defining. It got so bad that people started hacking away at their own server code to plug the Uru game client into, running illicit Uru shards. And some people started hacking into the Uru game client and creating new maps (called Ages) for the game. Finally, Cyan Worlds did the right thing and promised to open source the Uru client, which it has since done, and it has given its “blessing” to a fully-featured open source server clone. There’s now a surprisingly active little community called OpenUru around these initiatives. So that is my example of a “good” way to pivot. (Cyan Worlds is making money now from games like Stoneship and Bug Chucker for iOS and Android.)

4. Avoid Vanity Metrics Like the Plague

I will close by shouting a hearty “Hear Hear!” in response to Eric Ries’ call to stop using “vanity metrics” in your business (or business reporting for that matter). In other words, it doesn’t matter how many people have signed up on your site. It doesn’t matter how many rounds of VC money you’ve raised. It doesn’t matter how many articles have been written about you in the tech press. All that matters is that your customer growth is sustainable and the curve is exponential. Measure what will actually make your business successful. Did a higher percentage of visitors to your website sign up this week vs. last week? Did a higher percentage of free customers upgrade to become paying customers? Are more people engaging with your product on a regular basis this month vs. last month? Honestly, it doesn’t matter if you have 300 million accounts for your service if nobody’s logging on anymore and nobody wants to pay for it and advertisers aren’t biting.

Those kind of real-world metrics Eric calls “Innovation Metrics” because they encourage continual innovation to make the product work better and appeal to a correctly-targeted pool of customers, thereby increasing genuine business success. And the real-world success of startups is what we here at North Bay Startup are all about. So kudos to Eric Ries for sounding the call to do away with vanity metrics and focus instead on innovation metrics.

The Bottom Line

I hope you will buy The Lean Startup and give it a read. In the startup community, Eric Ries has proven himself a force to be reckoned with. You may not agree with everything he says, but just the process of thinking about what The Lean Startup talks about will sharpen your mind, enhance your startup skills, and make you a better person. Well, at least the first two. The last one is up to you.

Jared White is Editor-in-Chief of North Bay Startup. He has worked in the field of technology journalism for over 15 years and headed up several online blogs and media properties before founding NBS. He is also the owner and Creative Director at Siteshine, a digital media agency located in Santa Rosa, CA. You can reach him on Twitter @jaredcwhite.


Sign up for our Free email newsletter and get updates on the latest local startup news, tech tips, resources, contests, and more!

Leave a Reply to Ryan Critchett 4

  1. Great review Jared. I think pretty much everyone would agree with you regarding the MVP not being a landing page. Though wit the idea for the product I think its an important part of the testing phase on whether this is something customers will want.

    There is obviously a fine line between executing this correctly and incorrectly. If you offer a landing page with the features you say you are going to have and get people to sign up, and then don’t deliver that is a massive fail and people will lose trust in you.

    I don’t see anything wrong with putting a landing page up and buying some adwords to test and see if it something people will want.

    Joe McMackin on December 14, 2011 as 9:38 am
  2. Hi Jared,

    Yes, I heard the hype around this book awhile back and now reading your excerpts on it, I will likely purchase a copy and give it a thorough read. Thanks.

    Luke.

    Luke Carniglia on December 15, 2011 as 12:25 am
  3. Thanks guys. @joe, I think there are ways to put up lead gen page(s), communicate what the product is going to do, create buzz… I just think there needs to be some real product work underway and some genuine, long-term vision and planning in the works. I don’t want to subscribe to some temporary idea.

    Jared White on December 16, 2011 as 8:27 pm
  4. Definitely some good points. It’s important to know that this methodology doesn’t apply to all kinds of startups. Mine for example, required that I go all in to find my product market fit, not to test while trying to find it. I’ve iterated many times. 4 in fact. I did it in 7 months and spent less than 1500 bucks. Now I’m in a definite product market fit (or a service market fit)

    Ryan Critchett on February 12, 2012 as 6:38 pm
More in The Starting Point (1 of 8 articles)