Consumer Startup Metrics | Startup School

[Music] Welcome to metrics for Consumer startups. In our video on metrics for B2B startups, we talked about net dollar retention and gross margin metrics that are most important for B2B companies. Now we're going to dig into metrics that are particularly important for Consumer companies.

I was previously the founder of a consumer online bank called Monzo. It's now up to about 8 million customers in the UK, and I also worked at Grouper, a Y Combinator company that built a group social club and dating app. So, two consumer companies.

I've also worked with dozens of consumer companies here at YC. For Consumer companies, growth—headline user growth—is the most obvious metric that a lot of people track. The reason is often monetization comes later. You might have to build up some kind of network effect or viral coefficient—things we'll go into later.

A good growth rate is 15% month over month; at that rate, you'll 5x your user base every year. 10% monthly growth for a Consumer company is okay; it means you'll approximately triple your user base every year. 5% a month or lower is unlikely to reach breakout success, I’m afraid. But growth is much more complex than simply headline user numbers.

We'll split first of all into organic versus paid growth. So, organic growth is really anything you don’t pay for, and Founders running consumer companies often neglect this. We got to a million customers at Monzo before we spent any money on direct marketing or advertising, and really we did this in two ways: virality and network effect. It’s worth pausing on these because they’re so important.

Virality is the idea that one user using your product introduces it to other users somehow in the use of your product, whereas the network effect is the idea that the product gets better as more nodes in the network exist. I'll give you an example from Facebook's early days. So, when you took a picture on Facebook and uploaded it, you were prompted to tag your friends, even if they weren't already on Facebook. Those friends would get an email saying, "Hey, someone tagged a photo of you; sign up to view the photo." That’s a viral mechanic; by uploading the photo and tagging your friends, you spread it to new users.

Another example might be Wordle. So when people play Wordle, they post their score on social networks. You see those little green and gray dots; that's a signal for everyone else that they're playing this game and it attracts new people into the game. So that’s virality.

A network effect comes from Metcalfe's law: the value of the network is the square of the nodes in a network. What that basically means is the more people who use this thing, the more valuable it gets. WhatsApp is a great example of this. If you're the only one on WhatsApp, you know you can’t message with anyone; it's pretty useless. But the more people you add to the product, the more people you invite, the more useful it gets for you. You can message all these people around the world for free.

The best consumer companies incorporate both virality and network effect, which are different concepts but very closely related, in order to grow their user base organically. So let’s think about your product. How could you incorporate both of these things: virality and network effect?

First of all, for virality, what are the sharable moments? What are the points at which you know you've accomplished something new in the product? You've reached a new level in Duolingo, or with Wordle you've completed it in two tries. What are those points in the product where people naturally are inclined to share it? Then, how can you make it really easy with all of those sharing prompts that both iOS and Android offer?

The second is network effect. How could my product get better the more people who join? You have to shift from thinking about your product as a single-player journey to a multiplayer journey. For something like Monzo, which was a neo bank, the network effect had elements like being able to send and receive money really quickly within the bank. So we built a Venmo-style money transmission within the bank.

You could also open joint accounts or have big pots for groups of people who are going on a holiday. What we often saw is a group of six or seven people going on a holiday; only three people had Monzo. By the end of the holiday, all of the group had been convinced to sign up for Monzo, so they could jointly manage their expenses. That’s a network effect.

Working on these network effects and viral loops will pay back every day for the rest of the life of the company, whereas with ad spend, you spend the money one day, and tomorrow it's gone. You've got to keep spending that money to keep acquiring users. Viral loops and network effects pay back forever. So, every 1 or 2% extra you can optimize in your viral loops and your network effect will pay back for the rest of the life of the company.

Paid referral schemes are a sort of interesting blend; it seems like member-get-member, where if you refer a friend, you get $5 and they get $5. Or maybe with Uber, you get a free ride, and they get a free ride. I would treat this as paid acquisition; actually, you're spending money to acquire customers, and if you stop spending, those customers won't appear.

There are two things to watch out for if you’re doing these paid referral schemes. First of all is cannibalization; this is the idea that people would have referred their friends anyway, and now, by paying them, you're simply paying for users that would have signed up for free. You can test that by only running your paid referral schemes in certain parts of the country or certain cities, or turning it off for a period of time and seeing what natural organic rate of referral you get.

The second thing you need to watch out for is fraud. Honestly, people somehow milking your referral scheme. I had a friend who in the early days of Zipcar just set up a bunch of cheap Google ads that bid on Zipcar and then would redirect them straight to Zipcar and milk the referral scheme. He drove free Zipcars for a year and then got banned for life from Zipcar. There are always people doing this; you've just got to watch out for it—it's just an annoying cost.

Okay, so we talked about organic growth and particularly network effect and viral loops. Now we're going to talk about paid growth. So this is the idea that you do a pay-per-click campaign; maybe you do some TV advertising, or advertising in a newspaper or whatever. The first thing is, you have to have good tracking set up. You need to know where every user is coming from. Did they come from your Facebook ads, or your Instagram ads, or that TV ad?

The simplest way for pay-per-click ads is that UTM referral in the URL. If you can't get that, or I know recent changes to iOS have made it much harder to track, you should just ask your users when they signed up where they heard about you. You have to measure this paid versus free, and you've got to know where they came from.

It can get really convoluted with first-touch attribution, last-touch attribution, multi-touch attribution; it’s often way more hassle than it’s worth. Simply using those UTM referrals or asking your customers where they came from is probably the best way for most of you. Understanding where each user came from and then how much you spent on each channel lets you understand your customer acquisition cost—how much you spent for each channel to acquire each user.

It's important that you track this per channel and then, crucially, record that in your database—where each user came from—record it forever, because you can then monitor the performance of those customers over time. Back at Monzo, we found a particular money-saving blog that was very, very cheap to get users from. They sent us hundreds and hundreds of users when we were eventually paying for advertising; it seemed very, very cheap.

But the customers, it turns out, were deeply unprofitable. They would load money onto their Monzo card, go on holiday, and then instantly just go and take £1,000 out from the ATM and just spend in cash. We couldn’t understand why this was for a very long time; we researched it. The net result of it was we shut down that advertising channel, even though the cost of acquisition looked really, really good for that channel. The lifetime value of those customers was negative; the revenue and the profit they generated were negative for the company because they generated so much cost.

The second thing to watch out for in customer acquisition cost is you have to measure it to an active, monetized, retaining user—whatever that's defined as for your business. It might be a subscriber; for Monzo, it was a weekly active user. You have to measure it to get to a good user, not simply a sign-up. Because you might have an 80 or 90% drop-off rate in your first week, for example, you've got to track what it costs us to generate a user who sticks around for the long term—who performs and retains like one of our good users—that's your customer acquisition cost.

The best consumer companies have a split of organic versus paid growth of something like north of 80% organic to 20% paid, even 100% organic to 0% paid for some of the absolute best consumer companies like Facebook and WhatsApp in the early days. A 50/50 split between paid and organic is okay; anything below 50/50 for any period of time, honestly, is pretty worrisome. That's because if you're relying too much on the big ad platforms like Google and Meta to increase your growth, you're basically just going to have to pay them more money.

As you pay them more money to acquire more users, the cost to acquire each user goes up. These ad platforms are really well-tuned to extracting the maximum value from you and your competitors as possible. What happens is all these competing companies bid and bid to get more and more customers, and see they’re making less and less profit on each customer because they’re requiring more and more to acquire them. At the end of the day, only Google and Meta really win in that battle. You all take your margins to zero, and Google and Meta end up making all the money. That's why it's so dangerous to be so reliant on paid growth.

Similarly, a big platform shift, like the changes to the iOS platform, broke advertising tracking and just wiped out half of your business. I've seen so many companies overnight just die from changes that were made to the big advertising platforms. So please, please, please focus on organic growth—and that's viral loops and network effects.

To finish up on paid growth, I've not seen any great consumer company get to true scale where more than 50% of their sign-ups are coming from organic channels. I'm sure someone out there will watch this and find an example, but overwhelmingly the great consumer companies that get to IPO scale, to get to really big scale, have optimized their viral loops and their network effects to get the majority of their growth from organic channels.

Now we're going to dive into unit economics. This is the idea that you can measure, on a per-customer level, how much revenue they generate minus the variable or incremental cost associated with serving that one customer. So let's take Monzo, an online bank. When I was there, just a few years ago—these numbers aren't current—each customer generated about $50 in revenue, but we had a lot of costs to serve each customer.

So, for example, sending out a new plastic card and a replacement plastic card whenever they were lost was a variable cost. The cost of customer service; the more each customer contacted customer service, the more cost that would generate. Any fraud on the account or any transaction fees to make a bank payment or wire fees—all of those were costs that Monzo would bear.

We tracked those on a per-customer basis; this is very, very important because you can then understand not just how profitable or unprofitable the entire customer base is, but for each single customer and each cohort of customers, how they performed. For example, we discovered that customers at Monzo who traveled abroad very often would earn a lot more revenue, or customers who came in via a certain advertising channel I mentioned earlier might take money out of ATMs and incur massive costs there, or certain kinds of customers might contact customer service very, very regularly and incur a lot of cost there.

This really matters in an operationally complex business where you've got thin margins. It's really important that you understand on a per-customer basis how much revenue each customer generates and then how much variable cost each customer generates. You can optimize it; you can start reducing the cost, or you can start investing in the advertising channels that bring the kinds of customers who tend to generate higher revenue and lower cost.

So, rather than tracking these on a really broad basis, the more granular you can get on these costs, the better. Quick note: things that aren't included in variable costs are those fixed costs—the things that if you doubled or tripled your customer base, those costs wouldn't change at all. So those might be engineering salaries or the rent on your head office. Those are the things that you deduct afterwards to get to your total profit line at the bottom.

So for unit economics, you really want to take your revenue minus your variable costs; those are costs that vary as you increase the number of customers. As we talked about in the video on gross margins, scaling negative unit economics is very, very dangerous. In the early days of Monzo, we were at -3 or -4 per customer per year, and we scaled to half a million customers before we fixed them—earned a lot of capital. So beware, pay attention to unit economics and try to get them positive before you scale up your user base.

Okay, next retention. We talked about retention in our B2B video, and particularly net dollar retention. It's pretty easy in B2B SaaS companies when you have a recurring contract with a paying customer to know whether they've churned or not. With consumer, it's often a little harder, especially when they're not a subscriber but they just periodically use your product—like Airbnb.

The complicated question is: what is the right period to measure to tell if there's still an active customer or not? Should they be using a product every day, or every week, or month, or even year? For Facebook, an active user might be checking Facebook once or multiple times a day. But with Airbnb, you might only book a vacation home every six months.

So for your business, it's really important to think for a successful customer—someone who really likes the product—how often would they typically be using this service? For Monzo, we chose one transaction, at least one financial transaction every week. It can be hard to measure retention in consumer startups, especially when you've decided that the relevant usage period might be months or even years, bit like Airbnb or an airline booking.

What some companies have figured out is you can look for what's called a magic moment—this user behavior or activity that is correlated with or predicts long-term retention. The way you go about that is you analyze a group of your best users and compare their behavior—how they use the product—to the rest of your customer base.

A famous example was Facebook; they figured out that if you added seven friends in your first 10 days of usage, you'd overwhelmingly go on to become long-term happy users of Facebook. Whereas if you didn't add friends in your first 10 days, it’s much more likely you’ll churn off.

For Monzo, we had something similar because we had a bank account but also this Venmo-like functionality built into the product where you could send and receive money. Customers who added three friends from their phone book to their Monzo account so they could send and receive money as they signed up retained something like 20 percentage points better than users who didn’t have any friends in their first few days.

So once you figured out what your magic moment is—like adding friends for Facebook or Monzo, or it might be booking a stay and having a stellar experience as with Airbnb—you can try to re-engineer your product onboarding to make sure that as many users as possible hit this magic moment as soon as possible. So for Facebook, it was getting that add friend popup inside the signup flow, making sure that everyone who joined Facebook was prompted to add friends.

For your product, think about what that magic moment could be and how you can re-engineer the signup flow to get as many users as possible hitting that point. There is a mistake here, and I definitely fell victim to this. It’s getting too fixated on that precise definition of that magic moment. For Facebook, it probably didn’t matter that much whether it was seven friends or eight friends or six friends. Simply finding that kind of tipping point that looks about right from your metrics and then agreeing with everyone that that's what you're going to optimize for, rather than being really pedantic about the exact definition, is probably the best way to go.

The next thing we're going to talk about for Consumer startup metrics is the net promoter score. Again, this is a three-letter acronym—NPS—that sounds pretty complicated, but it’s pretty much a measure of how likely you are to recommend this product to a friend. You ask it to your customers, and it's a scale of 0 to 10. You basically take all the people who are promoters—that's a 9 or 10—and then you net off—hence the name— the detractors. That’s people who rate you 0 to 6, and that’ll give you a score.

So, if everyone is a detractor—all people who voted at zero—you'd have a negative 100. If 100% of people were promoters, and you had zero detractors, you’d have a plus 100 net promoter score. Another example: say you have 50 promoters scoring 9 or 10, 25 in the middle—25 neutral; you don’t do anything with those, you just ignore them—and then minus 25 detractors, those people scoring at 0 to 6. So, 50 plus 25, your NPS is positive 25.

If your net promoter score (NPS) is not extremely high as a new consumer company, you’re toast. You pretty much have to re-engineer the product to make it something that people love. The reason is it correlates extremely well with word-of-mouth referrals. I would argue that a positive 50 is pretty much a minimum baseline for any new consumer company you have to be in order of magnitude better than your competitors. Monzo, for example, hovered somewhere between positive 75 and positive 80 while I was there.

I looked up Tesla; it's currently at positive 96. So, the best consumer companies always, always have a sky-high net promoter score. If you look at old incumbents—people like cell phone companies or really old banks—their net promoter scores are often zero, or even negative. That’s a really good sign that you could disrupt those industries with great customer service and a great mobile proposition.

How you gather this score is extremely important. You ask this question to your customers; it might be every month or so, with a random sample, or at a certain point in the product. But the crucial thing is that you keep it consistent. If you change the point in the app or the point in the customer’s lifetime when you ask this question, you’re going to change the values you get. You’re going to change the responses, and your net promoter score will jump around wildly, and you won’t know whether it's something you've changed or simply an artifact of you having changed the collection method.

We fell into this trap at Grouper, the dating website I worked at. We changed the collection method, and the net promoter score decreased by 20 percentage points overnight. We couldn't figure it out. So, it's crucial you ask this in the same way of all customers at various points in time—you don't change this.

Finally, on NPS, you can influence this figure in particular: as a next question after people have answered whether they recommend this to a friend, you should ask a qualitative question—why? Why do you like the product, or what's wrong with it? Then in particular, you can go and look at all of the detractors, and you can fix all of the things they don’t like, and that's a surefire way to increase your net promoter score.

Okay, we've covered a lot of stuff in consumer metrics today. Let's quickly recap. First of all, is your growth rate at least 15% month-on-month? Growth in active users is the target. 10% is okay; 5% is probably not going to cut it for a Consumer startup. We talked about organic versus paid growth; you really need at least 50% organic to get to any kind of scale. If you’ve got more than 50% of your growth coming from paid channels, I'd suggest that in the long term, you have a problem that you need to fix.

So focus on virality and network effects to get those organic signups humming. If you're not already, start tracking unit economics; this is the amount of revenue each customer generates minus the variable cost that each customer creates. If your unit economics are negative, you must fix them before you scale.

Then we talked about retention. Retention needs to flatten off somewhere. If your retention periods are really long—if people only come back every 6 months or a year—look for a magic moment in your product. That moment where users finally discover the "aha" moment and are more likely to convert to long-term happy users, and the net promoter score—plus 50 as a minimum for great consumer startups.

Don’t mess around with how you’re measuring net promoter score; you're likely to throw off your metrics and confuse yourself. A final warning: these are all benchmarks; your company and your industry may well be different. You may have off-the-charts organic growth and very poor unit economics like early Monzo, or you might have really, really good economics and pretty slow growth that relies on paid ads.

Every business is different, and it’s hard to give cookie-cutter benchmarks for everything. So just think about these metrics and how they might be different for your business. I hope these are useful in running your company. Thanks very much for [Music] watching.