Metrics for Startups: A Practical Take (part 1)

December 2019 · 11 minute read

There’s this beautiful concept that Jonathan Hsu, from Tribe Capital, wrote down some time ago and stuck with me.

It more or less goes like this:

Hundreds of years ago, some Italian monks created a system to assess and discuss companies financial health. Today it goes by the name of accounting.

Startups, though, are structurally different from traditional companies. They prioritize growth over profit, in the hope of reaching a critical mass as fast as possible, resulting in horrible balance sheets.

Since accounting comes short for startups, we had to come up with a set of accounting rules that could assess the “growth health” of a company. An accounting for growth.

With growth accounting, we refer to product KPIs and measurement rules that can help both investors and founders understand how a startup product grows, sticks, and gets used. This can become a powerful proxy for future cash flows when (and if) associated with both a growth model and a business model.

Unfortunately, in my experience, this is not a practise as widespread as it should be.

On one side, EU early-stage investors tend to focus more on assessing the financials rather than on assessing the product. Quite understandably, then, few founders are incentivised to proactively make product assessment either.

On the other, growth accounting can look like a daunting task only suitable for data scientists and developers. For sure no university or MBA is teaching how to do serious product due diligence. And it looks like most of the learning content out there is more into theory than into practical tips.

The core concepts, however, are somewhat trivial, and their implementation is super smooth thanks to products ready to use.

Then again, it’s more critical than ever to learn such discipline after so many accelerators imploded, and their trickle-down know-how got lost.

First thing first: What is growth accounting?

It’s important to start by laying a common ground on what we’re talking about.

Hsu go deep into the rabbit hole in defining and outlining growth accounting. I had a few old-style accounting classes during college, and I can see Hsu’s point.

Even so, I nonetheless believe that when early-stage less is more, even with metrics.

Broadly speaking, what matters boils down to measuring:

Product intent means measuring how much the user base is growing, and where it is coming from. In this category, the most important metrics are month-on-month (m/o/m) growth rates and acquisition channels split (ie. how users discover the product).

Product habit means understanding how much the users consume the product after having tried it, and how this affects growth. Here we have typical engagement metrics like Monthly Active Users (MAU), daily ones (DAU), retention cohorts (how many users come back to using the product after a given time period), and churn rates (how many users who tried the product stopped using it).

Product behaviour means discovering how users use the product. Here we find conversion funnels and product-specific events (like “likes per user” for social apps).

Why is this relevant for investors and founders alike?

For investors, it’s easy to make the case of learning growth accounting: spotting and vetting product growth is the easiest (only?) way to assess the future potential of a company.

In an illiquid market like the one startup investors are operating in, you can’t rely on market signals to price a company. You can’t rely on future discounted cash flows either, because you have no idea of their size or magnitude. And you’re in a market with scarce and asymmetric information.

Long story short, product growth due diligence is crucial. Metrics carry knowledge, and growth accounting can help understand “how things are going”.

Obviously, founders can and should benefit from this “value in knowledge” as well. For them, though, growth accounting is of even bigger importance, because it brings something more to the table: a value in action.

Value in action

The idea is simple. Growth accounting (as financial accounting) is both a tool to understand the status quo, as well to act upon it.

Even more: it’s a precise methodology.

Growth accounting can help with several things. I’ll name and expand on a few here on.

In the case of consumer companies, it can:

  1. guide the company product development,
  2. keep growth model and business model aligned.

While for sales-driven companies, it can help:

  1. spot unsustainable long-term growth models by predicting future churn rates,
  2. spot unsustainable long-term sales models by proving (or disproving) product feasibility.

Let’s see why.

Growth Accounting for B2C companies

Prioritize Product Development

When you just started out, you don’t really know if your product does work.

In fact, I’d argue that the founder’s job is precisely to find a product-market fit that can be scaled.

How do you know if any given feature of the product is significantly contributing to reaching this fit?

Users retention cohorts can efficiently guide product development and provide a powerful way to prioritise features.

For example, let’s say that you have a social platform and you just introduced a way for people to upload and share pictures. You see that before rolling it out, only 5% of all the users acquired in a given week returned to the platform after 4 weeks. However, after its rollout, the 1-month weekly retention rate jumped to 25%. Users loved sharing pictures, so much that they got more likely to come back! You should prioritize the improvement of this feature.

 Cohort Analysis for product development prioritisation

Cohort Analysis for product development prioritisation

The process to make user retention cohorts is quite simple:

  1. you define a “user cohort” as all the users that you acquired in a given period, say a week
  2. you look at how many of those users used the product in the following periods – for instance, how many of them were active 7 days after they started using your product? and what about 3 weeks later?
  3. you confront those numbers with the numbers of previous and following cohorts, to see if your product is getting better or not over time, especially after rolling out important features.

Keep Growth Model and Business Model aligned

When building a self-serve product, sooner or later comes a radical choice: how big should the free-to-premium ratio be?

At one side of the spectrum, it could be 0%: free, and you’ll figure things out in the future (eg. Instagram when it launched). At the other, 100%: paid, no trial, no free period: all those who sign up end up paying (eg. Superhuman).

In the middle, there are two possibilities: time-constrained freemium models (like Netflix back in the days: 1 month free) or feature-constrained models (eg Slack or Strava).

Growth Accounting: Free to Premium funnel ratio

Free to Premium funnel ratio

Growth accounting can massively help companies align their product and business models.

A very low free-to-premium ratio has to be supported by a very high endogenous growth rate: if you have a very high value provided for free, your acquisition strategy should not be to bet heavily on paid acquisition: people—ideally—would have to come to the product because they found it by themselves, or because someone told them about it. Otherwise, the model can easily get unsustainable:

  1. growth correlates to acquisition costs (want more users? pay more), and conversion rates are too low to support such costs (too much value for free);
  2. trying to increase the conversion rate (ie. widening the funnel: give less for free and more if paid) increases acquisition costs too, and the model is still unsustainable.

Another example of misaligned growth and business model: let’s imagine a company with a very scarce free offering, say a meditation app with a free trial of 7 days. At some point, this company discovers some unexploited viral loop: they see that user cohorts invited by existing users are significantly more active and more likely to invite new users to the platform.

Free to Premium viral loop

Free to Premium viral loop

This company has the opportunity to increase the ability for its users to access the product for free, so that more people referred by existing users will sign up, propelling both growth and retention exponentially.

It’s a very delicate balancing effort, but a structured framework like this can make a difference when strategising over feature roadmap, growth model and revenue streams.

Why it’s relevant for B2B companies as well

A common thing I heard from (some) B2B founders is along the line of “Our success or failure lies on our sales side, how our customers use our product after we managed to sell it is not relevant at this stage for us”.

While another one: “We don’t need metrics to know if our service is useful or not: we have boots on the ground for it”.

Even though a human connection with the customer base has invaluable benefits, I still believe that growth accounting can, and should, be applied to sales-driven b2b companies as well.

Unsustainable growth

First, it can help both founders and investors spot unsustainable growth by predicting future churn rates.

A major red alarm should ring if a startup can distinctly see that no one cares about its product after being bought by a company. This can be seen by looking at product adoption and product usage.

Let’s consider, for instance, a B2B company selling a dashboard tool for insurance businesses that helps analyst price risks by leveraging AI. Let’s assume that this company just sold several annual contracts in the last few months, with a y/o/y growth in the order of 5x: some very impressive results.

However, their product adoption inside the risk management department of their customers looks like this:

B2B Adoption Rate

B2B Adoption Rate

On average only 10% of the whole department signs up for the product after the first week, and this percentage even decreases over time.

Their cohort analysis of those who signed up, then, looks like this:

B2B Cohort Analysis

B2B Cohort Analysis

That is: the probability that an employee, after signing up for the product, is still using it after a few weeks is below 5%, and decreasing over time.

A potential explanation for such behaviour can be that this company is doing exceptionally well in pitching to top management with spending power (maybe leveraging buzzwords like “AI” and “machine learning”), but its product is not being used at a fundamental level. Employees don’t even sign up for it, and those who do (maybe pressured by their bosses) stop using it after very little time.

The management of this startup should unquestionably work on making its product better since it’s very easy to predict an extraordinary churn rate getting closer and closer.

Unsustainable product

Another classic case is spotting an unsustainable product in the case of double-sided B2B2C startups.

The startup is marketing a free product to a consumer audience and re-selling that very audience to businesses. It’s the case of Freeda and EyeEm: I sell a service to companies because among my assets I have an audience of consumers. Typically, such services are—fundamentally—some form of advertisement.

The main premise of such a product is having an audience. I’ve seen plenty of pitches where the answer to the question “how do you get the users on board?” is “via paid acquisition”. While this can be a very easy way out of a tricky question, it can very easily fall into an unsustainable model, if those who you paid to acquire:

This means that the product sold is not sustainable, and it will most likely bring the sales costs up (because conversion rates will go down, ultimately due to a worsening product). A very likely downward spiral.

Let’s take, for example, the case of a platform for composers, where they can showcase their music, they can get likes, followers, etc. The business model of this platform is product placement, while the growth model on the user-side is getting composers into the platform via Facebook ads. Problem is that these composers massively churn, abandoning the platform quite soon, too soon to be monetized and to face the (business) competition coming from Facebook and Google.

That’s an unsustainable product, right there.

Putting together the CAC, engagement rate and churn/retention both on the B2B side as well as on the user side can make founders understand whether the model can stay afloat or not. Sometimes this can even mean changing business model altogether.

In other cases, this can help investors spot artificial / too capital intensive sales models that won’t scale in the future—avoiding a very likely write-off.

More advanced metrics and analysis

More advanced analysis can be performed when doing growth accounting, in particular on the side of “how do users use the app?” – for instance, by taking a look at marketplace liquidity, referrals mechanisms, funnel metrics, and so on.

However, such practises are very difficult to generalise, they can clutter the overall understanding of the big picture, and can bring to an incremental mindset that could be counter-productive if you’re early stage.

What’s next?

In the next post, I’ll write about how to very practically get this kind of numbers out of a product. No bells and whistles, no bubbling nor theory: a very practical overview of tools to implement what I discussed in this post.

If you want to be notified about when it’s going to be published, you can subscribe to my newsletter. It’s a non-lame 0-bullshit periodic newsletter on startups, tech-enabled scalability and data-driven impact:

If you know someone that can benefit from what I’ve written, send them the link! Sharing is caring 💌

If you have any feedback, be it positive or negative, I’d greatly appreciated if you’d let me know it! I heavily rely on them to get better, so thanks in advance. Tweet them @giansegato.