Metrics for Early Stage Startups: A Practical Take

November 2019 · 11 minute read

This series on “Product-Market Fit Accounting” is sponsored by Zanichelli Ventures, a leading Italian Ed-Tech venture fund.

There’s this beautiful concept that Jonathan Hsu, from Tribe Capital, wrote down some time ago and stuck with me.

It more or less goes like this: Hundreds of years ago, some Italian monks devised a very structured system with which they could assess companies financial health, and that still today we use for the same reason: accounting.

Startups are fundamentally different from traditional companies though, so we had to come up with a set of accounting rules that could assess the “growth health” of a company as well: an accounting for growth.

With growth accounting, we usually mean a set of product KPIs and measurement rules that can help both investors and founders understand how a startup product grows, sticks, and gets used. Ultimately, this can become a powerful proxy for future cash flows when (and if) associated with a growth model and a business model.

Unfortunately, in my experience, this is not a practise as widespread as it should be.

On one side, EU early-stage investors tend to err more on the side of the financial assessment than on the product one – and thus, quite understandably, few founders are incentivised to proactively make product assessment either.

On the other, growth accounting can look like a daunting task, where you have to be a data scientist, a programmer and a business-savvy nerd all at the same time to pull it off. For sure no university or MBA is teaching how to do serious product due diligence. And it looks like that most of the learning content out there is more into theory than into practical tips.

The core concepts, however, are somewhat trivial, and their implementation is super smooth thanks to products like Firebase, Google Analytics and Mixpanel. Then again, it’s more critical than ever to learn such discipline after so many accelerators imploded, and their trickle-down know-how got lost.

First thing first: What is growth accounting?

I think it’s very important to start by laying a common ground on what we’re talking about.

Hsu does really go deep into the rabbit hole in defining and outlining growth accounting, as well as other people like Andrew Chen do (I profoundly admire both of them, by the way). I had a few (old fashioned) accounting class during college, and I can see Hsu’s point in structuring its framework in such a way.

Even so, I nonetheless believe that early-stage founders should focus more on the fundamentals, especially if they’ve never been exposed to a data-driven approach to strategy and due diligence.

Broadly speaking, with fundamentals I mean creating a structured and standardised way of measuring:

Why is this relevant for investors and founders alike?

For investors, it’s easy to make the case of learning growth accounting (even this simplified and watered-down version): spotting and vetting product growth is the easiest (only?) way to assess the future potential of a company.

In an illiquid market like the one startup investors are operating in, you can’t rely on market signals to price a company. You can’t rely on future discounted cash flows either, because you have no idea of their size or magnitude. And you’re in a market with scarce and asymmetric information too, which makes traditional accounting tools useless.

Long story short, product growth due diligence is crucial to spot the alpha faster than the competition and avoid write-offs more efficiently.

For founders, though, this methodology is of even bigger importance.

In the case of consumer companies, growth accounting can help:

  1. guide the company product development,
  2. keep growth model and business model aligned.

While for sales-driven companies, it’s what can help:

  1. spot unsustainable long-term growth models by predicting future churn rates,
  2. spot unsustainable long-term sales models by proving (or disproving) product feasibility.

Growth Accounting for B2C companies

Product Development

When early-stage, you don’t really know if your product does work. In fact, your job is precisely to find a product-market fit that can be scaled.

How do you know if any given feature is significantly contributing in reaching this fit? Should you invest more resources on making it better, or on the contrary just kill it and move to the next one?

Users retention cohorts can efficiently guide product development and provide a powerful way to prioritise features.

The process is quite simple:

  1. you define a “user cohort” as all the users that you acquired in a given period, say a week
  2. you look at how many of those users used the product in the following periods – for instance, how many of them were active 7 days after they started using your product? and what about 3 weeks later?
  3. you confront those numbers with the numbers of previous and following cohorts, to see if your product is getting better or not over time, especially after rolling out important features.

For example, let’s say that you have a social platform and you just introduced a way for people to upload and share pictures. You see that before rolling it out, only 5% of all the users acquired in a given week returned to the platform after 4 weeks. However, after its rollout, the 1-month weekly retention rate jumped to 25%. Users loved sharing pictures! Improving such feature should then be prioritised.

Growth Accounting: Cohort Analysis for product development prioritisation

Growth Accounting for keeping Growth Model and Business Model aligned

When you’re building a self-serve product, you’re usually confronted with a radical choice: which size should your free-to-premium ratio have?

At one side of the spectrum, it could be 0%: free, and you’ll figure things out in the future (eg. Instagram when it launched). At the other, 100%: paid, no trial, no free period (eg. Superhuman).

In the middle, there are two possibilities: time-constrained freemium models like Netflix back in the days (1 month free) and AWS (1 year free), or feature-constrained models like Slack or Strava.

Growth Accounting: Free to Premium funnel ratio

Growth accounting, in this regards, can massively help companies have their product and business models aligned.

A very low free-to-premium ratio has to be supported by a very high endogenous growth rate: if you have a very permissive product and a very high value provided for free, your acquisition strategy should not be to bet heavily on paid acquisition: people – ideally – would have to come to the product because they found it by themselves, or because someone told them about it. Otherwise, the model can easily get unsustainable:

  1. growth linearly correlates to acquisition costs (want more users? pay more), and conversion rates are too low to support such costs (too value for free);
  2. trying to increase the conversion rate (ie. widening the funnel: give less for free and more if paid) inversely affects the acquisition costs, and the model is still unsustainable.

Another example of misaligned growth and business model: let’s imagine a company with a high free-to-premium ratio (for example, a meditation app with a free trial of 7 days). At some point, this company discovers some unexploited viral loop: they see that user cohorts invited by existing users are significantly more active and more likely to invite new users to the platform.

Growth Accounting: Free to Premium viral loop

This company has the opportunity to lower their funnel ratio, that is increasing the ability for its users to access the product for free, so that more people referred by existing users will sign up, propelling both growth and retention exponentially.

It’s a very delicate balancing effort, but a structured framework like this can make a difference when strategising over feature roadmap, growth model and revenue streams.

Why it’s relevant for B2B companies as well

A common thing I’ve heard from (some) B2B founders was along the line of “Our success or failure lies on our sales side, how our customers use our product after we manage to sell it is not remarkably relevant at this stage for us”.

While another one: “We don’t need numbers to know if our service is useful or not: we have boots on the ground for it”.

Even though a human connection with the customer base has invaluable benefits, I still believe that growth accounting can, and should, be applied to sales-driven b2b companies as well.

Unsustainable growth

First, it can help both founders and investors spot unsustainable growth by predicting future churn rates.

A major red alarm should ring if a startup can distinctly see that no one cares about its product after being bought by a company. This can be seen by looking at product adoption and product usage.

Let’s consider, for instance, a B2B company selling a dashboard tool for insurance businesses that helps them monitor the risk rate of their assets thanks to AI. Let’s assume that this company just sold several annual contracts in the last few months, with a y/o/y growth in the order of 2-3x: some very impressive results.

However, their product adoption inside the risk management department of their customers looks like this:

Growth Accounting: B2B Adoption Rate On average only 10% of the whole department signs up for the product after the first week, and this percentage even decreases over time.

Their cohort analysis of those who signed up, then, looks like this:

Growth Accounting: B2B Cohort Analysis

That is: the probability that an employee, after signing up for the product, is still using it after a few weeks is below 5%, and decreasing over time.

A potential explanation for such behaviour can be that this company is doing exceptionally well in pitching to top management with spending power (maybe leveraging buzzwords like AI and machine learning), but its product is not being used at a fundamental level. Employees don’t even sign up for it, and those who do (maybe pressured by their bosses) stop using it after very little time.

The management of this startup should unquestionably work on making its product better since it’s very easy to predict an exceptionally high churn rate getting closer and closer.

Unsustainable product

Another classic case is spotting an unsustainable product in the case of double-sided B2B2C startups.

The startup is marketing a free product to a consumer audience and re-selling that very audience to businesses – the case of Freeda and EyeEm: I sell a service to companies because among my assets I have an audience of consumers. Typically, such services are, on a fundamental level, some form of advertisement.

The main premise of such a product is having an audience. I’ve seen plenty of pitches where the answer to the question “how do you get the users on board?” is “via paid acquisition”. While this can be a very easy way out of a tricky question, it can very easily fall into an unsustainable model, if those who you paid to acquire:

This means that the product sold is not sustainable, and it will most likely bring the sales costs up (because conversion rates will go down, ultimately due to a worsening product). A very likely downward spiral.

Putting together the CAC, engagement rate and churn/retention both on the B2B side as well as on the user side can make founders understand whether the model can stay afloat or not. Sometimes this can even mean changing business model altogether.

In other cases, this can help investors spot artificial / too capital intensive sales models that won’t scale in the future – avoiding a very likely write-off.

More advanced metrics and analysis

More advanced analysis can be performed when doing growth accounting, in particular on the side of “how do users use the app?” – for instance, by taking a look at marketplace liquidity, referrals mechanisms, funnel metrics, and so on.

However, such practises are very difficult to generalise, they can clutter the overall understanding of the big picture, and can bring to an incremental mindset that could be counter-productive if you’re early stage.

What’s next?

In the next post, I’ll write about how to very practically get this kind of numbers out of a product. No bells and whistles, no bubbling nor theory: a very practical overview of tools and code snippets to implement what I discussed in this post.

If you want to be notified about when it’s going to be published, you can subscribe to my newsletter. It’s a non-lame 0-bullshit periodic newsletter on startups, tech-enabled scalability and data-driven impact:

If you know someone that can benefit from what I’ve written, send them the link! Sharing is caring 💌

If you have any feedback, be it positive or negative, I’d greatly appreciated if you’d let me know it! I heavily rely on them to get better, so thanks in advance. Tweet them @giansegato.