Metrics and Channels

Before I begin, I apologize for some of the formatting in this post. WordPress’ auto formatting is absolutely not cooperating and, as far as I know, there is no way to turn it off. Shipped is better than perfect, though, so here we are anyways.

I recently watched a presentation by Sean Ellis on driving sustainable growth at a startup. It is rather old (about 2 years) but I think there is some value that can still be extracted from it.

In this presentation, Sean breaks up the growth of a startup into 3 phases:

  1. Product/Market Fit
  2. Finding an efficient conversion mechanism
  3. Scale

For the purpose of this post, I’m going to assume that Vidyard has found a product/market fit and is now trying to develop a growth model that is scalable. So, the way I look at it, my role at Vidyard would be to help the company with #2 and #3 on the above list.

In this phase of growth, the key is to find the channels that are the best at converting with a good ROI. Once those channels have been determined, it’s time to focus resources on them. This means lots of experimenting, testing, and tweaking. One thing I’ve done to help me along this path is sign up for this KISS Metrics webinar on smarter A/B testing and this General Assembly presentation on user acquisition.

What to Measure?

To determine where to focus your resources, you must first determine what metrics you want to use to compare said channels. Choosing the right metrics to measure is as critical as finding the right channel. If you aren’t measuring the right things, then you could be optimizing for the wrong goals.

The set of metrics you measure is partially dependent on what channel you’re looking at. For example, measuring the cost of user acquisition in dollars probably doesn’t make sense for your blog (unless you’ve paid for a guest post or something of that nature).

You should be asking yourself questions about each channel and then choose the right metrics to help you answer those questions. Starting with questions makes sense to me because it forces you to think about why you’re using a particular metric and you’re not just measuring something for the sake of measuring it. Here are some common metrics framed with questions.

Where did this user come from?

Source of traffic

What percent of users are leaving this week/month/year?

Churn rate (you can go DEEP with this)

How much traffic is coming from this channel?

# of unique visitors

What percent of visitors signed up for a trial?

% visitors to trial

What percent of trial users ended up purchasing a product?

% trial to paid

How much is a user from this channel worth, on average?

Average lifetime value =  lifetime x average gross revenue per user

How much did it cost us to get each user?

Cost of acquisition in $ or in terms of time spent

And so on…

Here are a couple of specific things that I think will not only help with growth but will also allow us to get a better understanding of how the product is actually used by the customers. The following list is by no means exhaustive, just a starting point for deeper exploration and questioning.

What’s the difference between users who sign up for a paid plan after the trial and the users who leave after the trial is over?

Are there any specific things that one does that the other does not?

Amount of time spent using the application?

How qualified is the lead?

What’s the difference between long-term and short-term customers (I’m going to pick 90 days arbitrarily)?

Do they have different levels of success with the product in the first 90 days?

How do the customers define success?

How did they use the product differently, if at all?

What kind of user is brought in from a particular channel?

Content creators vs. video marketers vs. content curators

Do different channels yield users with different use cases?

How can we customize the experience to maximize the value from a particular user type? (eg. targeted landing pages if we find that the majority of leads from a particular channel are of a particular type)

The Channel

When looking at channels themselves, there seem to be two types. First there are those that are obvious such as SEO, SEM, social, blogs, and so on. Then there are those that are less obvious but much more clever. These are often built into the product as a distribution mechanism and are created in-house.

An amazing example of this is how AirBnB essentially piggybacked onto Craigslist’s established distribution network. Things like this are very special and get me excited about growth hacking because this is where the creativity really comes in. With that said though, startups have become successful without a clever growth hack so I guess it could simply come down to a matter of necessity.

Finally, in order to separate the wheat from the chaff, you’ve got to measure the channels independently of one another. This allows you to see which channel contributes the most to unique visitors, conversions, and so on. There’s a good post over at the Intercom blog that goes a bit deeper into this.

So that wraps up a rather conceptual post that summarized some of the things I’ve recently learned. Trying to position Vidyard within Sean Ellis’ framework was a great exercise and it would be awesome if I’ll be able to apply some of this in the near future.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s