B2B Prototyping part 4: How to measure attention and engagement

Welcome to Attention Matters, the newsletter from Storythings which gives you practical insights and tools on how to tell better stories and grow your audiences’ attention.

In today’s newsletter, I’m going to look at how to measure attention and engagement. I’m going to focus on looking for actual human behaviours, not just platform metrics, and what we can learn from content subscription models. Talking of which - we’ve been hovering just under 2,000 subscribers to Attention Matters for the last month or so, which means you’re part of a really lovely, growing community. If you like reading our series, and someone you know or work with would find it useful, please forward it on to them. And if there’s any feedback or suggestions for how I could make it more useful, please let me know!

Right then. Before I start talking numbers, here’s a recap of where we are in the series:

- Create space in busy schedules to build and test prototypes
- Collaborate with internal and external partners on prototypes 
- Test these ideas with your internal and external audiences 
- Create metrics to measure attention and engagement (that’s this post!)
- Scale ideas from prototypes into full projects
- Know when to stop or kill and idea, even if you love it
- Demonstrate how prototyping saves your company money and time
- Build a sustainable workflow and culture for prototyping in your team

THE PROBLEM

I mentioned last week that I’m an absolute nerd about how we measure attention. In fact, I wrote a series back in 2023 for Attention Matters about key digital metrics and what they really mean. That series showed that a lot of digital metrics end up telling you more about the platforms than your actual users. They actually obscure your understanding of the real humans you’re trying to reach, and focus more on getting you to do what their algorithms, and business models, want you to do.

I could write a lot more about this, but in essence, measuring audiences over the last hundred or so years has centred around two problems. In the broadcast era, we had tiny amounts of data (generated by small samples) but they were produced independently and most people in the industry believed them. In the digital era, we have huge amounts of data (generated by billions of users) but they are mostly produced by the platforms themselves, and there is a lot of scepticism and mistrust. If you want to dive deeper into this, I highly recommend Tim Hwang’s book Subprime Attention Crisis. It’s smart, useful and short.

So we’re in an era where the platforms stand between us and our audiences, and they mark their own homework. This is not a good thing.

THE INSIGHT

Although digital platforms still dominate the market for our attention, there has been a strong move amongst content businesses away from the so-called Traffic era (as described in Ben Smith’s brilliant book) towards more direct relationships with audiences. This move has been led by subscription-based businesses, whether they are newsletters, publications, podcasts or video series. The focus is on understanding exactly what value propositions get users to subscribe to a new service, and making sure that they continue to get enough value to stay with you.

Instead of looking at platform based engagement metrics - likes, views, follows and shares - they focus on business based metrics - acquisition, retention and churn. By focusing on direct transactional behaviours, you’re getting a much more effective feedback loop from your audience. It’s been said that Netflix’s most important metric is the name of the first show a new subscriber watches after they’ve taken out a new subscription. That was clearly the show that they valued enough to get over the hump of the subscription process and into the Netflix service.

Someone taking out or cancelling a subscription is a genuine, human action, driven by a conscious choice they’re making about your content. It’s a much more complex thing to measure than the swathes of engagement data platforms slough off and present to you. But even if you’re not asking users to pay for your content, it’s worth designing your content metrics around direct, measurable, human behaviours. These will do the key thing that all audience metrics need to do: make you even more curious about your audience.

THE ACTION

I talked last week about how user testing should focus on qualitative questions about how your content could fit into people’s lives. This post from FT Strategies about how they focus on subscriber retention is a great example. I’ve included a graphic summarising how they think about the audience journey over time, and there’s a couple of things I wanted to highlight as you develop your own audience metrics.

1 - Look for signs of commitment
The north star of your audience metrics should be something that is real evidence of a commitment. A lot of our online behaviours are exploratory and casual - searching, swiping and scrolling. Platform metrics often interpret this behaviour as intentful, but in reality, it’s just exploring. The behaviours you need to focus on must be deliberate, not casual. Using the Acquisition, Retention and Churn model, ask yourself what ‘Acquisition’ means for your content formats. Can you create an option for users to directly commit to your format, through some form of subscription? What is the behaviour that shows, beyond doubt, a user wants to see more?

2 - Look for early signs of habit forming
The FT Strategies article says that the first 90 days of a new subscription is crucial to retention. Research by the German newspaper Die Welt shows that 50% of new subscribers leave in the first three months, but after that, churn drops to 1-2% per month. So think about how you can help build habits - can you help them by creating an onboarding process that drives them to archive articles? Can you give them a daily content gift for the first week? Or could you ask them to design their experience so they get your content at a day or time that suits them?

3 - Ask people how they use your content
Talking of habits, you should make audience surveys a regular habit, so you can find out what people really value in your content formats. This doesn’t need to be a huge statistically significant research project. The aim here is not to have lots of people telling your they love your content, but to find out more about how it delivers value for them. So don’t ask them to rate you, ask them where they fit you into their lives, what they would like more of, or how they share your content. I’d much rather have lots of data about use than about appreciation. This will help you develop the ‘retention’ stage of your audience model.

4 - Ask one question when people drop out
Churn data is probably the most valuable, as you’re learning about another very inentful and tangible decision. After being a subscriber for a while, someone has decided, for whatever reason, that they don’t get enough value to stay subscribed. That might hurt personally, but it’s not about you. Ask for a bit of feedback about why they subscribed, and what drove that action. Then you have valuable data to feed into the next iteration of your content format, and the opportunity, as the FT Strategies article points out to Re-Acquire them as a subscriber in the future.

As I was writing this newsletter today, I took a break to present an audience framework we recently delivered for a Storythings client, and the Q&As afterwards were about quite a few of the issues I covered in today’s episode. In particular, I spoke a lot about how audience models and frameworks are not there to replace our curiosity about our audiences, but to enhance our curiosity. If you find that you have models for your audience in your organisation that have become lazy shorthands for making decisions, it means you need to get curious again.

If you want help staying curious, we’d love to help. Hit the big button below!

Next episode, we’ll look at how to scale prototypes from R&D projects into the big wide world. This is not simple and I’ve failed at it many times in my career. But that’s the point of prototyping - as a great writer once said: “Try again. Fail Again. Fail Better.”

See you next week!
Matt