It’s Monday, and an API that my service depends on broke. In fact, I spent the whole day working around it. And I’m still waiting for the team to fix it. Let’s hope they do it soon.

Or the whole thing could blow up. And I’ll have to start from scratch.

But that’s a problem for tomorrow so let’s forget about it for now. Because baby, there’s air in my chest and a fire in my belly!

Anyways, enough about me.

I am but a humble messenger of a life-changing message.

Let’s get into Power Users 💥

What are Power Users?

Power users are people that

• Love your product

• Are highly engaged

• Make your product more valuable

They are Reddit’s moderators. They are Delta’s frequent fliers. And they are Instagram’s bikini clad influencers and their armies of thirsty simps.

They are experts of your product, return customers, and your app’s social glue.

Power users matter.

Why do power users matter?

They spend a lot of time in the product or give it value in another way. They capture more attention, time, and money than the average user. Simply put, power users add an outsized amount of value to your application.

Power users are your application’s biggest promoters. They increase the value of your product. They dump $100 dollar bills on you by the wheelbarrow.

But power users serve different roles in different applications. And sometimes they’re not important at all.

How do you measure power users?

Introducing the Power User Curve

The “Power User Curve”—also known as the activity histogram—is a histogram of users’ engagement. Facebook’s growth team called it “L30”. L30 means the total number of days active over the last 30 days.

L7—last 7 days. You get the picture.

Here’s what a typical curve would look like for a successful social media application.

Notice the convex, “smile” shape.

On the left side, there’s a high percentage of users that visit once a month. This is typical. Most people are casual users. But on the right side, we have the people that return to the site every single day (30/30). This is a very active set of users.

One thing this metric does well is identify customer usage patterns. And teams define “usage” in many ways:

• Visits

• Logins

• Likes

• Messages

• Comments

• Subscriptions

• “Big Red Button Clicks”

You name it, we can identify patterns—and power users—for it.

But just because you can identify them, doesn’t mean they matter.

When do power users not matter?

It depends on the product.

The ideal shape of the curve itself will also differ across applications. For example, the shape of a healthy social media app will look different compared to a healthy banking app. Meaning, power users are not important in all contexts.

The importance of power users depends on the application type and its business model. If usage is not correlated with revenue or value, power users are less important.

For engagement-hungry products, they matter a lot:

• Messaging

• Social Media

• Project management tools

Power users are more important for apps that need attention and habit building. Think network-based and productivity applications like Instagram and Asana. In both cases, the application has wormed its way into your routine.

But when engagement is not important, it matters less:

• Banking app

• Tax software

What matters for a banking app is money in accounts. What matters for tax software is total users. So activity itself is not an important signal. In fact, it could be a sign that something is going wrong.

Let’s look at what different curves tell us about our product.

Different Curve Shapes

For these daily-apps a smile curve is important. It means you have a good number of new users, and some of them really stick. But if your curve looks like this, you have a problem:

For a social media app, a curve that looks like this says “high churn” and “bad product market fit”. Imagine Instagram was only for sharing pictures of steaming bags of garbage. Their curve would look like this. Curiosity may get you to visit but, in the long run, it’s not very appealing!

But if this was a benefits portal, it wouldn’t be too bad. This shape is fine for products that aren’t meant for daily use.

But what if the curve looks like this?

If you want people to use your product every day, this might be great. But it could also mean:

• Your advertising isn’t working

• No one is sharing your product

• The value of your product is not immediately clear

If your curve looks like this and you can’t increase the lifetime value (LTV) of your power users, growth will stall.

So, you’ve identified your power users. But what do they do?

Let’s take this information and do something with it.

Who are your power users?

L30 can help you measure how many power users you have, but it doesn’t tell much else. We need to figure out who they are and what they do. We need to get granular.

Segment your users

Try creating histograms for different kinds of users (aka avatars). Then track their activities.

For example, accountants have different activity patterns than individual filers. Accountants as a group are power users. But combining them with the entire user base isn’t very helpful.

To get better insight into each group, you should make separate curves for each.

It can also be helpful to track cohorts of users. Did they onboarded during a particular ad campaign? Did they latch onto an excellent feature?

Once you’ve identified who these core users are, figure out what buttons they’re pressing!

• What does activity look like?

• What are their favorite features?

• How do they use the product?

• What does their activity pattern look like?

• Is there friction in achieving their goal?

• How can we smooth out that process?

This is where things get fun.

Answering these questions can tell you:

• What product-market fit looks like

• Which features are actually important

• And how to define your feature roadmap

Take early Instagram. I bet the most active users were not addicted to playing with photo filters. They were addicted to liking posts, responding to comments, and following other users.

They can also help you design your next ad campaign. Do non-power users understand the use-case that power users do?

It can also inform your system’s performance requirements. Does your system have enough bandwidth to satisfy your power users?

What latency, memory, and reliability level do you need to make power users happy?

In fact, you will have to cater to your power users. They might be a special case because they use your product a lot. Often, their experience induces tail latencies. They have the highest percentiles of slow response times. For example, Amazon has response time requirements for the 99.9th percentile. The slowest 0.1% of requests must be less than a certain threshold.

This is only 1 in 1,000 requests. But customers with the slowest requests often have the most data on their accounts because they have the most purchases. In other words, they’re the most valuable customers.

From Designing Data-Intensive Applications:

“It’s important to keep those customers happy by ensuring the website is fast for them: Amazon has also observed that a 100 ms increase in response time reduces sales by 1%, and others report that a 1-second slowdown reduces a customer satisfaction metric by 16%” — Kleppmann, Martin

Power users are important. They’ve made the most purchases, content, etc. They’re your most valuable customers.

TLDR: Value and Product

If engagement is an important to your product’s value proposition, power users are important. That’s because they drive value to your product. In the form of:

• Revenue

• Attention

• Community

But when engagement doesn’t lead to value, power users don’t matter as much.

Identifying power users can tell you a lot about your product:

• Areas for growth

• Features that people love

• Performance requirements

Use a histogram to identify usage patterns. 30 day and 7 day scales are the most common.

And as always, please give me feedback on X. What do you want more or less of? Other suggestions? Please let me know. Just send a tweet to @JakeAndStake.

Much love.

Jake

Welcome to Hidden States, your curated source for AI news.

Nvidia challenged by Google, Intel, Qualcomm and Arm

Can Google, Intel, Qualcomm and Arm take down Nvidia? That’s the aim of their recent collab: The Unified Acceleration Foundation (UXL) Foundation.

Summary

• The leader in accelerated computing, Nvidia captured the AI dev community by making it easy to program GPUs using CUDA.

• But a group of tech heavyweights, including Google, Intel, Qualcomm, and Arm, just joined forces to challenge Nvidia's grip on the AI chip market.

• Their goal: Build an open-source software suite that allows AI code to run on any hardware regardless of chip architecture.

• They're building it on Intel’s OneAPI, a standard for accelerated computing across multiple GPUs. This competes directly with Nvidia’s CUDA platform.

  ---

Baidu to Power Apple AI in China

Baidu will provide the AI backend for Apple phones in China. Chinese state authorities have strict regulations regarding social media apps from other countries, given their heavy-handed censorship.

Summary

• Baidu is one of China’s leading technology giants, specializing in search, mapping, and artificial intelligence. They are China’s Google.

• Baidu beat out Alibaba, another hyperscaler interested in partnering with Apple.

• Apple is also in negotiations with Google to integrate Gemini with iPhones.

Anthropic’s Claude 3 Beats GPT-4

Claude 3 Opus now ranks #1 on the Large Model Systems Organization (LMSYS) Chatbot Arena Leaderboard.

Summary

• LMSYS Chatbot Arena is a crowdsourced open platform for LLM evaluations.

• The winner of the LLM space is still very much undecided. GPT-4 held the top spot since May 2023 before being dethroned by Claude 3.

• OpenAI is expected to release GPT-5 some time this year.

  ---

Amazon Doubles Investment in Anthropic

Amazon announced an additional $2.75 billion investment in Anthropic (on top of an initial investment of $1.25 billion) totaling $4 billion.

Summary

• This is Amazon’s largest outside investment to-date.

• Anthropic has raised $7.3 billion over the last year.

• Amazon will have a minority stake in Anthropic and will not hold a board seat.

• Anthropic will use AWS as its primary cloud provider, leveraging the cloud giant’s chips—Trainium and Inferentia.

• AWS, Anthropic, and Accenture are partnering to provide AI solutions to clients.

  ---

Databricks Unveils DBRX

DBRX is Databricks’ open source Mixture of Experts LLM. It’s a 132B MoE with 36B active parameters trained from scratch on 12T tokens

Summary

• Of the open source models, DBRX performs best in language understanding, programming, and math. (Comparisons include Mixtral MoE, Llama-2 70B, and Grok-1)

• Databricks outperforms models on efficiency. Inference is 2x faster than LLaMA2-70B and is less than half the size (40%) of Grok-1 in total and active parameter count.

  

  

Back in the game. I’m expanding into the broader technological frontier.

This is a short one, but I have a few larger articles lined up. They’ll just take some time to work through. In the meantime, here’s an update on Apple and their new multimodal model.

In an effort to not be forgotten—or maybe push their stock price back up—Apple has announced their own foray into AI. No, not an update to Siri. Though I know we’re all dying for one.

Apple published a paper unveiling their in-house multimodal model called MM1. For the uninitiated, a multimodal model is a model that can interpret many different inputs. Apple’s new family of models only used text and image data for training, but multimodal models could interpret audio and video too.

The largest MM1 model only has 30 billion parameters—very small compared to advanced models like GPT-4 (1.76 trillion) and Gemini Ultra (1.56 trillion). Parameters are internal variables (weights) that are used to compute the end prediction. In essence, the more parameters, the more sophisticated the pattern recognition and output accuracy.

However, there is a trade-off. The larger the model, the more computation required. That’s why companies large and small are hungry for Nvidia GPUs.

But with a model like this, Apple gets to lean into their strengths (client hardware) and avoid their weaknesses (cloud computing). Most consumer hardware doesn’t have a huge GPU, so smaller models are more reasonable to run locally. I think Apple creating a model like this (and contributing to open source) is a smart move, even if this was probably all they could muster given their late entry.

Apple already has the brand and some of the best consumer chips. Can small language models be good enough for users? I think not. LLMs aren’t even great/accurate/etc. right now. We’ll probably see thin clients for the next few years, but who knows. It’s still very early.