It was a Tuesday morning in January. I opened my laptop, pulled up Mixpanel, and looked at the numbers. They were good. Monthly actives were up. Trial signups had the best week in three months. Session length was climbing. The graph was pointing in the right direction.
I felt nothing.
Not satisfaction. Not relief. Not even the mild pleasure of seeing a good number. Just — nothing. And that absence of feeling was more informative than anything in the dashboard.
What we were actually measuring
Here's what was in that Mixpanel dashboard:
- Daily active users
- Monthly active users
- Trial signups (weekly + monthly)
- Session length (median)
- Feature adoption rate (which features get used)
- Onboarding funnel (where people drop off)
- Retention cohorts (D7, D30, D90)
These are perfectly standard SaaS metrics. There's nothing wrong with them individually. But looking at them all together that Tuesday morning, I realized something: none of them told me whether any of our customers' humans were saving time because of us.
That's supposed to be the whole point. A customer's engineer should be spending less time on administrative overhead because the Cyborg handles it. A customer's founder should be sleeping better because the Cyborg filed the report at 6am before anyone woke up. A customer's team should be having more substantive conversations because the Cyborg handled the coordination.
Was that happening? The dashboard had no idea. It could tell me people were logging in. It couldn't tell me if their working lives were actually better.
The delete
I closed the laptop, went for a walk, and came back with a decision: delete it.
Not archive it. Not hide it. Delete the dashboard. All of it.
I'm not going to pretend this was a calm, considered decision. It was partly frustration — the frustration of having built something that was technically tracking things but not answering the question I actually cared about. And partly it was just wanting to force a rethink, to make the absence of a dashboard uncomfortable enough that we'd build the right one instead.
So I deleted it. Our growth dashboard went from a Mixpanel interface with seven charts to a blank screen.
What happened next: nobody noticed for three days. Then one person on the team mentioned it. I told them what I'd done and why. They thought about it for a moment and said, "Yeah, I never checked it anyway." That was also informative.
Why session length is a terrible metric for Aliens
After the delete, I spent a week thinking about what metrics would actually tell me the right things. And I kept coming back to one issue: for Aliens, many of the standard metrics point in the wrong direction.
Take session length. For most SaaS products, longer sessions are better — it means users are engaged. For Aliens, a long session by the human on the Cyborg dashboard could mean one of two things: either they're getting a lot of value from reviewing the Cyborg's work (good), or the Cyborg isn't autonomous enough and requires constant supervision (bad). You can't tell which without asking.
Or take daily active users. If our customers are logging in every day to check on their Cyborg, that might mean the Cyborg is creating dependency. What I actually want is for customers to check in once a week, feel confident everything is running well, and go do something more important. A great week for Aliens might look like low DAU. A standard dashboard would flag that as a problem.
The metrics optimized for engagement were actively misaligned with what Aliens is supposed to deliver. We're supposed to free people up. Measuring how often they use us is the wrong question.
The 5-question Friday rubric
We rebuilt our weekly review around five questions. These aren't metrics we pull from a database — they're questions we answer by talking to customers, reading the Cyborg's daily reports, and checking the audit logs. Every Friday, five questions:
THE FRIDAY RUBRIC · V1.2
| # | Question | What a "yes" looks like |
|---|---|---|
| Q1 | Did any customer's human save 5+ hours this week because of their Cyborg? | A customer tells us, or we can see it in the Cyborg's output log — hours of tasks completed that would otherwise have required human time |
| Q2 | Did any Cyborg fail in a way that embarrassed a customer? | No tickets, no escalations, no apology emails sent — or if it did happen, we caught it before the customer did |
| Q3 | Did we ship something new this week that we're proud of? | A feature, a fix, a playbook improvement, a connector — something that makes the product meaningfully better |
| Q4 | Did we say no to something this week that would have compromised quality? | A feature request we declined, a customer ask we pushed back on, a shortcut we refused to take |
| Q5 | Would a customer recommend us to their investor? | We ask this directly in monthly check-ins — the answer is yes, or we know specifically why not |
These five questions take about 45 minutes on a Friday afternoon. They require actual thinking — you can't auto-populate them from a database. Q1 requires talking to customers or reading their Cyborg's daily reports. Q2 requires checking the error logs and support queue. Q3 requires the team to have a shared sense of what "proud" means. Q4 requires someone to have actually said no to something. Q5 requires honest feedback from people paying us money.
The point of making them hard is that easy questions get easy answers. If I can pull a metric in two seconds, I'll check it in two seconds and move on. If I have to actually think about whether a customer saved meaningful time this week, I'll actually think about it.
What this changed
Three things changed after we switched to the Friday rubric.
First, our conversations with customers changed. We started asking different questions during check-ins. Not "how often are you using the dashboard?" but "did the Cyborg save you time on anything notable this week?" Those conversations surface information that no dashboard can capture: the Monday morning the Cyborg had prepared the board report by the time the founder woke up, or the week the Cyborg caught an error in a customer invoice before it was sent.
Second, our engineering priorities changed. When Q1 ("did a human save 5+ hours?") is the primary question, the features that look good on an activity dashboard become less interesting. What becomes interesting is reliability — does the Cyborg actually complete tasks, or does it get stuck? Does the daily report actually reflect what happened? Does the kill switch actually work in under 30 seconds? None of those things show up in session length metrics.
Third, I started sleeping better. Not because things got easier — we had plenty of difficult weeks — but because I stopped optimizing for the wrong things. When you're optimizing for "does the human who hired this Cyborg have more time for the work they actually want to do?" you know when you're succeeding. The answer is yes or no, and you can find out by asking.
On the metrics we still track
We do still track some numbers. Revenue, obviously. Churn. Onboarding completion rate (because if people don't get through onboarding, we can't help them). Infrastructure uptime. Time-to-halt for the kill switch (tested quarterly). And we track what the Cyborgs actually output: tasks completed, tasks blocked, escalations raised, daily reports delivered.
But we don't track DAU. We don't track session length. We don't track feature adoption as a primary metric. And we don't have a growth dashboard that makes us feel good when the line goes up, regardless of whether the underlying thing the line is measuring matters.
The best dashboard is the one that makes you think, not the one that makes you feel good about inaction.
A dashboard that tells you engagement is up is comfortable. It gives you something to point to. A rubric that asks "did a human's working life actually get better this week?" is uncomfortable — because some weeks the honest answer is no, and you have to sit with that. But uncomfortable questions are the ones worth asking. The comfortable questions are how you end up building the wrong thing for years while the graphs look great.