-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
222 lines (207 loc) · 21.1 KB
/
Copy pathindex.html
File metadata and controls
222 lines (207 loc) · 21.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Methodology research — feature ranking & metric definition</title>
<style>
:root{
--ink:#1c2230; --ink-soft:#465065; --line:#e3ddd0; --paper:#faf7f0;
--card:#ffffff; --accent:#b5482e; --accent-soft:#fbeae4;
--gold:#9a7b2e; --gold-soft:#f6efdc; --teal:#1f6f6b; --teal-soft:#e2f0ee;
--blue:#2d4a7c; --blue-soft:#e6ecf6; --purple:#5b3b78; --purple-soft:#efe6f6;
--shadow:0 1px 2px rgba(28,34,48,.06),0 8px 24px rgba(28,34,48,.06);
}
*{box-sizing:border-box}
body{margin:0;background:var(--paper);color:var(--ink);
font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
line-height:1.62;font-size:16px;-webkit-font-smoothing:antialiased}
.wrap{max-width:1040px;margin:0 auto;padding:0 24px 96px}
header.masthead{border-bottom:3px solid var(--ink);margin-bottom:8px;padding:48px 0 26px}
.kicker{font-size:12px;letter-spacing:.18em;text-transform:uppercase;color:var(--accent);font-weight:700;margin:0 0 10px}
h1{font-family:Georgia,"Times New Roman",serif;font-size:46px;line-height:1.05;margin:0 0 14px;letter-spacing:-.01em}
.sub{color:var(--ink-soft);margin:0;max-width:68ch;font-size:17px}
.goal{margin-top:22px;background:var(--ink);color:#f3efe6;border-radius:12px;padding:22px 26px;box-shadow:var(--shadow)}
.goal .lbl{color:#f3d9a0;text-transform:uppercase;letter-spacing:.1em;font-size:11px;font-weight:800;display:block;margin-bottom:6px}
.goal h2{font-family:Georgia,serif;font-size:23px;margin:0 0 6px;color:#fff;line-height:1.25}
.goal p{margin:6px 0 0;color:#cfc9bb;font-size:15.5px}
h2.sec{font-family:Georgia,serif;font-size:28px;margin:50px 0 8px;letter-spacing:-.005em}
.secsub{color:var(--ink-soft);margin:0 0 22px;max-width:68ch}
.how{display:grid;grid-template-columns:repeat(3,1fr);gap:14px;margin:18px 0 0}
.how .step{background:var(--card);border:1px solid var(--line);border-radius:12px;padding:18px 18px;box-shadow:var(--shadow)}
.how .step .n{font-family:Georgia,serif;color:var(--accent);font-size:32px;font-weight:700;line-height:1;margin-bottom:4px}
.how .step h3{font-family:Georgia,serif;font-size:17px;margin:8px 0 4px}
.how .step p{margin:0;font-size:14.5px;color:var(--ink-soft)}
.how .step a{color:var(--teal);text-decoration:none;border-bottom:1px solid var(--teal-soft);font-weight:600}
.how .step a:hover{color:var(--accent);border-color:var(--accent-soft)}
.group{margin:36px 0 0}
.group h3{font-family:Georgia,serif;font-size:21px;margin:0 0 4px;display:flex;align-items:center;gap:10px}
.group .pill{font-size:10px;letter-spacing:.1em;text-transform:uppercase;font-weight:800;color:#fff;padding:3px 9px;border-radius:5px}
.pill.s{background:var(--accent)}.pill.i{background:var(--teal)}
.pill.e{background:var(--blue)}.pill.g{background:var(--gold)}.pill.q{background:var(--purple)}
.pill.ailab{background:#3a6859}
.group .gdesc{color:var(--ink-soft);margin:2px 0 14px;font-size:14.5px}
.pgs{display:grid;grid-template-columns:repeat(2,1fr);gap:10px}
.pg{background:var(--card);border:1px solid var(--line);border-left:4px solid var(--ink);border-radius:8px;padding:13px 16px;box-shadow:var(--shadow)}
.pg.s{border-left-color:var(--accent)}.pg.i{border-left-color:var(--teal)}
.pg.e{border-left-color:var(--blue)}.pg.g{border-left-color:var(--gold)}.pg.q{border-left-color:var(--purple)}
.pg.ailab{border-left-color:#3a6859}
/* Card title link — only the direct-child <a> gets the bold block treatment.
Nested jargon links inside .pg .what would otherwise inherit display:block. */
.pg > a{color:var(--ink);text-decoration:none;font-weight:700;font-size:15.5px;display:block}
.pg > a:hover{color:var(--accent)}
.pg .from{font-size:12.5px;color:var(--ink-soft);margin-top:3px;letter-spacing:.02em}
.pg .what{font-size:13.5px;color:var(--ink-soft);margin-top:4px}
/* Jargon link-out — subtle, hover reveals accent. Mirror of framework.css a.j */
a.j{color:var(--ink);text-decoration:none;border-bottom:1px dotted var(--ink-soft)}
a.j:hover{color:var(--accent);border-bottom:1px solid var(--accent)}
.keyshelf{background:var(--gold-soft);border:1px solid #d8c98f;border-left:5px solid var(--gold);border-radius:10px;padding:18px 22px;margin:22px 0 0}
.keyshelf b{color:#7a5a08}
.keyshelf a{color:#7a5a08;font-weight:700;text-decoration:none;border-bottom:1px solid #c6ad55}
.keyshelf a:hover{color:var(--accent);border-color:var(--accent-soft)}
.note{background:#fff;border:1px solid var(--line);border-left:5px solid var(--ink-soft);border-radius:10px;padding:16px 22px;margin:24px 0 0;color:var(--ink-soft);font-size:14.5px}
.note b{color:var(--ink)}
footer{border-top:1px solid var(--line);padding-top:20px;color:var(--ink-soft);font-size:13px;margin-top:56px}
@media(max-width:780px){h1{font-size:34px}.how,.pgs{grid-template-columns:1fr}}
</style>
</head>
<body>
<nav class="sitenav">
<style>
.sitenav{position:fixed;top:12px;right:16px;z-index:100;font-size:13px;font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
.sitenav details{background:#1c2230;color:#f3efe6;border-radius:8px;box-shadow:0 4px 16px rgba(0,0,0,.22)}
.sitenav summary{list-style:none;cursor:pointer;padding:8px 14px;font-weight:700;letter-spacing:.04em;user-select:none}
.sitenav summary::-webkit-details-marker{display:none}
.sitenav summary::after{content:" \25BE";color:#cfc9bb;font-size:10px;margin-left:2px}
.sitenav details[open] summary::after{content:" \25B4"}
.sitenav .navmenu{position:absolute;top:calc(100% + 6px);right:0;background:#fff;color:#1c2230;border:1px solid #e3ddd0;border-radius:10px;padding:16px 18px;box-shadow:0 8px 28px rgba(28,34,48,.22);display:grid;grid-template-columns:repeat(2,minmax(180px,1fr));gap:14px 26px;min-width:400px;max-width:92vw;max-height:80vh;overflow-y:auto}
.sitenav .navgrp h4{margin:0 0 5px;font-size:10.5px;letter-spacing:.09em;text-transform:uppercase;color:#465065;font-weight:700;border-bottom:1px solid #e3ddd0;padding-bottom:4px}
.sitenav .navgrp a{display:block;color:#1c2230;text-decoration:none;font-size:13px;padding:2px 0;line-height:1.5}
.sitenav .navgrp a:hover{color:#b5482e}
.sitenav .navgrp a.cur{font-weight:700;color:#b5482e}
@media(max-width:600px){.sitenav .navmenu{grid-template-columns:1fr;min-width:240px;max-width:88vw}.sitenav{top:8px;right:8px}}
</style>
<details>
<summary>📑 Jump to</summary>
<div class="navmenu">
<div class="navgrp"><h4>Start here</h4>
<a class="cur" href="index.html"><b>← Home (goal & map)</b></a>
<a href="impact-saas-companies.html">SaaS / B2B field study</a>
<a href="impact-consumer-companies.html">Consumer-tech field study</a>
<a href="methodologies-comparison.html"><b>All methods compared →</b></a>
<a href="experiment-trustworthiness.html">How 40k tests actually work →</a>
<a href="jargon.html">Jargon (glossary)</a>
</div>
<div class="navgrp"><h4>Scoring</h4>
<a href="rice-framework.html">RICE (Intercom)</a>
</div>
<div class="navgrp"><h4>Define-first</h4>
<a href="north-star-framework.html">North Star (Amplitude / Slack)</a>
<a href="v2mom-framework.html">V2MOM (Salesforce)</a>
<a href="pyramid-of-clarity-framework.html">Pyramid of Clarity (Asana)</a>
<a href="heart-framework.html">HEART (Google)</a>
<a href="dibb-framework.html">DIBB (Spotify)</a>
</div>
<div class="navgrp"><h4>Written discipline</h4>
<a href="pr-faq-framework.html">PR-FAQ / Working Backwards (Amazon)</a>
<a href="stripe-shaping-framework.html">Stripe shaping</a>
</div>
<div class="navgrp"><h4>Experimentation (SaaS)</h4>
<a href="microsoft-exp-framework.html">Microsoft ExP / CUPED</a>
<a href="linkedin-xlnt-framework.html">LinkedIn T-REX</a>
</div>
<div class="navgrp"><h4>Experimentation (Consumer)</h4>
<a href="netflix-experimentation.html">Netflix · ABlaze</a>
<a href="booking-experimentation.html">Booking.com</a>
<a href="airbnb-erf-framework.html">Airbnb ERF</a>
<a href="uber-xp-framework.html">Uber XP</a>
<a href="doordash-switchback-framework.html">DoorDash switchback</a>
<a href="lyft-experimentation.html">Lyft</a>
<a href="pinterest-ab-framework.html">Pinterest</a>
</div>
<div class="navgrp"><h4>AI labs</h4>
<a href="anthropic-pm-on-ai-exponential.html">Anthropic · PM on AI exponential</a>
<a href="google-customer-zero-2026.html">Google · "Customer zero" 2026</a>
</div>
</div>
</details>
</nav>
<div class="wrap">
<header class="masthead">
<p class="kicker">Methodology research · for team review</p>
<h1>How to rank features and define metrics — a methodology survey</h1>
<p class="sub">A field study of how 18 of the most-studied software companies (plus two AI labs) actually rank features and decide which metrics matter. The point is to compare the published methods side-by-side so the team can pick (or mix) the one that suits us.</p>
<div class="goal">
<span class="lbl">Goal of this research</span>
<h2>Find a suitable method — to rank features and define metrics — that we can apply across the whole company.</h2>
<p>We're surveying what other successful product companies actually do (with sources), so we can have a grounded discussion about which approach (or which combination) to adopt. This is research material to read and discuss, not a recommendation.</p>
</div>
</header>
<div class="keyshelf">
<b>Shortcut for reviewers who want to skim:</b> The <a href="methodologies-comparison.html">methods comparison table</a> is the single most useful page — it lays out every method side-by-side with what each does, what it doesn't do, and where the evidence comes from. From there, click into the deep-dives for whatever looks promising.
</div>
<h2 class="sec">All pages, by category</h2>
<p class="secsub"><b>Three camps answer the core question</b> — <em>"100 features, 10 engineers, which 10 first?"</em>: <b>Scoring</b> (rank features with a formula or category), <b>Define-first</b> (name the metric or goal, features fall out by alignment), and <b>Written discipline</b> (a structured doc is the funding gate). Two more groups are kept on this page for context but answer different questions: <b>Experimentation</b> validates what was already shipped (post-build, not pre-build ranking), and <b>AI labs</b> publish build-side velocity claims, not prioritisation rubrics.</p>
<p class="secsub">Every method below is sourced; "From" is the company the method is documented at. For who-uses-what in real teams, see the <a href="impact-saas-companies.html" style="color:var(--teal);font-weight:700">SaaS / B2B</a> and <a href="impact-consumer-companies.html" style="color:var(--teal);font-weight:700">Consumer-tech</a> field studies. Sourcing caveats are on the deep-dive pages.</p>
<div class="group" id="scoring-group">
<h3><span class="pill s">Scoring</span> Scoring & prioritization frameworks</h3>
<p class="gdesc">Produce a score or a category to rank features. Numeric formulas (RICE, Opportunity Scoring, Cost of Delay) surface disagreement via Reach/Impact/Confidence-style trade-offs; categorical buckets (Kano, MoSCoW) sort features by satisfaction effect or delivery-timebox priority. RICE has a full deep-dive on this site; the other four are surfaced below with canonical external sources, deep-dives queued.</p>
<div class="pgs">
<div class="pg s"><a href="rice-framework.html">RICE — Reach × Impact × Confidence ÷ Effort</a><div class="from">From: Intercom</div><div class="what">The most-cited scoring formula. Fixed scales, transparent math, easy to audit.</div></div>
<div class="pg s"><a href="https://en.wikipedia.org/wiki/Kano_model" target="_blank" rel="noopener">Kano model — Must-be / Performance / Attractive ↗</a><div class="from">From: Noriaki Kano (1984, "Attractive quality and must-be quality")</div><div class="what">Five-category model sorting features by how their presence/absence affects customer satisfaction: must-be, one-dimensional, attractive, indifferent, reverse. Categorical, not numeric. <em>Canonical external source · deep-dive queued.</em></div></div>
<div class="pg s"><a href="https://en.wikipedia.org/wiki/MoSCoW_method" target="_blank" rel="noopener">MoSCoW — Must / Should / Could / Won't ↗</a><div class="from">From: Dai Clegg (1994, Oracle UK → DSDM Consortium)</div><div class="what">Four-bucket prioritization around a delivery timebox. Categorical and stakeholder-consensus-driven; widely used in Agile / DSDM / Scrum. <em>Canonical external source · deep-dive queued.</em></div></div>
<div class="pg s"><a href="https://en.wikipedia.org/wiki/Outcome-Driven_Innovation" target="_blank" rel="noopener">Opportunity Scoring (ODI) — Importance + max(Importance − Satisfaction, 0) ↗</a><div class="from">From: Tony Ulwick (HBR Jan 2002, "Turn Customer Input into Innovation")</div><div class="what">Customers rate each outcome's importance and current satisfaction (1–10); the formula surfaces high-importance / low-satisfaction outcomes as top opportunities. Numeric; built on jobs-to-be-done theory. <em>Canonical external source · deep-dive queued.</em></div></div>
<div class="pg s"><a href="http://leanmagazine.net/lean/cost-of-delay-don-reinertsen/" target="_blank" rel="noopener">Cost of Delay (CoD) — economic prioritization ↗</a><div class="from">From: Don Reinertsen (2009, "The Principles of Product Development Flow")</div><div class="what">Prioritize by the economic cost of postponement, often as CoD ÷ cycle time (WSJF). Treats prioritization as a queuing problem; lean-product-development origin. <em>First-party interview · deep-dive queued.</em></div></div>
</div>
</div>
<div class="group">
<h3><span class="pill g">Define-first</span> Pick the metric (or goal) before the work</h3>
<p class="gdesc">Name the target — a company-level outcome metric or a cascaded goal — <em>before</em> committing to build. Then "which 10 features first" becomes "the 10 that most move the named target." North Star is the company-wide outcome-metric variant; HEART, DIBB, V2MOM, and Pyramid of Clarity are different shapes of the same cascade discipline. Cheap discipline, high payoff.</p>
<div class="pgs">
<div class="pg g"><a href="north-star-framework.html">North Star metric & input model</a><div class="from">From: Amplitude, Slack</div><div class="what">One outcome number for the whole company; inputs cascade down to team-level <a class="j" href="jargon.html#okr">OKRs</a>. Features get picked by how much they move an input.</div></div>
<div class="pg g"><a href="heart-framework.html">HEART + Goals-Signals-Metrics</a><div class="from">From: Google (CHI 2010)</div><div class="what">5 user-experience dimensions, each laddered down to a specific metric.</div></div>
<div class="pg g"><a href="dibb-framework.html">DIBB — Data → Insight → Belief → Bet</a><div class="from">From: Spotify</div><div class="what">Every bet must trace back through evidence; antidote to "we just feel like it."</div></div>
<div class="pg g"><a href="v2mom-framework.html">V2MOM — Vision · Values · Methods · Obstacles · Measures</a><div class="from">From: Salesforce (Benioff, 1999)</div><div class="what">Top-down alignment cascade; every team's plan rolls up to the company V2MOM.</div></div>
<div class="pg g"><a href="pyramid-of-clarity-framework.html">Pyramid of Clarity</a><div class="from">From: Asana</div><div class="what">Mission → Strategy → Objectives → Key Results → Projects.</div></div>
</div>
</div>
<div class="group">
<h3><span class="pill q">Written Discipline</span> Make the team write the bet before they make it</h3>
<p class="gdesc">No formula, no number — just a structured document peer reviewers can engage with. Works only if reading and pushing back is taken seriously.</p>
<div class="pgs">
<div class="pg q"><a href="pr-faq-framework.html">PR-FAQ · Working Backwards</a><div class="from">From: Amazon</div><div class="what">Write the launch press release + customer FAQ <em>before</em> you build. If it's not compelling, don't build.</div></div>
<div class="pg q"><a href="stripe-shaping-framework.html">Shaping + deep-dives + long-term filter</a><div class="from">From: Stripe</div><div class="what">No scoring formula. Qualitative judgement reinforced by writing culture and a long-term direction.</div></div>
</div>
</div>
<div class="group">
<h3><span class="pill e">Experimentation</span> Post-build validation — "did it work?"</h3>
<p class="gdesc">These methods answer a different question from the rest of this page: <em>of the features you've already shipped, which actually worked?</em> They presume the prioritization decision is already made. Useful for the <em>next</em> round of picks, not the first one. The only category that produces a <em>causal</em> answer — at the cost of presuming you can ship and measure many variants in parallel.</p>
<div class="pgs">
<div class="pg e"><a href="microsoft-exp-framework.html">ExP · OEC · CUPED</a><div class="from">From: Microsoft (Bing, Office, Azure)</div><div class="what">The classic platform — single <a class="j" href="jargon.html#oec">Overall Evaluation Criterion</a> + variance reduction.</div></div>
<div class="pg e"><a href="linkedin-xlnt-framework.html">LiX → XLNT → T-REX</a><div class="from">From: LinkedIn</div><div class="what">Same discipline as Microsoft, told as a platform-investment story.</div></div>
<div class="pg e"><a href="netflix-experimentation.html">ABlaze + Causal Inference</a><div class="from">From: Netflix</div><div class="what">A/B for what's randomizable; quasi-experiments for what isn't.</div></div>
<div class="pg e"><a href="booking-experimentation.html">Booking's democratized experimentation</a><div class="from">From: Booking.com</div><div class="what">~25,000 tests/year. Hypothesis-first, <a class="j" href="jargon.html#srm">SRM</a>-checked, standard metric library.</div></div>
<div class="pg e"><a href="airbnb-erf-framework.html">ERF — Experimentation Reporting Framework</a><div class="from">From: Airbnb</div><div class="what">Shared metric <a class="j" href="jargon.html#dsl">DSL</a> + standard reports + variance reduction.</div></div>
<div class="pg e"><a href="uber-xp-framework.html">XP + universal holdout</a><div class="from">From: Uber</div><div class="what">Staged ramp 1→5→25→50→100% + quarterly cumulative-effect measurement.</div></div>
<div class="pg e"><a href="doordash-switchback-framework.html">Switchback experiments</a><div class="from">From: DoorDash</div><div class="what">For marketplace <a class="j" href="jargon.html#sutva">SUTVA</a> violations — randomize region × time-window cells.</div></div>
<div class="pg e"><a href="lyft-experimentation.html">A/B + time-split + bandits</a><div class="from">From: Lyft</div><div class="what">Three designs for three kinds of marketplace problem.</div></div>
<div class="pg e"><a href="pinterest-ab-framework.html">Pinterest's A/B platform</a><div class="from">From: Pinterest</div><div class="what">Lightweight config + cross-platform consistency + real-time monitoring.</div></div>
</div>
</div>
<div class="group">
<h3><span class="pill ailab">AI Labs</span> Capability-led, no published pre-build score</h3>
<p class="gdesc">Two on-the-record accounts from frontier AI labs in 2026. <em>Neither is a feature-prioritisation method.</em> Anthropic's is a personal essay from Cat Wu on five PM habits for a fast-moving model. Google's is a documented gap: a great deal said about <em>how</em> they build (velocity claims with verbatim Sundar Pichai quotes), nothing published about <em>how</em> they decide what to build, keep, or kill in the agentic era.</p>
<div class="pgs">
<div class="pg ailab"><a href="anthropic-pm-on-ai-exponential.html">PM on the AI exponential · five habits</a><div class="from">From: Anthropic (Claude Code)</div><div class="what">Short sprints, demos over docs, side quests <em>alongside</em> the roadmap, capability-first prototypes, daily-use the product. Not a method — habits for working when the model improves week-over-week.</div></div>
<div class="pg ailab"><a href="google-customer-zero-2026.html">"Customer zero" & the velocity story</a><div class="from">From: Google (Sundar Pichai, 2026)</div><div class="what">Verbatim 2026 build-side claims (75% AI-generated code, 6× faster migration, weeks-not-months training). No published decision methodology. Project Mariner shut down May 4, 2026 without public rationale.</div></div>
</div>
</div>
<div class="note">
<b>How this material was built.</b> Every claim about a company's published method links to that company's own engineering blog, an academic paper, or a published book. Where sourcing is secondary (e.g. Spotify's DIBB has no first-party Spotify post), that's flagged on the deep-dive page. Worked examples that aren't drawn directly from published material are explicitly labelled as illustrative.
</div>
<footer>
Methodology research deck · 2026. Built to inform a team decision about which method (or combination) to adopt. Not a recommendation — a comparable side-by-side view of what successful companies actually publish about how they do it.
</footer>
</div>
</body>
</html>