Choreography: from feedback to insight
The choreography that takes the mountain of loose feedback, support tickets, review comments, NPS responses, and turns it into an insight that changes the roadmap. AI groups and narrates; the judgment about what actually matters is still yours.
You have two hundred support tickets, eighty NPS responses with open comments, and a dozen app-store reviews piled up from the month. Tomorrow's the prioritization meeting and someone's going to ask "what are users complaining about most?". You know the answer is in there. You just can't read all of it by hand before your coffee gets cold.
The month's close finished last night. Contribution margin dropped four points against budget, but revenue hit target. The CFO wants a one-page memo by ten.
A client sends a service contract at five on Thursday afternoon asking for a review by Friday morning. You ask AI to draft the termination clause, it hands back polished text in seconds.
You have two hundred comments from a social campaign and need to deliver a sentiment summary to the board by tomorrow. Reading them one by one would take all day, and the real pattern is hidden in the noise.
The climate survey closed and your area dropped twelve points on the belonging axis. Tomorrow at nine you present a one-page action plan to the board, and the explanation is scattered across two hundred forty open responses.
You have two hundred support tickets, eighty NPS responses with open comments, and a dozen app-store reviews piled up from the month. Tomorrow's the prioritization meeting and someone's going to ask "what are users complaining about most, and what do we do about it?". You know the answer is in there, along with a bunch of one-off complaints that don't mean anything. The work of separating signal from noise, and deciding what makes the roadmap because of it, is yours.
This month's forecast closed below target, but the number of proposals sent hit a record. The sales director wants to understand, before the pipeline call, whether the problem is generation or conversion, and the answer is scattered across a hundred thirty opportunities in the CRM.
Delivery SLA blew up for the first time this half. Volume grew, but cost per order rose right along with it, and the explanation is scattered across reports from each hub.
The internal audit flagged three controls with adherence below the threshold. The risk committee wants a one-page memo by four in the afternoon, and the evidence is scattered across sixty control-test records.
The production error rate rose after the last deploy, but average latency actually improved. The CTO wants to know by ten whether to roll back, and the explanation is scattered across hundreds of log lines.
The new checkout flow's usability test wrapped up. Sixteen recorded sessions, each with fifteen minutes of people talking, and you need a summary by three to decide whether to ship the flow.
A board meeting is set for Friday, and the European expansion scenario still isn't a slide. You have dozens of analyst reports to synthesize into a clear recommendation.
Man, how many times have you opened a folder of feedback and felt that mix of "there's gold in here" and "I'm never going to read all of this"? Two hundred tickets, eighty NPS responses, a dozen reviews. You know the story is in there. You just can't pull it out before tomorrow's meeting.
Think about it: the problem was never the feedback. It's been sitting there since yesterday, each piece on its own. The problem is the distance between the pile of loose comments and the sentence that makes someone decide to change the roadmap. That distance is a choreography. It has steps, it has an order, and whoever dances out of order trips.
The core idea of this lesson. AI shortens the mechanical part of the choreography, what used to take your whole day now takes minutes, but the step that decides, separating what actually matters from what's just one-off noise, is still yours.
01The choreography has five steps, and the order isn't negotiable
Choreography is the right word because each step only works after the one before it. You don't write the insight before knowing what repeats. You don't decide what matters before grouping the feedback. Whoever skips a step delivers a pretty report that changes nothing on the roadmap.
There are five steps. Gathering the raw feedback, grouping by theme, asking "so what" (what actually matters), narrating the insight for whoever decides, and auditing before it becomes a decision. AI comes in strong on steps one, two, and four. You're irreplaceable on step three and step five.
02Steps 1 and 2: AI gathers the mountain and points out what repeats
Step one is pure grunt work. Gathering feedback from wherever it lives: Zendesk support tickets, open-ended NPS responses, app-store reviews, lost-deal notes. Before, you did this reading line by line, eyes hurting. Now you hand over the material and the instruction: "group by recurring theme and tell me the frequency of each group".
Step two is actually grouping. AI reads all of it and hands back the themes that repeat: "twenty-three tickets about checkout slowness", "twelve reviews complaining about excessive notifications", "eight NPS responses mentioning lack of integration". That's grouping, not conclusion. It's the X-ray tech pointing out the spot, not the diagnosis. Frequency is a fact. What it means isn't on the table yet.
03Step 3: "so what?" is where frequency isn't the same as importance
Here the choreography changes hands, and this is product's most treacherous step. The most frequent theme isn't always the most important one. Twenty-three tickets about checkout slowness sound loud, but if they come from three big accounts that open a ticket every week about the same unresolved thing, that's amplified noise, not a new signal. Meanwhile those eight comments about lack of integration might come from eight different enterprise accounts, each about to cancel. Low frequency, extremely high urgency.
This is the step that separates a PM from a ticket counter. AI gives you the count. You know which customer segment matters most, which complaint is a symptom of a bigger problem, which is just the usual noise. Simple frame: frequency is quantity, severity is impact. The product that decides by count alone prioritizes wrong with a pretty ruler.
Learn more: why the rarest comment is sometimes the most valuable
In a pile of feedback, the common pattern dominates the count, and it's easy to read that as "what matters most". But product research has a classic rule: the most frequent complaint is usually about a problem everyone already knows exists. The rare sentence, that shows up once or twice, in a different tone from the rest, is often where the pain nobody had named yet lives, or the real reason a big customer is about to cancel. AI is great at showing you what repeats. It doesn't know, on its own, that the outlier weighs more than the average. That's your reading.
04Step 4: the insight's narrative has one shape
Whoever decides doesn't want the pile of tickets. They want the short story in the order that's consumed fast: what users are saying, why it matters now, what to do about it. AI drafts a great first version of this storyline, because it's structure. You supply the reading from step three (what actually matters and why), it stitches the narrative together, and you adjust the tone and emphasis.
05Step 5: every quote attributed to a user goes through an audit
This is this module's golden rule, so read it slowly: no quote, no frequency number, goes into the insight without you checking the source. AI is great at summarizing and narrating, and it's perfectly capable of paraphrasing a comment until it says something the person never said, or inflating a theme's frequency because two tickets looked similar and weren't. In an insight that's going to become a roadmap decision, a made-up quote isn't a small mistake. It's a feature built on top of a voice that doesn't exist.
Auditing here is simple and fast: take the two or three quotes that support the main recommendation and check, against the original source, whether that person said exactly that. Also check whether the frequency number holds up when you reopen the original tickets in the group. If the insight says "twenty-three users asked for X", check that the twenty-three tickets exist and are actually about X.
Do it now
Take your most recent your real task, a pile of feedback that needs to become an insight (tickets, NPS, reviews, lost-deal notes). Run the choreography once:
- Gather: hand the raw material to AI with the grouping instruction ("group by recurring theme and tell me the frequency of each group").
- Group: ask for the themes that repeat most, with the count for each. Don't accept a conclusion yet, just the groups.
- So what: for each theme, write by hand a sentence about the customer segment behind it and the real urgency. High frequency isn't synonymous with high importance.
- Narrate: ask AI for the storyline in the order what they say, why it matters, what to do, feeding it your reading from step three.
- Audit: pick the insight's two strongest quotes and check, against the original source, whether that person said exactly that.
Time yourself. Compare it to how long this used to take.
Practice
1. In the choreography from feedback to insight, which step should NOT be outsourced to AI?
2. A feedback theme has high frequency (many tickets) but low real urgency, while another has low frequency but comes from enterprise accounts about to cancel. What's the correct read?
For the board
On the mountainthe AI gathers two hundred tickets and points at what repeats. That it does better and faster than you.
On the so whatfrequency is a fact, importance depends on who complained and why. That judgement is yours.
On the quoteevery line attributed to a user goes through audit before it becomes a slide.
Thanks for the feedback. It helps sharpen the next lesson.