The Complaint Rate and the Days That Stand Out
Part 4 of the "Behind the Map" series — one shared rule for "this deserves a second look"
Abstract
A star average moves sluggishly — a single bad day gets absorbed into it long before it noticeably shifts the mean. This article shows how YelmeAtlas defines a complaint, how a complaint rate gets computed over a time window, and the one arithmetically simple rule that marks a day as "unusual" — the same rule that feeds both the API and the PDF report, so the two can never disagree.
1. Diagnosis: the average buries events
An app sitting at an overall 4.3-star average can still have had a single, dramatic day last week — after a broken update, say. The overall average integrates over months or years; a single bad day barely moves it. Anyone watching only the average notices the event, at the earliest, once the damage is already done — or not at all, if things settle back down afterward.
Two questions stay unanswered as long as only one overall average exists: "is the situation worse than usual right now?" and "was there a day that stood out from the pattern?"
2. Existing approaches and their limits
- Watch the overall star average. Reacts too slowly to single events (see above).
- Manually scan every day. Doesn't scale — exactly the problem from part 1 of this series.
- A fixed alert threshold ("more than 5 complaints in a day"). Ignores app size: 5 complaints is a real outlier for a small app and noise for a large one.
3. The YelmeAtlas approach
Two layers, cleanly separated: first, a clear, simple definition of what even counts as a complaint; second, a relative, size-independent rule for when a day counts as unusual.
Layer 1 — the complaint. A review with 2 stars or fewer counts as a complaint. No text analysis, no interpretation — a number any customer can re-derive themselves without understanding the system internals. The complaint rate is simply: complaints divided by total reviews in the window.
Layer 2 — the unusual day. A day counts as unusual if its complaint count is at least twice the window's daily average — and at the same time reaches an absolute floor of 3 complaints.
4. The algorithm in detail
flowchart TD
A["Complaint count per day in the window"] --> B["Compute daily average<br/>(total complaints ÷ window days)"]
B --> C["Threshold = max(3, 2.0 × average)"]
C --> D{"Day meets or exceeds the threshold?"}
D -->|yes| E["Marked unusual,<br/>sorted by complaint count"]
D -->|no| F["Ordinary day, no marking"]
The formula itself, straight from the implementation:
threshold = max(3, 2.0 × daily_average)
day is unusual when: complaints_on_this_day ≥ thresholdWhy two conditions instead of one? The absolute floor (3) stops a tiny, quiet app from raising an alert at just two complaints — two grumbles on an otherwise silent day are a relative spike but an absolute nothing. The factor (2.0×) prevents the opposite: a heavily-discussed app constantly flagging "alert" even though the number is well within normal range. Only when both conditions hold does a day actually deserve a look.
One rule, two consumers. This exact same function feeds both the unusual days in the trend view and the "spike" hint in the PDF report. There is no second, independent implementation of the same idea — if there were, the API and the report could flag different days as unusual for the same window, and nobody could say which statement was right.
The complaint rate itself is additionally compared against a fixed attention threshold of 40 %: if the rate exceeds that value, the whole window is marked "concerning" — regardless of whether it also contained a single unusual day or not.
5. A measured example
An app with a daily average of 4 complaints has 9 complaints on one day. The threshold is max(3, 2.0 × 4) = 8. Since 9 ≥ 8, the day gets flagged. Same window, a very small app with a daily average of 0.5 complaints has 2 complaints on one day. The threshold is max(3, 2.0 × 0.5) = max(3, 1) = 3. Since 2 < 3, the day is not flagged — even though, relatively speaking, it was four times the average. That's exactly the safeguard the absolute floor provides.
6. Limits of the method
- The rule measures frequency, not severity. A single one-star outburst about a data-loss bug counts the same as ten mild complaints about advertising, as long as the count matches.
- The factor of 2.0 and the floor of 3 are deliberately chosen, fixed constants — not a statistical model with a standard deviation. That makes the rule auditable by anyone who knows the two numbers, but also less sensitive to very irregular, rare patterns.
- An unusual day says "this is worth a look" — it doesn't say why. The cause (an update, an incident, a campaign) still has to be matched by a human afterward.
7. Conclusion
The strength of this rule isn't mathematical sophistication — it's consistency: one simple formula, shared across every place that claims something is "unusual." No customer has to trust a black box — the formula fits in two lines and checks out on a calculator. That's the difference between a system you have to believe and one you can verify yourself.
The next part brings complaint rate, topics, and unusual days together: how they turn into a report that doesn't just show numbers, but makes concrete recommendations — with a built-in honesty floor that's allowed to say "the data isn't enough for that." → Part 5: The Report That's Allowed to Say No (coming next)