Lorixen.
EN

Using AI to spot recurring problems in customer reviews

Short answer: Export your reviews to a CSV, strip names and emails, and ask an AI model to propose a short list of themes with a definition and example quotes for each. Then have it tag every review with one or more of those themes, and do the counting yourself in a spreadsheet, because models are unreliable at counting. Spot-check a sample of tags against the raw reviews, and turn the top two or three themes into specific product, packaging or product-page fixes. Repeat monthly with the same theme list.

Reading 40 reviews tells you a lot. Reading 4,000 is where patterns hide: the zipper that fails after two months, the sizing that runs small only in one colour, the courier that keeps crushing boxes in winter. AI tools are good at grouping messy text into themes. They're bad at arithmetic and at admitting they've invented a quote. The pipeline below uses each for what it does well.

Step 1: Export the reviews

Most review apps export to CSV. Judge.me's help centre, for example (as of November 2026), describes exporting from Reviews > Export, with options for published, hidden or archived reviews, selected reviews, a single product, or a date range. Its export includes the review title, body, rating, date, source, status, reviewer name and email, product ID and handle, and any public reply. Exports over 25 reviews arrive by email as a download link, and a single file is capped at 100,000 reviews.

Export at least three to six months of reviews so a seasonal blip doesn't look like a trend.

Step 2: Clean and anonymize

Before anything goes near an AI tool:

  • Delete reviewer names and email addresses. You don't need them to find themes.
  • Keep: a row ID you create, product handle, rating, date and the review text.
  • Remove duplicates and obvious spam.
  • Split very long files into batches by product line so prompts stay focused.

Check what your AI tool's business terms say about data use, and check the privacy rules where you operate before uploading customer text.

Step 3: Ask for a theme list, not an answer

Start with a sample (say 200 reviews mixed across ratings). Ask the model to propose themes, not to judge your product.

You are analyzing customer reviews for an online store.
From the reviews below, propose 8 to 12 themes that describe
specific product or service issues or praise.
For each theme give: a short name, a one-sentence definition,
and 2 example quotes copied exactly from the reviews with their row IDs.
Do not count anything. Do not invent quotes.

Read the list critically. Merge vague themes ("quality issues") into specific ones ("stitching comes apart at the shoulder"). Add any theme you know exists that it missed. This edited list is your codebook; keep it stable month to month so trends are comparable.

Step 4: Tag every review against the codebook

Now give the model your codebook and a batch of reviews and ask it to tag each row.

Using only the themes in this codebook, tag each review.
Return one line per review: row_id, theme names (or "none"),
and sentiment (positive/negative/mixed).
If a review fits no theme, return "none". Do not create new themes.

If you're using an API rather than a chat window, use a structured output feature so every response follows the same format. OpenAI's documentation describes Structured Outputs as ensuring responses follow a JSON Schema you supply, so required fields aren't omitted and values stay within the options you defined. That makes the next step reliable.

Step 5: Count in the spreadsheet

Paste the tags back next to your reviews in Google Sheets or Excel. Then count with formulas, not with the model.

  • COUNTIF counts rows matching a criterion; Google's help notes the criterion can include wildcards, so =COUNTIF(D2:D5000,"*sizing*") counts every review tagged with a sizing theme even when it has several tags.
  • Add columns for month and product so a pivot table shows which product and which month each theme peaks in.
  • Calculate each theme's share of negative reviews, not just raw counts, because review volume changes month to month.

If you ask a model "how many reviews mention sizing?", it may give a confident, wrong number. The spreadsheet won't.

Step 6: Verify before you act

Check How What a fail looks like
Quotes are real Search the CSV for each quoted phrase Phrase not found, or found in a different review
Tags are accurate Read 20 random tagged reviews per top theme More than a few clearly mis-tagged
"None" isn't hiding problems Read 20 untagged negative reviews A recurring issue with no theme
Counts match Compare formula totals with a manual filter Totals differ
Trend is real Compare with the same month last year, if available Spike matches a seasonal pattern
Theme is specific enough to act on Can someone fix it this month? "Customers unhappy with quality"

If a check fails, fix the codebook or the prompt and re-run the batch.

Step 7: Turn themes into fixes

For each of the top two or three negative themes, write one line: theme → owner → action → how you'll know it worked.

Hypothetical examples:

  • "Runs small in the shoulders" → product → add a fit note and a measurement chart to the product page → share of sizing complaints for that style drops next quarter.
  • "Arrived crushed" → fulfillment → switch to a sturdier box for glass items → fewer damage mentions and replacement requests.
  • "Instructions unclear" → content → add a 60-second setup video and a printed quick-start card → fewer "how do I" reviews and support tickets.

Positive themes matter too: if many reviews praise a feature you barely mention, move it into the first lines of the product description.

Step 8: Repeat monthly

  • Export the new month's reviews.
  • Tag them against the same codebook.
  • Update the spreadsheet and pivot.
  • Review the top movers in a 30-minute meeting.
  • Add a new theme only when you've seen it repeatedly; note the date you added it.

Limits to keep in mind

  • Counts: do them in the spreadsheet.
  • Invented quotes: always verify against the raw text.
  • Sarcasm and mixed reviews: models misread them; your spot checks catch most.
  • Small numbers: five complaints in a month may be noise. Look for repetition across months.

FAQ

Can I just paste all my reviews into a chat window?

For a few hundred, yes, if they're anonymized. For thousands, batch by product and use the tagging approach so results are consistent.

Which AI tool should I use?

Any capable model works for theming. Prioritize one with clear business data terms and, if you automate, structured output support.

How often should I change the theme list?

Rarely. Stable themes make month-to-month comparison meaningful.

Does this replace reading reviews?

No. It tells you which reviews to read first.

Turning review text into a short, trustworthy list of fixes is a repeatable process, and we can build that monthly pipeline for your store.

Sources

WhatsAppLet’s talk