Accented names that won't match: Unicode normalisation
Short answer: The same visible word can be stored as different sequences of characters. "Montréal" typed on one device may use a single "é" character, while text pasted from another source uses "e" followed by a separate accent mark. To your database they are different strings, so logins fail, duplicates slip through and searches miss. Normalise text to one Unicode form, usually NFC, whenever it enters your app, and use accent- and case-insensitive comparison where people expect it.
Why identical-looking text fails to match
Unicode often has more than one way to write the same character. An accented letter can be one precomposed code point, or a base letter plus a combining accent. Both display identically. MDN's documentation for JavaScript's String.prototype.normalize() shows exactly this: two versions of the name "Amélie" that look the same but are not equal, and do not even have the same length, until both are normalised to NFC, after which they compare as equal.
In a business app this shows up in irritating ways:
- A customer named Hélène signs up on a phone, later tries to log in from a laptop, and the app says no account exists for her email or username.
- Your CRM ends up with two records for "Café Rive-Sud" because one was typed and one was imported from a supplier file.
- Site search for "creme brulee" finds nothing because the product is called "Crème brûlée".
- A coupon code or SKU pasted from a PDF contains look-alike characters and is rejected.
For a Canadian business working in English and French, accented names, street names and product titles are everyday data, so these bugs appear quickly.
The four normalisation forms
Unicode Standard Annex #15 defines two kinds of equivalence. Canonical equivalence covers characters that are the same thing written differently, such as precomposed and decomposed accents. Compatibility equivalence is looser and covers formatting variants that mean roughly the same thing.
From these come four forms, which MDN lists as the options to normalize(): NFC, NFD, NFKC and NFKD. NFC and NFD handle canonical equivalence, composing or decomposing characters. NFKC and NFKD also fold compatibility variants. The annex gives examples: in the K forms, the "fi" ligature becomes separate "f" and "i", a superscript 5 becomes a plain 5, circled digits become ordinary digits and the fraction ¼ becomes 1/4. MDN notes that if you call normalize() without an argument, NFC is used.
Which form to use where
- Store text in NFC. It keeps characters looking as typed, is compact, and is what most systems produce. Normalise at the boundary: form submissions, API input, imports and file uploads.
- Use NFKC for identifiers. Usernames, coupon codes, reference numbers and SKUs benefit from folding look-alike variants, so "fi" and "fi" do not become two different codes. Apply it, plus lowercasing where your rules allow, before checking uniqueness.
- Do not use NFKC on prose. It can change meaning or formatting in names and descriptions, such as turning a superscript into a plain digit. Keep the stored display version in NFC.
Matching and searching
Normalisation makes identical text equal. It does not make "e" equal "é", which is what people often expect when searching. For that you need a comparison that ignores accents and case.
In JavaScript, Intl.Collator with sensitivity set to "base" compares strings so that only differences in base letters count; MDN's examples show a ≠ b, but a = á and a = A. That suits sorting and filtering lists in the browser. On the server, many databases and search engines offer accent-insensitive collations or "unaccent" functions. A common pattern is to store a second, search-only column: the text normalised, lowercased and stripped of accents, then search against that.
Be careful with email addresses. The part after the @ is a domain and is case-insensitive, but how mail servers treat the local part varies. Many apps lowercase the whole address and normalise it to NFC for comparison, while keeping the original for display.
Where normalisation usually goes wrong
- Normalising in only one place. If sign-up normalises but login does not, you have made the bug worse. Put it in one shared function used everywhere.
- Forgetting imports. Bulk imports and supplier feeds are where decomposed characters most often arrive. Run them through the same function.
- Unique indexes on raw text. Two "different" strings that look identical can both pass a uniqueness check. Build unique indexes on the normalised value.
- Changing the rules on existing data. If you add normalisation later, migrate existing rows and check for collisions before switching the unique index on.
A quick test you can run today
You do not need special tools to find out whether your app has this problem. Create a test account using a name with accents copied from a word processor, then try logging in after typing the same name on a phone keyboard. Paste a product name containing "é" into your site search, and type it without the accent. Import a small file with accented customer names and see whether your duplicate check catches a record that already exists. If any of these fail, normalisation is missing somewhere along the path, and the failing step tells you where to start.
A worked example
A hypothetical Quebec City bakery chain ran an online pre-order app. Customers kept reporting "account not found" errors, and staff saw duplicate customers such as two separate records for one woman named Geneviève. Investigation showed that some phones and a CSV import produced decomposed accents, while the web form produced precomposed ones.
The developers added one input function that converts all incoming text to NFC, generates an NFKC-and-lowercase version of usernames and emails for uniqueness checks, and fills a search column with accents removed. After migrating existing records and merging the duplicates it revealed, login complaints stopped and searching "eclair" found "Éclair au chocolat".
FAQ
Does my framework already handle this?
Sometimes partially. Some validation libraries and databases normalise, many do not. Test by pasting a decomposed accented name into your forms and checking what is stored.
Will normalising change how text looks?
NFC should not change what users see. NFKC can, which is why it suits identifiers rather than names or descriptions.
Is this only about accents?
No. It also covers full-width characters, ligatures, circled numbers and other variants that show up in copied text.
If customers with accented names keep running into login or duplicate-record problems, let us know where text enters your app.