Accents & diacritics

Strip the marks that break search, matching and filenames.

Written for: People moving text between systems

What this family is for

Diacritics carry meaning in the language they belong to — and cause problems everywhere else. They break full-text search, slug matching, filenames and any column that accepts only ASCII. These tools remove the marks deliberately, tell you exactly what is being removed, and note where stripping them loses information you may care about.

Stripping, normalising or converting — which one you need

These three operations look similar and solve different problems. Stripping removes the marks entirely, which is right when the receiving system cannot hold them. Normalising keeps the marks and makes the encoding consistent, which is right when it can. Transliteration changes the script, which is right when the reader cannot read the original at all.

The most common mistake is to strip when the target supported Unicode all along. A search index, a database with utf8mb4, and a modern browser can all hold é perfectly well — stripping it there loses information for no benefit, and makes the stored value differ from what the user typed.

If you have… Use Because
A search key or slug that must be pure ASCII Strip accents The target genuinely cannot hold the marks
Text going into a Unicode-capable database Normalise to NFC Keeps meaning, fixes the encoding mismatch
A readable Latin form for a non-Latin script Transliteration tool Stripping cannot produce letters that were never there
Consistent comparison of two spellings Normalise both sides Comparing a normalised value to an unnormalised one still fails

1 tools in Accents & diacritics

Common mistakes with these tools

  • Stripping Vietnamese and expecting to restore it — Tone marks carry meaning and cannot be recovered from the bare letters. Keep the original alongside the folded form.
  • Forgetting đ and Đ — They are distinct letters, not d-with-a-mark, so generic decomposition leaves them intact. The Vietnamese tool maps them explicitly.
  • Normalising only one side of a comparison — If the stored value is NFC and the input is NFD, they still compare unequal. Both sides need the same treatment.

Frequently asked questions

Is removing diacritics the same as removing the letter?

No: the base letter stays and only the marks are dropped, so Tiếng Việt becomes Tieng Viet, not Ting Vit. The exception is đ/Đ, which is a distinct letter and is folded to d/D explicitly.

Is this reversible?

Only partly. A tone mark removed from Vietnamese cannot be restored from the bare letters, so treat the output as a one-way fold for search and filenames, and keep the original text.

Should I normalise instead of stripping?

If the target system supports Unicode, yes — Unicode normalisation keeps the marks while making the encoding consistent. Strip only when the receiving system genuinely cannot hold them.

Other tool families