Whitespace and Invisible Character Cleaner

What this tool does

Text pasted from a browser, a PDF, a spreadsheet export or a chat application routinely carries characters that cannot be seen: a zero-width space between two words, a non-breaking space where an ordinary one was expected, or a byte order mark at the start of a value. To a human the text looks correct. To a database it is a different string, so a unique key collides, a lookup returns nothing, or a sort order looks random.

This page first reports how many of these characters are present, then removes the ones you select, so you can confirm the diagnosis rather than guessing.

Method

The cleaner targets specific Unicode ranges rather than stripping anything unusual, so legitimate scripts are never damaged:

Option	Characters removed	Code points
zero-width	Zero-width space, joiners, directional marks, byte order mark	U+200B–U+200F, U+202A–U+202E, U+2060–U+2064, U+FEFF
non-breaking spaces	Non-breaking space, narrow no-break space, figure space	U+00A0, U+202F, U+2007
collapse repeats	Runs of spaces or tabs become one space; three or more newlines become two	—
trim lines	Leading and trailing whitespace on every line, and at the start and end of the text	—

Letters carrying diacritics, Thai and Devanagari marks, emoji and ordinary punctuation are all left untouched, because none of them fall in the ranges above.

Example

Before	Problem	After
`hello\u200Bworld`	zero-width space inside a word	`helloworld`
`1 000\u00A0VND`	non-breaking space	`1 000 VND`
`key`	leading and trailing spaces break a lookup	`key`

Not all whitespace is the space bar

A string that looks like it has ordinary spaces may contain any of a dozen whitespace characters, and they do not compare equal. The common ones:

Character	Code point	Looks like	Comes from
No-break space	U+00A0	A space	HTML ` `, Word, French typography
Narrow no-break space	U+202F	A thin space	Formatted documents
Ideographic space	U+3000	A wide space	CJK input methods
Zero-width space	U+200B	Nothing	Line-breaking hints copied from a page
BOM	U+FEFF	Nothing	Start of a UTF-8 file from some exporters
Tab	U+0009	A wide gap	Spreadsheet copies, indentation

Only the last two are visible as anything other than a space, which is why the others cause matching bugs that look inexplicable: the value prints identically to the expected one and compares unequal.

Why it breaks lookups

A database lookup for an email address fails if the stored value has a trailing no-break space and the input does not — or the reverse. A CSV import that ignores a column often has a BOM in its first header. A username that "does not exist" can contain a zero-width space pasted from a chat application. In each case the fix is normalisation at the boundary, not a change to the lookup.

Cleaning without damaging the text

This tool reports what it found before removing it, which matters because a blanket strip is not always safe. Removing a zero-width joiner from an emoji sequence changes how it renders. Removing bidi marks from Arabic or Hebrew text breaks its display order. The cleaner targets whitespace and the invisible format characters that break identifiers, and leaves the joiners and direction marks alone.

For an identifier field — email, username, key — it is reasonable to strip aggressively. For prose, normalising to NFC and collapsing ordinary whitespace is the safe pass, and anything more is a decision that should be made deliberately.

Line endings

Windows uses \r\n, Unix uses \n, and some files use \r alone. A tool that only handles \n leaves a carriage return at the end of every line, which is invisible in most editors and breaks comparison and sorting. This tool normalises line endings as part of cleaning, so the result is consistent regardless of where the text came from.

Frequently asked questions

Does it remove line breaks?

No. It removes invisible characters, collapses runs of spaces and trims each line, but it keeps the line structure. If the problem is hard-wrapped prose from a PDF, use Remove Line Breaks; if it is extra blank lines, the cleaner can collapse those without joining the text.

What are zero-width characters?

Characters that occupy no visible space but still count as data: the zero-width space, the zero-width joiner and non-joiner, and the byte order mark. They usually arrive when text is copied from a web page, a PDF or a chat app.

What is a non-breaking space?

A space that looks like a normal space but is a different character (U+00A0). It stops a line from breaking between two words, and it is the most common reason “identical” text fails to match.

Why does cleaning text matter?

Search, comparison, unique database keys, sort order and code all treat these characters as significant. Removing them makes text behave the way a human reader expects.

Will this remove needed spaces?

No. Ordinary spaces are preserved unless you ask for runs of them to be collapsed. Only invisible, non-standard characters are removed.

How many invisible characters are in my text?

The counter above the result shows how many zero-width, non-breaking and other unusual space characters were found before cleaning.

Go deeper

Longer explanations of the problems this tool solves:

Sources

Unicode Standard, general category Cf (format) and the special space characters in Zs.
Unicode Standard, U+FEFF byte order mark and its use as a zero-width no-break space.