Skip to content

Unicode in JavaScript strings: code points to graphemes

  • Why '🎁'.length is 2, and the three ways to count “characters”: code units, code points and grapheme clusters.
  • The Unicode tools added over time: ES2015 code point escapes, codePointAt, fromCodePoint, string iteration, normalize() and the regex u flag; ES2019 well-formed JSON.stringify; ES2024 isWellFormed() and toWellFormed().
  • How to validate, normalize, count and truncate user text without corrupting it.

TL;DR: a JavaScript string is a sequence of UTF-16 code units. Characters outside the Basic Multilingual Plane, such as most emoji, take two units (a surrogate pair), so length, indexing and slice can split them. ES2015 added code-point-aware APIs (\u{...}, codePointAt, String.fromCodePoint, for...of, normalize(), the u regex flag). ES2024 added isWellFormed() and toWellFormed() for lone surrogates. For what users perceive as one character, use Intl.Segmenter (ECMA-402, 2022).

OrdersPortal lets customers add a gift message of up to 20 characters. A naive implementation uses length and slice:

Running the samples: save a sample as a .mjs file and run it with Node.js 22 (node sample.mjs), or paste it into a browser console. All outputs shown were produced with Node.js 22.23 (ICU with Unicode 17.0); grapheme results can differ in engines with older Unicode data.

const gift = 'Thanks 🎁';
console.log(gift.length); // 9: the emoji is two UTF-16 code units
console.log(JSON.stringify(gift.slice(0, 8))); // cuts the emoji in half
console.log(gift.charCodeAt(7).toString(16), gift.charCodeAt(8).toString(16));
console.log([...gift].length); // 8 code points
console.log(gift.codePointAt(7).toString(16));
console.log(String.fromCodePoint(0x1f381) === '\u{1F381}', '\u{1F381}' === '🎁');
console.log(/^.$/.test('🎁'), /^.$/u.test('🎁'));

Output:

9
"Thanks \ud83c"
d83c df81
8
1f381
true true
false true

The message the customer sees has eight characters, length says nine, and slice leaves a lone surrogate (\ud83c) that renders as � and breaks some encoders.

Unit What it is How to count in JavaScript
Code unit 16-bit UTF-16 value str.length, str[i], charCodeAt, slice
Code point One Unicode scalar, U+0000 to U+10FFFF [...str].length, for...of, codePointAt
Grapheme cluster What a reader perceives as one character Intl.Segmenter with granularity: 'grapheme'
  • \u{1F381} writes any code point in string literals (and in regex patterns with u). The old \uXXXX form only reaches four hex digits, so astral characters needed two escapes.
  • codePointAt(i) returns the full code point when i points at a high surrogate; charCodeAt returns only the unit.
  • String.fromCodePoint(...) is the inverse; String.fromCharCode works with units.
  • String iteration (for...of, spread, Array.from) yields code points, so it never splits a surrogate pair.
  • The u flag makes a regex work on code points: . matches a whole emoji, and \u{...} escapes and Unicode property escapes become available.
  • normalize(form) converts between Unicode normalization forms: "NFC" (default), "NFD", "NFKC" and "NFKD". Any other value throws a RangeError.

Graphemes, normalization and well-formed strings

Section titled “Graphemes, normalization and well-formed strings”
const family = '👩‍👩‍👧'; // one visible symbol
console.log(family.length, [...family].length);
const segmenter = new Intl.Segmenter('en', { granularity: 'grapheme' });
console.log([...segmenter.segment(family)].length);
const composed = 'Café'; // é as U+00E9
const decomposed = 'Café'; // e + U+0301 COMBINING ACUTE ACCENT
console.log(composed === decomposed, composed.length, decomposed.length);
console.log(composed === decomposed.normalize('NFC'));
console.log([...segmenter.segment(decomposed)].length);
const broken = 'ok \uD83C'; // lone high surrogate
console.log(broken.isWellFormed(), JSON.stringify(broken.toWellFormed()));
console.log(JSON.stringify(broken)); // ES2019: escaped, not invalid UTF-8
try { encodeURIComponent(broken); } catch (e) { console.log(e.constructor.name, '-', e.message); }
console.log(new TextEncoder().encode(broken)); // replaced by U+FFFD (EF BF BD)
console.log('fi'.normalize('NFKC'), '①'.normalize('NFKC'));

Output:

8 5
1
false 4 5
true
4
false "ok �"
"ok \ud83c"
URIError - URI malformed
Uint8Array(6) [ 111, 107, 32, 239, 191, 189 ]
fi 1

What this shows:

  • The family emoji is five code points (three people joined by two zero-width joiners) but one grapheme. Code points are better than code units, but still not “characters”.
  • Café can be stored two ways that look identical but compare unequal. Normalize before comparing, hashing or using text as a key.
  • A string with a lone surrogate is not well formed. isWellFormed() detects it; toWellFormed() replaces each lone surrogate with U+FFFD. encodeURIComponent throws on it, TextEncoder silently replaces it, and since ES2019 JSON.stringify outputs an escape sequence instead of invalid UTF-16.
  • NFKC also folds compatibility characters (fi to fi, ① to 1). That is useful for search keys, but it changes meaning, so do not apply it to stored text.

OrdersPortal cleans and limits the gift message at checkout, and builds accent-insensitive search keys for customer names:

// gift-message.mjs - clean and limit the gift message customers type at OrdersPortal checkout.
const graphemes = new Intl.Segmenter('en', { granularity: 'grapheme' });
export function countCharacters(text) {
let count = 0;
for (const _ of graphemes.segment(text)) count++;
return count;
}
// Cut at a grapheme boundary so we never split an emoji or an accent from its letter.
export function truncate(text, max) {
let out = '';
let count = 0;
for (const { segment } of graphemes.segment(text)) {
if (count === max) return out + '…';
out += segment;
count++;
}
return out;
}
export function cleanGiftMessage(raw, max = 20) {
const text = raw
.toWellFormed() // lone surrogates (bad paste, broken client) become U+FFFD
.normalize('NFC') // one canonical form before storing, comparing or counting
.trim();
return { text: truncate(text, max), characters: countCharacters(text) };
}
// Accent-insensitive search key: decompose, then drop combining marks.
export function searchKey(name) {
return name.normalize('NFD').replace(/\p{M}/gu, '').toLowerCase();
}
console.log(cleanGiftMessage('Happy birthday, Zoë! 🎉👩‍👩‍👧'));
console.log(cleanGiftMessage('Thanks \uD83C'));
console.log(cleanGiftMessage('Café voucher').text.length);
console.log(searchKey('Zoë Müller') === searchKey('ZOE MULLER'));

Output:

{ text: 'Happy birthday, Zoë!…', characters: 23 }
{ text: 'Thanks �', characters: 8 }
12
true

The order of operations matters: make the string well formed, normalize, then count and cut. The third line shows NFC turning the five-unit Café into four units before storage. searchKey is for matching only; display the original name. For language-aware comparison and sorting, use Intl.Collator (for example with sensitivity: 'base') instead of stripping marks.

  • Using length as a character limit. It counts code units. Count graphemes for user-facing limits, and count UTF-8 bytes (new TextEncoder().encode(s).length) for storage limits.
  • Cutting with slice, substring or [i]. These can split surrogate pairs and separate accents from letters. Cut at grapheme boundaries.
  • Reversing with split(''). '🎁ab'.split('').reverse().join('') swaps the surrogates and produces garbage. Reverse an array of graphemes instead.
  • Comparing without normalizing. Visually identical strings can differ. Normalize to NFC at the boundary where text enters the system.
  • Regexes without u or v. Without a Unicode flag, . and character classes work on code units, and \p{...} is a plain p. See RegExp features.
  • Assuming toUpperCase() keeps length. 'ß'.toUpperCase() is 'SS'. Locale rules also differ: use toLocaleUpperCase('tr') for Turkish dotted and dotless i.
Task Tool
Code points, escapes, decoding codePointAt, String.fromCodePoint, for...of
User-visible character count, truncation, cursor movement Intl.Segmenter (grapheme)
Comparing, deduplicating, using text as a key normalize('NFC') first
Data from untrusted sources heading to fetch, URLs or TextEncoder isWellFormed() to reject, or toWellFormed() to repair
Locale-aware sorting or matching Intl.Collator
Word and sentence boundaries Intl.Segmenter with word or sentence

Code point APIs are enough for protocol work such as parsing or escaping. Anything a person reads or types needs graphemes.

According to MDN’s browser compatibility data:

API Chrome Firefox Safari Node.js
codePointAt, String.fromCodePoint (ES2015) 41 29 9 4.0.0
String iterator (ES2015) 38 36 9 0.12.0
normalize() (ES2015) 34 31 10 0.12.0
RegExp u flag (ES2015) 50 46 10 6.0.0
isWellFormed(), toWellFormed() (ES2024) 111 119 16.4 20.0.0
Intl.Segmenter (ECMA-402, 2022) 87 125 14.1 16.0.0

Edge has supported the ES2015 APIs since Edge 12 and the newer ones from the same version numbers as Chrome. Safari on iOS matches desktop Safari, except Intl.Segmenter, which arrived in iOS 14.5. Deno and Bun support all of them. Internet Explorer supports none. Intl.Segmenter needs a runtime built with full ICU data, which is the default for Node.js. TypeScript types isWellFormed in lib.es2024.string.d.ts and Intl.Segmenter in lib.es2022.intl.d.ts.

1. Why is '🎁'.length equal to 2? Strings are UTF-16 code unit sequences. U+1F381 is above U+FFFF, so it is stored as a surrogate pair, and length counts units.

2. What is the difference between [...str].length and a grapheme count? Spreading iterates code points. A grapheme cluster can contain several code points: a letter plus combining marks, a flag made of two regional indicators, or an emoji joined with ZWJ characters. Intl.Segmenter counts clusters.

3. Why can two strings that look identical compare unequal? They may use different normalization forms, such as precomposed é versus e plus a combining accent. normalize('NFC') on both makes them equal.

4. Senior scenario: OrdersPortal must enforce a 20-character display-name limit in the UI, the API and a database column defined in bytes. How do you make all three agree? Define “character” as an extended grapheme cluster and enforce it in one shared function. At the API boundary, repair or reject lone surrogates, normalize to NFC, then check the grapheme count. Separately check the UTF-8 byte length against the column size, because 20 graphemes can be far more than 20 bytes. Make the UI use the same counting rule (with Intl.Segmenter), but treat the server as the authority. Add tests with emoji sequences, combining marks and CJK text, and remember grapheme rules change with Unicode versions.

  • JavaScript strings are UTF-16 code units; length, indexing and slice work on units.
  • ES2015 added code point APIs and the u flag; ES2024 added isWellFormed() and toWellFormed().
  • Users see grapheme clusters: count and cut with Intl.Segmenter.
  • Normalize to NFC before comparing or storing, and repair lone surrogates before encoding.