Unicode Info

What is Unicode?

Unicode is the standard that gives every existing character a unique number, whether it belongs to a language, a symbol set, an emoji or an ideogram. That number is the code point (for instance, U+1F600 maps to 😀).

Under UTF-8 encoding, each code point takes anywhere from 1 to 4 bytes. ASCII, running from U+0000 to U+007F, fits in a single byte; an emoji usually needs 4.

Runs entirely in your browser.

Understanding Unicode: from code points to bytes on disk

Unicode is the universal character encoding standard maintained by the Unicode Consortium, designed to support the worldwide interchange, processing and display of every modern written language plus a large number of historic scripts, mathematical notation, technical symbols and emoji. Its first version, Unicode 1.0, was published in 1991 as a response to the chaos of dozens of incompatible 8-bit code pages (Windows-1252, ISO-8859-1, Shift_JIS, Big5, KOI8-R and so on), where the same byte value 0xE9 could mean "é" in Latin-1, a katakana character in Shift_JIS or a control byte elsewhere. The promise of Unicode was simple: one number, called a code point, for every character on Earth.

A Unicode code point is written as U+ followed by four to six hexadecimal digits. The code space runs from U+0000 to U+10FFFF, for a theoretical maximum of 1,114,112 code points. Of those, 2,048 are reserved as surrogates (used only by UTF-16), 66 are non-characters, and 137,468 are Private Use Area slots, leaving roughly 974,530 code points available for public assignment. As of Unicode 16.0 (September 2024) and the in-development Unicode 17.0, more than 154,000 characters from 168 scripts have been assigned, including emoji, ancient cuneiform, Egyptian hieroglyphs and modern Chinese, Japanese and Korean ideographs.

Planes and blocks: how 1.1 million slots are organised

The Unicode code space is divided into 17 planes of 65,536 code points each, numbered 0 to 16. Each plane is further subdivided into named blocks of variable size (from 16 up to 65,536 code points).

• Plane 0 — BMP (Basic Multilingual Plane), U+0000–U+FFFF: contains characters for almost every modern language — Latin, Cyrillic, Greek, Arabic, Hebrew, Devanagari, Thai, the most common CJK ideographs (URO) — plus punctuation, currency symbols and arrows.
• Plane 1 — SMP (Supplementary Multilingual Plane), U+10000–U+1FFFF: historic scripts (Linear B, Old Italic, Phoenician, Gothic), musical notation, mathematical alphanumerics and, importantly for modern apps, almost every emoji. The grinning face 😀 is U+1F600.
• Plane 2 — SIP (Supplementary Ideographic Plane), U+20000–U+2FFFF: rare CJK ideographs not present in earlier national standards.
• Plane 3 — TIP (Tertiary Ideographic Plane): oracle bone script and additional CJK ideographs.
• Planes 4–13: currently unassigned, reserved for future scripts.
• Plane 14 — SSP (Supplementary Special-purpose Plane): tag characters and variation selectors.
• Planes 15–16 — Private Use Areas (PUA): never assigned by Unicode; reserved for application-specific symbols (icon fonts, internal apps).

Encodings: UTF-8, UTF-16, UTF-32

A code point is an abstract integer; encoding is the rule that turns it into actual bytes. Joel Spolsky's classic warning still holds: "There ain't no such thing as plain text" — you cannot interpret a string without knowing its encoding.

• UTF-8 (defined by RFC 3629): variable-width, 1 to 4 bytes per code point. ASCII (U+0000–U+007F) stays a single byte, so English text is byte-for-byte identical to legacy ASCII. It is the dominant encoding on the web (~98% of pages) and the default in Linux, macOS, JSON, XML and HTML5.
• UTF-16: either 2 or 4 bytes. BMP characters take one 16-bit code unit; supplementary characters take a surrogate pair (a high surrogate in D800–DBFF plus a low surrogate in DC00–DFFF). It is the internal string representation of Java, JavaScript, .NET and Windows APIs.
• UTF-32: fixed 4 bytes per code point. Simple to index but wasteful in memory; rarely used for storage, sometimes used internally.

U+0041 "A"   UTF-8: 41           UTF-16: 0041        UTF-32: 00000041
U+00E9 "é"   UTF-8: C3 A9        UTF-16: 00E9        UTF-32: 000000E9
U+20AC "€"   UTF-8: E2 82 AC     UTF-16: 20AC        UTF-32: 000020AC
U+1F600 "😀"  UTF-8: F0 9F 98 80  UTF-16: D83D DE00   UTF-32: 0001F600

General categories and character properties

Every Unicode code point has a General Category property — a two-letter code where the first letter is one of seven major classes:

• L — Letter (Lu uppercase, Ll lowercase, Lt titlecase, Lm modifier, Lo other).
• M — Mark (Mn nonspacing, Mc spacing combining, Me enclosing) — accents, vowel marks.
• N — Number (Nd decimal digit, Nl letter, No other).
• P — Punctuation (Pc, Pd, Ps, Pe, Pi, Pf, Po).
• S — Symbol (Sm math, Sc currency, Sk modifier, So other — emoji live here).
• Z — Separator (Zs space, Zl line, Zp paragraph).
• C — Other (Cc control, Cf format, Cs surrogate, Co private use, Cn unassigned).

Normalisation: NFC, NFD, NFKC, NFKD

Many characters can be written in more than one way. The letter "é" can be a single precomposed code point U+00E9, or "e" + combining acute accent (U+0065 + U+0301). They look identical but are different byte sequences, which breaks naïve string comparison. Unicode defines four normalisation forms:

• NFC — canonical composition (precomposed). Recommended for storage and the web.
• NFD — canonical decomposition (base + combining marks). Useful for diacritic-insensitive search.
• NFKC / NFKD — compatibility forms that also fold visually-similar variants (fi → fi, ㈱ → (株)). Lossy but handy for security comparisons.

Emoji and ZWJ sequences

Modern emoji are not all single code points. Many are ZWJ sequences: base emoji joined by the Zero-Width Joiner U+200D, optionally with skin-tone modifiers (Fitzpatrick) or variation selectors. The family 👨‍👩‍👧 is four code points glued together:

👨‍👩‍👧 = U+1F468 (man) + U+200D (ZWJ) + U+1F469 (woman) + U+200D (ZWJ) + U+1F467 (girl)
       = 5 code points, 17 UTF-8 bytes, 1 visible "character"

The user-visible unit is called an extended grapheme cluster and is defined by Unicode Standard Annex #29. Cursor movement, selection and the visual length of a string must operate on grapheme clusters — not on bytes, not on code units, and usually not even on code points.

Pitfalls for developers

• Bytes ≠ code points ≠ characters. "😀".length in JavaScript is 2 (UTF-16 code units), not 1.
• Regex: . matches one code unit by default. Use the u flag in JS and consider \p{L} for Unicode-aware classes.
• Sorting: byte order is not alphabetical order. Use locale-aware collation (ICU, Intl.Collator).
• Slicing strings by index can split surrogate pairs or grapheme clusters, producing invalid output and visible glitches.
• Database VARCHAR(n): check whether n means bytes or characters; MySQL's historic utf8 alias is actually a 3-byte truncated UTF-8 that cannot store emoji — use utf8mb4.
• Homoglyph attacks: Cyrillic "а" (U+0430) looks identical to Latin "a" (U+0061). Normalise and compare domain names with IDNA rules.

FAQ

How many characters does Unicode have? About 154,000 assigned as of Unicode 16.0, out of a theoretical maximum of 1,114,112 code points.

Why does my emoji become "??" or boxes? The receiving system either lacks a font with that glyph, or the text was transcoded through a non-UTF-8 layer that stripped the high bytes.

Is UTF-8 always better than UTF-16? For Western text and the web, yes. For CJK-heavy text in memory, UTF-16 can be slightly smaller; that is one reason Java and Windows kept it.

What is a BOM? A Byte Order Mark (U+FEFF) signals UTF-16/32 endianness. UTF-8 BOMs are legal but discouraged because they confuse Unix tooling.

Where is the authoritative reference? The Unicode Consortium publishes the full standard, code charts and the Unicode Character Database (UCD) at unicode.org.