1001Ferramentas
🔎 Utilities

BOM Detector

Check whether an uploaded file starts with a Byte Order Mark (UTF-8, UTF-16-LE, UTF-16-BE, UTF-32). Useful for encoding issues.

Aguardando arquivo…

Finding the encoding from the first bytes

Some text files begin with a byte order mark, three to four bytes declaring the encoding before the first real character. It is effectively mandatory for UTF-16 and UTF-32, where it indicates byte order, and optional in UTF-8, where it merely hints. Since those bytes are invisible in any editor, a file with an encoding problem gives no clue at all on screen.

Pick the file and the page reads its beginning and identifies the mark: UTF-8, UTF-16 in either order, or UTF-32 in either order. When there is no mark, it says so — which does not mean the file is UTF-8, only that it does not declare itself. The file is read in your own browser and is not sent anywhere.

The order of the checks matters more than it seems. The UTF-32 little-endian mark starts with the same two bytes as the UTF-16 little-endian mark, so testing UTF-16 first would misclassify every UTF-32 file. That is why the four-byte patterns are checked before the two-byte ones — a detail easy to get wrong in a hand-rolled implementation.

Frequently asked questions

Does no mark mean UTF-8?
It means nothing for certain. A UTF-8 file with no mark is the most common case in Unix environments, but a Windows-1252 or ISO-8859-1 file has no mark either. Without a declaration, the encoding has to be inferred from content or agreed out of band.
Should I remove the UTF-8 mark?
Generally yes, unless something requires otherwise: the Unicode specification discourages it in UTF-8, and it breaks script shebangs, causes output before headers in PHP and makes JSON parsers reject the file. The known exception is Excel, which handles accented CSV better when the mark is present.
Can encoding be detected without a mark?
Only by estimation. Detection libraries analyse byte distribution and compare it against the pattern of each encoding and language, which works well on long text and misfires on short. Where possible, the right move is agreeing the encoding rather than guessing it.

Related Tools