Byte order mark in JSON: what “Unexpected UTF-8 BOM” means and how to fix it
Updated
Three invisible bytes at the start of the file. Invisible to you, not to the parser. A byte order mark (BOM, U+FEFF) is what some editors and Windows tools put in front of UTF-8 text, and most JSON parsers refuse a document that starts with one.
What a BOM in front of JSON looks like
In an editor the file looks perfect. In bytes it starts with EF BB BF, the UTF-8 encoding of U+FEFF, before the first brace. Either command shows the first three bytes: with a BOM, od prints ef bb bf and Python prints b'\xef\xbb\xbf'.
head -c 3 data.json | od -An -tx1
python3 -c "print(open('data.json', 'rb').read(3))"Remove those three bytes and the very same document is valid. Nothing else in the file changes, because the BOM was never part of the data:
{"a":1}What RFC 8259 says about the BOM
RFC 8259, section 8.1 is unusually direct: implementations must not add a byte order mark to the beginning of JSON sent over a network. The next sentence lets parsers ignore one “in the interests of interoperability”. That is a permission, not a duty.
JSON travels as UTF-8, which has no byte order to mark, so the BOM carries no information. And section 2 allows exactly four whitespace characters before the value; U+FEFF is not one of them. A strict parser therefore rejects it, a lenient one skips it, and both are within the standard. That is why one file opens in jq and fails in JSON.parse.
BOM error messages by parser
Input: the bytes EF BB BF followed by {"a":1}; for string APIs, a string starting with U+FEFF. Each message was produced by running that parser on 2026-11-08; Firefox’s comes from the SpiderMonkey source. The third column says what happens when you read such a file the usual way. In several messages the BOM itself sits between the quotes: it is there, just invisible.
| Parser | Message for a leading BOM | Reading a file with a BOM |
|---|---|---|
| Chrome 156, Node 24–25 (V8) | Unexpected token '', "{"a":1}" is not valid JSON | readFileSync(path, "utf8") keeps it; TextDecoder and Node’s response.json() drop it |
| Firefox (SpiderMonkey) | unexpected character | From source; at line 1 column 1 |
| Safari 27 (JavaScriptCore) | JSON Parse error: Unrecognized token '' | No position |
Python 3.9–3.14: json.loads(str) | Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0) | encoding="utf-8-sig" and json.loads(bytes) skip it |
| Go 1.25–1.26 | invalid character 'ï' looking for beginning of value | ï is the first BOM byte read alone; os.ReadFile keeps the BOM |
| Go 1.27 | invalid character '\ufeff' looking for beginning of value | Same cause, now named exactly |
| Java, Jackson 2.15–2.23 (String input) | Unexpected character ('' (code 65279 / 0xfeff)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') | byte[] input skips the BOM |
| C#, System.Text.Json 10 | '0xEF' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0. | File.ReadAllText drops it; File.ReadAllBytes keeps it |
| C#, Newtonsoft.Json 13 | Unexpected character encountered while parsing value: . Path '', line 0, position 0. | The invisible BOM is printed after the colon |
PHP 8.3: json_decode | Syntax error | file_get_contents keeps it |
| Rust, serde_json 1.0 | expected value at line 1 column 1 | Nothing in the message points to the BOM |
| jq 1.7.1 | No error | jq skips a leading BOM |
| MySQL 8.4: JSON column | Invalid JSON text: "Invalid value." at position 0 | Position 0 is the first BOM byte |
| JSON Statham Validator | Byte order mark, line 1, column 1 | Code bom; an opened file keeps its BOM, so you see it |
How to remove a BOM before parsing JSON
Strip U+FEFF from the start of the text, or let the reader do it. Every snippet was run on a file that starts with a BOM, and each one works on files without it too.
JavaScript (Node.js)
import { readFileSync } from 'node:fs';
const text = readFileSync('data.json', 'utf8');
const data = JSON.parse(text.replace(/^\uFEFF/, ''));Python
import json
with open("data.json", encoding="utf-8-sig") as f:
data = json.load(f)utf-8-sig reads UTF-8 with and without a BOM, so it is a safe default for JSON files you didn’t write. The json module docs note that json.loads also accepts bytes and works out the encoding itself.
Go
import (
"bytes"
"encoding/json"
"os"
)
func loadJSON(path string, v any) error {
data, err := os.ReadFile(path)
if err != nil {
return err
}
data = bytes.TrimPrefix(data, []byte("\xef\xbb\xbf"))
return json.Unmarshal(data, v)
}Java (Jackson)
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
static JsonNode load(ObjectMapper mapper, Path path) throws IOException {
return mapper.readTree(Files.readAllBytes(path));
}
static JsonNode parse(ObjectMapper mapper, String text) throws IOException {
return mapper.readTree(text.replaceFirst("^\uFEFF", ""));
}C# (System.Text.Json)
using System.Text.Json;
var text = File.ReadAllText("data.json");
using var doc = JsonDocument.Parse(text.TrimStart('\uFEFF'));File.ReadAllText already detects and drops the BOM; TrimStart covers text that arrives some other way. Bytes passed straight to JsonDocument.Parse are not cleaned: that is where 0xEF comes from.
PHP
$text = file_get_contents('data.json');
$text = preg_replace('/^\xEF\xBB\xBF/', '', $text);
$data = json_decode($text, true);Where BOMs come from and how to stop them
- Windows PowerShell 5.1.
-Encoding UTF8there means UTF-8 with a BOM. PowerShell 6 and later default to UTF-8 without one (about_Character_Encoding); on 5.1, write with[System.IO.File]::WriteAllText($path, $json). - Editors. Notepad offers both “UTF-8” and “UTF-8 with BOM” when saving. In VS Code the status bar shows the encoding; click it, pick Save with Encoding, then UTF-8.
- C#.
File.WriteAllText(path, text)writes no BOM; passingEncoding.UTF8as the third argument adds one. - Python.
encoding="utf-8-sig"adds a BOM when writing. Use it for reading only. - A check in CI. If
head -c 3 file.json | od -An -tx1printsef bb bf, fail the build before a strict parser does it in production.
Find the BOM in the validator
Try it: open the BOM sample in the Validator. A file you open keeps its BOM, so the validator reports bom at line 1, column 1 instead of hiding the problem. Press Show me (F8), then Delete. A U+FEFF further into the document is no longer a byte order mark, just an invisible character between tokens, and gets Unexpected character. Inside a string it is an ordinary character and perfectly valid.
- Unexpected character: other invisible characters, such as U+00A0 and U+200B.
- Unexpected token in JSON: what V8 says when the very first character is wrong.
- Empty input: when there are no bytes at all.
- What is JSON and how to open a .json file
- All JSON errors
JSON byte order mark FAQ
Is a BOM allowed in JSON?
Not in JSON you send: RFC 8259 says implementations must not add one to JSON on the network. Parsers may ignore it, so some accept it, but JSON.parse, Python’s json.loads on a string and Go reject it.
How do I see whether a file has a BOM?
Look at the first three bytes: EF BB BF is a UTF-8 BOM. On Linux and macOS run head -c 3 file.json | od -An -tx1; in VS Code the status bar says “UTF-8 with BOM”.
What does “decode using utf-8-sig” mean in Python?
It names the codec that skips the BOM. Open the file with encoding="utf-8-sig", or pass the raw bytes to json.loads, which detects the BOM itself.
Does removing the BOM change my data?
No. The BOM is not part of any value, so removing it changes nothing in the document. A U+FEFF inside a string is data, and stays.