ENRU

Byte order mark in JSON: what “Unexpected UTF-8 BOM” means and how to fix it

Updated

Three invisible bytes at the start of the file. Invisible to you, not to the parser. A byte order mark (BOM, U+FEFF) is what some editors and Windows tools put in front of UTF-8 text, and most JSON parsers refuse a document that starts with one.

What a BOM in front of JSON looks like

In an editor the file looks perfect. In bytes it starts with EF BB BF, the UTF-8 encoding of U+FEFF, before the first brace. Either command shows the first three bytes: with a BOM, od prints ef bb bf and Python prints b'\xef\xbb\xbf'.

head -c 3 data.json | od -An -tx1
python3 -c "print(open('data.json', 'rb').read(3))"

Remove those three bytes and the very same document is valid. Nothing else in the file changes, because the BOM was never part of the data:

{"a":1}

What RFC 8259 says about the BOM

RFC 8259, section 8.1 is unusually direct: implementations must not add a byte order mark to the beginning of JSON sent over a network. The next sentence lets parsers ignore one “in the interests of interoperability”. That is a permission, not a duty.

JSON travels as UTF-8, which has no byte order to mark, so the BOM carries no information. And section 2 allows exactly four whitespace characters before the value; U+FEFF is not one of them. A strict parser therefore rejects it, a lenient one skips it, and both are within the standard. That is why one file opens in jq and fails in JSON.parse.

BOM error messages by parser

Input: the bytes EF BB BF followed by {"a":1}; for string APIs, a string starting with U+FEFF. Each message was produced by running that parser on 2026-11-08; Firefox’s comes from the SpiderMonkey source. The third column says what happens when you read such a file the usual way. In several messages the BOM itself sits between the quotes: it is there, just invisible.

ParserMessage for a leading BOMReading a file with a BOM
Chrome 156, Node 24–25 (V8)Unexpected token '', "{"a":1}" is not valid JSONreadFileSync(path, "utf8") keeps it; TextDecoder and Node’s response.json() drop it
Firefox (SpiderMonkey)unexpected characterFrom source; at line 1 column 1
Safari 27 (JavaScriptCore)JSON Parse error: Unrecognized token ''No position
Python 3.9–3.14: json.loads(str)Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)encoding="utf-8-sig" and json.loads(bytes) skip it
Go 1.25–1.26invalid character 'ï' looking for beginning of valueï is the first BOM byte read alone; os.ReadFile keeps the BOM
Go 1.27invalid character '\ufeff' looking for beginning of valueSame cause, now named exactly
Java, Jackson 2.15–2.23 (String input)Unexpected character ('' (code 65279 / 0xfeff)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')byte[] input skips the BOM
C#, System.Text.Json 10'0xEF' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.File.ReadAllText drops it; File.ReadAllBytes keeps it
C#, Newtonsoft.Json 13Unexpected character encountered while parsing value: . Path '', line 0, position 0.The invisible BOM is printed after the colon
PHP 8.3: json_decodeSyntax errorfile_get_contents keeps it
Rust, serde_json 1.0expected value at line 1 column 1Nothing in the message points to the BOM
jq 1.7.1No errorjq skips a leading BOM
MySQL 8.4: JSON columnInvalid JSON text: "Invalid value." at position 0Position 0 is the first BOM byte
JSON Statham ValidatorByte order mark, line 1, column 1Code bom; an opened file keeps its BOM, so you see it

How to remove a BOM before parsing JSON

Strip U+FEFF from the start of the text, or let the reader do it. Every snippet was run on a file that starts with a BOM, and each one works on files without it too.

JavaScript (Node.js)

import { readFileSync } from 'node:fs';

const text = readFileSync('data.json', 'utf8');
const data = JSON.parse(text.replace(/^\uFEFF/, ''));

Python

import json

with open("data.json", encoding="utf-8-sig") as f:
    data = json.load(f)

utf-8-sig reads UTF-8 with and without a BOM, so it is a safe default for JSON files you didn’t write. The json module docs note that json.loads also accepts bytes and works out the encoding itself.

Go

import (
	"bytes"
	"encoding/json"
	"os"
)

func loadJSON(path string, v any) error {
	data, err := os.ReadFile(path)
	if err != nil {
		return err
	}
	data = bytes.TrimPrefix(data, []byte("\xef\xbb\xbf"))
	return json.Unmarshal(data, v)
}

Java (Jackson)

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

static JsonNode load(ObjectMapper mapper, Path path) throws IOException {
    return mapper.readTree(Files.readAllBytes(path));
}

static JsonNode parse(ObjectMapper mapper, String text) throws IOException {
    return mapper.readTree(text.replaceFirst("^\uFEFF", ""));
}

C# (System.Text.Json)

using System.Text.Json;

var text = File.ReadAllText("data.json");
using var doc = JsonDocument.Parse(text.TrimStart('\uFEFF'));

File.ReadAllText already detects and drops the BOM; TrimStart covers text that arrives some other way. Bytes passed straight to JsonDocument.Parse are not cleaned: that is where 0xEF comes from.

PHP

$text = file_get_contents('data.json');
$text = preg_replace('/^\xEF\xBB\xBF/', '', $text);
$data = json_decode($text, true);

Where BOMs come from and how to stop them

  • Windows PowerShell 5.1. -Encoding UTF8 there means UTF-8 with a BOM. PowerShell 6 and later default to UTF-8 without one (about_Character_Encoding); on 5.1, write with [System.IO.File]::WriteAllText($path, $json).
  • Editors. Notepad offers both “UTF-8” and “UTF-8 with BOM” when saving. In VS Code the status bar shows the encoding; click it, pick Save with Encoding, then UTF-8.
  • C#. File.WriteAllText(path, text) writes no BOM; passing Encoding.UTF8 as the third argument adds one.
  • Python. encoding="utf-8-sig" adds a BOM when writing. Use it for reading only.
  • A check in CI. If head -c 3 file.json | od -An -tx1 prints ef bb bf, fail the build before a strict parser does it in production.

Find the BOM in the validator

Try it: open the BOM sample in the Validator. A file you open keeps its BOM, so the validator reports bom at line 1, column 1 instead of hiding the problem. Press Show me (F8), then Delete. A U+FEFF further into the document is no longer a byte order mark, just an invisible character between tokens, and gets Unexpected character. Inside a string it is an ordinary character and perfectly valid.

JSON byte order mark FAQ

Is a BOM allowed in JSON?

Not in JSON you send: RFC 8259 says implementations must not add one to JSON on the network. Parsers may ignore it, so some accept it, but JSON.parse, Python’s json.loads on a string and Go reject it.

How do I see whether a file has a BOM?

Look at the first three bytes: EF BB BF is a UTF-8 BOM. On Linux and macOS run head -c 3 file.json | od -An -tx1; in VS Code the status bar says “UTF-8 with BOM”.

What does “decode using utf-8-sig” mean in Python?

It names the codec that skips the BOM. Open the file with encoding="utf-8-sig", or pass the raw bytes to json.loads, which detects the BOM itself.

Does removing the BOM change my data?

No. The BOM is not part of any value, so removing it changes nothing in the document. A U+FEFF inside a string is data, and stays.