{"id":"CVE-2026-85670","aliases":["RHLW-2026-b11abda52745af4c"],"summary":"tokenizers BpeBuilder Buffer Overflow via merge token","published_at":"2026-09-04T14:32:25.391Z","modified_at":"2026-11-05T17:25:54Z","products":[{"slug":"lightwell","name":"Red Hat Lightwell (gehärtete Images)","category":"red-hat"}],"rating":{"severity":"high","score":7.6,"vector":"CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:L/A:H","attack_path":"internet_unauthenticated","user_interaction":true,"malicious":false},"fix_available":true,"kev":{"listed":false,"ransomware":false},"public_exploit":false,"exploits":[],"epss":{"score":0.004670000169426203,"percentile":0.4121299982070923,"date":"2026-11-19T00:00:00Z"},"priority":9.1,"details":"tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.","affected_stack":[{"ecosystem":"Red Hat Lightwell:PyPI","package":"tokenizers","purl":"pkg:pypi/tokenizers","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"0.23.1+rhlw.1"}]}],"versions":["0.0.1","0.0.10","0.0.11","0.0.12","0.0.13","0.0.2","0.0.3","0.0.4","0.0.5","0.0.6","0.0.7","0.0.8","0.0.9","0.1.0","0.1.1","0.10.0","0.10.1","0.10.2","0.10.3","0.11.0","0.11.1","0.11.2","0.11.3","0.11.4","0.11.5","0.11.6","0.12.0","0.12.1","0.13.0","0.13.1","0.13.2","0.13.3","0.14.0","0.14.1","0.15.0","0.15.1","0.15.2","0.19.0","0.19.1","0.2.0","0.2.1","0.20.0","0.20.1","0.20.2","0.20.3","0.20.4","0.20.4rc0","0.21.0","0.21.0rc0","0.21.1","0.21.1rc0","0.21.2","0.21.2rc0","0.21.4","0.22.0","0.22.0rc0","0.22.1","0.22.1rc0","0.22.2","0.22.2rc0","0.23.0rc0","0.23.1","0.23.1rc0","0.3.0","0.4.0","0.4.1","0.4.2","0.5.0","0.5.1","0.5.2","0.6.0","0.7.0","0.8.0","0.8.0.dev0","0.8.0.dev1","0.8.0.dev2","0.8.0rc1","0.8.0rc2","0.8.0rc3","0.8.0rc4","0.8.1","0.8.1rc1","0.8.1rc2","0.9.0","0.9.0.dev0","0.9.0.dev1","0.9.0.dev2","0.9.0.dev3","0.9.0.dev4","0.9.0rc1","0.9.0rc2","0.9.1","0.9.2","0.9.3","0.9.4"],"fixed":["0.23.1+rhlw.1"],"provider":"osv","source":"RHLW-2026-b11abda52745af4c","relation":"self"}],"affected_other":[{"ecosystem":"CVE","package":"huggingface/tokenizers","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"last_affected":"0.23.2"}]}],"provider":"cvelistv5","source":"CVE-2026-85670","relation":"self"}],"product_status":[{"vendor":"Red Hat","product":"All currently supported Red Hat products","cpe":"cpe:/a:redhat","status":"not_affected","packages":[],"package_count":0,"fixed":[],"advisories":[],"note":"vulnerable_code_not_present","provider":"redhat-vex","source":"CVE-2026-85670"}],"records":[{"provider":"cvelistv5","source_id":"CVE-2026-85670","relation":"self","url":"https://www.cve.org/CVERecord?id=CVE-2026-85670","license":"CC-BY-4.0","published_at":"2026-09-04T14:32:25.391Z","modified_at":"2026-09-04T17:49:42.263Z","summary":"tokenizers BpeBuilder Buffer Overflow via merge token","details":"tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary key, then writes each concatenated merge rule into it. A merge whose concatenated token exceeds the longest vocabulary key overruns the buffer, which Rust turns into a panic that aborts the process in Rust and FFI embeddings. This occurs at load time with no encoding required, so an attacker who supplies a crafted tokenizer.json can cause a denial of service. A secondary defect at the same location can cause a usize underflow (panic in debug, potential memory corruption in release) when continuing_subword_prefix is set and a merge token is shorter than the prefix. Observed in version 0.23.1.","severity":[{"type":"CVSS_V4","score":"CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N","source":"CVE-2026-85670"},{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H","source":"CVE-2026-85670"}],"affected":[{"ecosystem":"CVE","package":"huggingface/tokenizers","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"last_affected":"0.23.2"}]}],"provider":"cvelistv5","source":"CVE-2026-85670","relation":"self"}],"product_status":[],"references":[{"type":"WEB","url":"https://github.com/huggingface/tokenizers/issues/2094"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs"},{"type":"WEB","url":"https://www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token"}]},{"provider":"osv","source_id":"RHLW-2026-b11abda52745af4c","relation":"self","url":"https://osv.dev/vulnerability/RHLW-2026-b11abda52745af4c","license":"CC-BY-4.0","published_at":"2026-10-19T14:27:21Z","modified_at":"2026-10-27T08:45:03.134558Z","summary":"Lightwell Security Advisory: tokenizers 0.23.1+rhlw.1","details":"Red Hat Lightwell has released tokenizers 0.23.1+rhlw.1 with a backported security fix.\n\nCVE-2026-85670: tokenizers BpeBuilder Buffer Overflow via merge token\n\nThis patched artifact is a drop-in replacement for tokenizers 0.23.1 from PyPI.","severity":[{"type":"CVSS_V4","score":"CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:P/VC:N/VI:N/VA:H/SC:N/SI:N/SA:N","source":"RHLW-2026-b11abda52745af4c"},{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:H","source":"RHLW-2026-b11abda52745af4c"}],"affected":[{"ecosystem":"Red Hat Lightwell:PyPI","package":"tokenizers","purl":"pkg:pypi/tokenizers","ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"0.23.1+rhlw.1"}]}],"versions":["0.0.1","0.0.10","0.0.11","0.0.12","0.0.13","0.0.2","0.0.3","0.0.4","0.0.5","0.0.6","0.0.7","0.0.8","0.0.9","0.1.0","0.1.1","0.10.0","0.10.1","0.10.2","0.10.3","0.11.0","0.11.1","0.11.2","0.11.3","0.11.4","0.11.5","0.11.6","0.12.0","0.12.1","0.13.0","0.13.1","0.13.2","0.13.3","0.14.0","0.14.1","0.15.0","0.15.1","0.15.2","0.19.0","0.19.1","0.2.0","0.2.1","0.20.0","0.20.1","0.20.2","0.20.3","0.20.4","0.20.4rc0","0.21.0","0.21.0rc0","0.21.1","0.21.1rc0","0.21.2","0.21.2rc0","0.21.4","0.22.0","0.22.0rc0","0.22.1","0.22.1rc0","0.22.2","0.22.2rc0","0.23.0rc0","0.23.1","0.23.1rc0","0.3.0","0.4.0","0.4.1","0.4.2","0.5.0","0.5.1","0.5.2","0.6.0","0.7.0","0.8.0","0.8.0.dev0","0.8.0.dev1","0.8.0.dev2","0.8.0rc1","0.8.0rc2","0.8.0rc3","0.8.0rc4","0.8.1","0.8.1rc1","0.8.1rc2","0.9.0","0.9.0.dev0","0.9.0.dev1","0.9.0.dev2","0.9.0.dev3","0.9.0.dev4","0.9.0rc1","0.9.0rc2","0.9.1","0.9.2","0.9.3","0.9.4"],"fixed":["0.23.1+rhlw.1"],"provider":"osv","source":"RHLW-2026-b11abda52745af4c","relation":"self"}],"product_status":[],"references":[{"type":"ADVISORY","url":"https://github.com/CVEProject/cvelistV5/tree/main/cves/2026/85xxx/CVE-2026-85670.json"},{"type":"ADVISORY","url":"https://nvd.nist.gov/vuln/detail/CVE-2026-85670"},{"type":"REPORT","url":"https://github.com/huggingface/tokenizers/issues/2094"},{"type":"PACKAGE","url":"https://github.com/huggingface/tokenizers"},{"type":"ARTICLE","url":"https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs"}]},{"provider":"redhat-vex","source_id":"CVE-2026-85670","relation":"self","url":"https://access.redhat.com/security/cve/CVE-2026-85670","license":"CC-BY-4.0","published_at":"2026-09-04T14:32:25.391Z","modified_at":"2026-11-05T17:25:54Z","summary":"tokenizers: Hugging Face tokenizers: Denial of service and potential memory corruption via crafted file","details":"A flaw was found in the Hugging Face tokenizers library. A remote attacker could provide a specially crafted `tokenizer.json` file, which, when processed, leads to an out-of-bounds buffer access. This vulnerability can cause the application to terminate, resulting in a denial of service. In certain configurations, a related defect could also lead to memory corruption.","severity":[{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:L/I:L/A:H","source":"CVE-2026-85670"}],"affected":[],"product_status":[{"vendor":"Red Hat","product":"All currently supported Red Hat products","cpe":"cpe:/a:redhat","status":"not_affected","packages":[],"package_count":0,"fixed":[],"advisories":[],"note":"vulnerable_code_not_present","provider":"redhat-vex","source":"CVE-2026-85670"}],"references":[{"type":"WEB","url":"https://security.access.redhat.com/data/csaf/v2/vex/2026/cve-2026-85670.json"},{"type":"WEB","url":"https://access.redhat.com/security/cve/CVE-2026-85670"},{"type":"WEB","url":"https://www.cve.org/CVERecord?id=CVE-2026-85670"},{"type":"WEB","url":"https://nvd.nist.gov/vuln/detail/CVE-2026-85670"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers/issues/2094"},{"type":"WEB","url":"https://www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token"}]}],"references":[{"type":"WEB","url":"https://github.com/huggingface/tokenizers/issues/2094"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers"},{"type":"WEB","url":"https://github.com/huggingface/tokenizers/blob/v0.23.2/tokenizers/src/models/bpe/model.rs"},{"type":"WEB","url":"https://www.vulncheck.com/advisories/tokenizers-bpebuilder-buffer-overflow-via-merge-token"},{"type":"ADVISORY","url":"https://github.com/CVEProject/cvelistV5/tree/main/cves/2026/85xxx/CVE-2026-85670.json"},{"type":"ADVISORY","url":"https://nvd.nist.gov/vuln/detail/CVE-2026-85670"},{"type":"WEB","url":"https://security.access.redhat.com/data/csaf/v2/vex/2026/cve-2026-85670.json"},{"type":"WEB","url":"https://access.redhat.com/security/cve/CVE-2026-85670"},{"type":"WEB","url":"https://www.cve.org/CVERecord?id=CVE-2026-85670"}],"sources":[{"provider":"cvelistv5","source_id":"CVE-2026-85670","url":"https://www.cve.org/CVERecord?id=CVE-2026-85670","license":"CC-BY-4.0","modified_at":"2026-09-04T17:49:42.263Z","relation":"self"},{"provider":"osv","source_id":"RHLW-2026-b11abda52745af4c","url":"https://osv.dev/vulnerability/RHLW-2026-b11abda52745af4c","license":"CC-BY-4.0","modified_at":"2026-10-27T08:45:03.134558Z","relation":"self"},{"provider":"redhat-vex","source_id":"CVE-2026-85670","url":"https://access.redhat.com/security/cve/CVE-2026-85670","license":"CC-BY-4.0","modified_at":"2026-11-05T17:25:54Z","relation":"self"}]}
