html-encoding-sniffer
Sniff the encoding from a HTML byte stream
Last updated 3 years ago by domenic .
MIT · Repository · Bugs · Original npm · Tarball · package.json
$ gnpm install html-encoding-sniffer 
SYNC missed versions from official npm registry.

Determine the Encoding of a HTML Byte Stream

This package implements the HTML Standard's encoding sniffing algorithm in all its glory. The most interesting part of this is how it pre-scans the first 1024 bytes in order to search for certain <meta charset>-related patterns.

const htmlEncodingSniffer = require("html-encoding-sniffer");
const fs = require("fs");

const htmlBytes = fs.readFileSync("./html-page.html");
const sniffedEncoding = htmlEncodingSniffer(htmlBytes);

The passed bytes are given as a Uint8Array; the Node.js Buffer subclass of Uint8Array will also work, as shown above.

The returned value will be a canonical encoding name (not a label). You might then combine this with the whatwg-encoding package to decode the result:

const whatwgEncoding = require("whatwg-encoding");
const htmlString = whatwgEncoding.decode(htmlBytes, sniffedEncoding);

Options

You can pass two potential options to htmlEncodingSniffer:

const sniffedEncoding = htmlEncodingSniffer(htmlBytes, {
  transportLayerEncodingLabel,
  defaultEncoding
});

These represent two possible inputs into the encoding sniffing algorithm:

  • transportLayerEncodingLabel is an encoding label that is obtained from the "transport layer" (probably a HTTP Content-Type header), which overrides everything but a BOM.
  • defaultEncoding is the ultimate fallback encoding used if no valid encoding is supplied by the transport layer, and no encoding is sniffed from the bytes. It defaults to "windows-1252", as recommended by the algorithm's table of suggested defaults for "All other locales" (including the en locale).

Credits

This package was originally based on the excellent work of @nicolashenry, in jsdom. It has since been pulled out into this separate package.

Current Tags

  • 3.0.0                                ...           latest (3 years ago)

6 Versions

  • 3.0.0                                ...           3 years ago
  • 2.0.1                                ...           5 years ago
  • 2.0.0                                ...           5 years ago
  • 1.0.2                                ...           7 years ago
  • 1.0.1                                ...           8 years ago
  • 1.0.0                                ...           8 years ago
Downloads
Today 0
This Week 0
This Month 0
Last Day 0
Last Week 0
Last Month 0
Dependencies (1)
Dev Dependencies (3)
Dependents (1)

Copyright 2013 - present © cnpmjs.org | Home |