dtoro
d4d99a7473
Problem: the hexagonal refactor churns the backend tree for nine more phases; the UI delivery stack (web/ SPA, cmd/desktop Wails wrapper, compose/web image) must move to its own repo first so doc/layout rewrites land once on a backend-only tree. Change: - New repo git.hubris.network/dtoro/oikos-web (v0.33.0): web/, desktop/ (updateURL repointed to oikos-web releases), compose/, own CI (web + desktop jobs), own deploy script (CI-green gate, TOCTOU guard, version-tagged images, prune-to-3), own webhook receiver on :9798 + launchd unit, own compose project publishing the same 8091:80. - Cutover executed on mac-mini in order: oikos stack's web service stopped+removed, oikos-web project brought up on 8091; outer Caddy untouched (targets the published port) — serving + Authentik flow + /wails 404 quirk verified post-cutover. - Stripped from oikos: web/, cmd/desktop/, compose/web/, desktop CI workflow, ci.yml web job, Makefile ui/desktop/desktop-package/install targets, the compose web service, oikos-web from deploy.sh's fallback prune list; wails + go-keyring dropped from go.mod, vendor synced. - README / CONTRIBUTING / AGENTS.md / .agents dev+operations docs now point at the new repo; mbse + mascot design docs carry a path note. Risk: production SPA serving depends on the new pipeline now; rollback is versioned-image re-up of the old web service from a pre-split checkout (port 8091). Desktop builds installed before the split still check dtoro/oikos releases — one manual reinstall, noted in the oikos-web release notes. Verification: go vet, make test (race), make generate-check, golangci (no new findings; baseline down 400→365); post-cutover curls — localhost:8091 200, /wails/runtime.js 404, outer Caddy 302 Authentik.
41 lines
2.2 KiB
Markdown
41 lines
2.2 KiB
Markdown
# Determine the Encoding of a HTML Byte Stream
|
|
|
|
This package implements the HTML Standard's [encoding sniffing algorithm](https://html.spec.whatwg.org/multipage/syntax.html#encoding-sniffing-algorithm) in all its glory. The most interesting part of this is how it pre-scans the first 1024 bytes in order to search for certain `<meta charset>`-related patterns.
|
|
|
|
```js
|
|
const htmlEncodingSniffer = require("html-encoding-sniffer");
|
|
const fs = require("fs");
|
|
|
|
const htmlBytes = fs.readFileSync("./html-page.html");
|
|
const sniffedEncoding = htmlEncodingSniffer(htmlBytes);
|
|
```
|
|
|
|
The passed bytes are given as a `Uint8Array`; the Node.js `Buffer` subclass of `Uint8Array` will also work, as shown above.
|
|
|
|
The returned value will be a canonical [encoding name](https://encoding.spec.whatwg.org/#names-and-labels) (not a label). You might then combine this with the [whatwg-encoding](https://github.com/jsdom/whatwg-encoding) package to decode the result:
|
|
|
|
```js
|
|
const whatwgEncoding = require("whatwg-encoding");
|
|
const htmlString = whatwgEncoding.decode(htmlBytes, sniffedEncoding);
|
|
```
|
|
|
|
## Options
|
|
|
|
You can pass two potential options to `htmlEncodingSniffer`:
|
|
|
|
```js
|
|
const sniffedEncoding = htmlEncodingSniffer(htmlBytes, {
|
|
transportLayerEncodingLabel,
|
|
defaultEncoding
|
|
});
|
|
```
|
|
|
|
These represent two possible inputs into the [encoding sniffing algorithm](https://html.spec.whatwg.org/multipage/syntax.html#encoding-sniffing-algorithm):
|
|
|
|
- `transportLayerEncodingLabel` is an encoding label that is obtained from the "transport layer" (probably a HTTP `Content-Type` header), which overrides everything but a BOM.
|
|
- `defaultEncoding` is the ultimate fallback encoding used if no valid encoding is supplied by the transport layer, and no encoding is sniffed from the bytes. It defaults to `"windows-1252"`, as recommended by the algorithm's table of suggested defaults for "All other locales" (including the `en` locale).
|
|
|
|
## Credits
|
|
|
|
This package was originally based on the excellent work of [@nicolashenry](https://github.com/nicolashenry), [in jsdom](https://github.com/tmpvar/jsdom/blob/16fd85618f2705d181232f6552125872a37164bc/lib/jsdom/living/helpers/encoding.js). It has since been pulled out into this separate package.
|