Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Alpine apk passthrough

How yoe consumes prebuilt Alpine packages through module-alpine, and the sharp edges around metadata, noarch routing, and -dev subpackages. Read this before adding a new feed, debugging a “no such package” install error, or expanding the Alpine surface beyond module-core’s source-built userland.

See Catalog and Materialization for the resolver-side mechanics that back the passthrough path — how alpine_feed registers a synthetic module, when units actually materialize, and what the working set looks like. This page focuses on what passes through and why.

Why this exists

Yoe started by treating every package as something it builds from source — each unit produced a $DESTDIR of files and internal/artifact/apk.go packaged that destdir into a fresh .apk with a yoe-generated PKGINFO and a project-key signature. For module-alpine units (which fetch a prebuilt Alpine apk), the same path applied: extract Alpine’s apk into $DESTDIR, then rebuild a yoe-flavoured apk on top.

That works as long as Alpine’s PKGINFO is just a list of names — but Alpine’s apks carry a lot more:

  • replaces = busybox so two packages can both ship a path without apk failing the install.
  • provides = so:libcrypto.so.3=3.5.4-r0 so packages that link against a shared library find their dep cleanly.
  • provides = cmd:sh=1.37.0-r14 so file-deps like /bin/sh resolve.
  • triggers = /usr/lib/firmware* so kernel module updates re-fire hot-plug helpers.
  • .pre-install / .post-install / .trigger shell scripts that run at install-time inside a chroot of the rootfs.

Regenerating PKGINFO from a hand-written .star drops every field the generator didn’t enumerate. Most visibly: busybox’s .post-install is where applet symlinks (/bin/sh, /sbin/init, …) get created via /bin/busybox --install -s. Without it the kernel boots and finds no working init.

The current architecture — passthrough — sidesteps all of this by publishing Alpine’s apk verbatim, only swapping the signature.

Passthrough, in two pieces

1. alpine_feed materializes synthetic units lazily

module-alpine/MODULE.star is two alpine_feed(...) calls — one per Alpine repo section (main, community) — and a small units/*-enable.star companion layer for service enablement. No per-package .star files; the ~3,700-entry Alpine main catalog is checked in as one decompressed APKINDEX text file per arch.

# module-alpine/MODULE.star (production shape)
alpine_feed(
    name    = "main",
    url     = "https://dl-cdn.alpinelinux.org/alpine",
    branch  = "v3.21",
    section = "main",
    index   = "feeds/main",                            # in-tree APKINDEX dir
    keys    = ["keys/alpine-devel@lists.alpinelinux.org-6165ee59.rsa.pub"],
)

Each alpine_feed call registers a SyntheticModule named <parent>.<feed-name> (alpine.main, alpine.community). The resolver materializes a *Unit on demand the first time a closure walk references the package name — see Catalog and Materialization for the cache + lookup flow. The materialized unit carries PassthroughAPK = "<pkgname>-<pkgver>.apk" and an install task constructed from the upstream APKINDEX entry.

The companion layer is small and hand-written. Today: units/docker-enable.star and units/navidrome-enable.star — units with no build steps that declare services = ["docker"] / ["navidrome"] so the runlevel symlink lands in the package’s data tar. See the “package-driven service enablement” decision in CLAUDE.md for why service enablement is its own unit rather than a flag on the feed.

2. The executor calls RepackAPK instead of CreateAPK

internal/build/executor.go branches on unit.PassthroughAPK:

  • Empty → artifact.CreateAPK(destDir, ...) — the original “build a fresh apk” path. Source-built module-core units take this path.
  • Set → artifact.RepackAPK(srcAPK, ...) — splits Alpine’s apk into its three concatenated gzip streams, drops the Alpine signature, re-signs the control stream (PKGINFO + install scripts) with the project key, and concatenates [new_sig, control, data] into the published apk.

RepackAPK pipeline

RepackAPK does not rewrite anything inside the control or data segments. Alpine’s PKGINFO, replaces, provides, triggers, install scripts, file checksums — all unchanged.

The destdir extraction (the materialized unit’s install task) is still useful: yoe’s per-unit sysroots are built by hardlinking each dep’s destdir into <unit>/sysroot/, so a unit that gcc -lfoos against a module-alpine library finds headers and shared objects there. Image-time apk add reads the published apk (passthrough) and never looks at the destdir.

Legacy: alpine_pkg class

classes/alpine_pkg.star is still in the module but no production unit uses it. It was the pre-feed entry point — one hand-written .star file per package, calling alpine_pkg(...) with hashes baked in. The class is preserved for one-off external uses (a downstream module that wants to wrap a single upstream apk without standing up a whole feed); for the in-tree module-alpine, feeds replaced it. If you’re tempted to add a per-package .star calling alpine_pkg, ask first whether the feed should have the package already — adding a hand-written wrapper for a name the feed already exposes will produce a conflicting registration.

Two metadata systems, two purposes

After passthrough, every unit has metadata in two places:

  • The materialized *Unit (Name, Version, RuntimeDeps, Provides, Replaces, Distro, …) drives yoe’s resolver: build-order DAG, runtime-closure walk for image artifacts, virtual-package routing (linux → linux-rpi4), TUI USED-BY/PULLS-IN trees, the per-image distro visibility filter (an alpine image’s closure doesn’t see debian-tagged units and vice versa). For feed-materialized units this view is constructed from the APKINDEX entry at lookup time (apkindex.MaterializeUnit in internal/apkindex/); for source-built and companion units, it’s whatever the .star file declared.
  • Upstream PKGINFO (inside the apk’s control segment) drives apk-tools at install time on the target: real shared-library deps, file-dep resolution, install-script execution, conflict checking.

These overlap conceptually but serve different stages. yoe’s resolver doesn’t see so:libcrypto.so.3 because it’s not in the materialized unit’s runtime_deps; apk-tools doesn’t see yoe’s virtual linux because that’s a yoe concept, not an apk one.

The materialized unit therefore should mirror enough of upstream’s metadata that yoe’s resolver makes the same decisions apk-tools would — without duplicating every field. For feed-materialized units this happens at lookup time: MaterializeUnit reads the APKINDEX entry’s dep list, runs each token through the project-wide provides table (cross-feed siblings included via multiFeedProviders), and produces a *Unit whose RuntimeDeps name yoe units rather than apk virtuals. Hand-written companion units like docker-enable set yoe-specific overrides (services = ["docker"]) that the feed couldn’t infer from APKINDEX alone.

noarch routing — the four-part fix

apk-tools is unforgiving here: it constructs fetch URLs from PKGINFO’s arch field, not from where it found the index entry. So a noarch apk has to physically live at <repo>/noarch/<file> — putting it under <repo>/<arch>/ and listing it in <arch>/APKINDEX doesn’t make apk look there. Conversely, apk’s solver only reads one arch’s APKINDEX per repository invocation, so noarch entries also have to appear in the per-arch index.

The full design now in tree:

  1. executor.go routes noarch passthrough apks to <repo>/noarch/. The arch comes from upstream PKGINFO via artifact.ReadAPKArch, not from the build arch.
  2. GenerateIndex scans the sibling <repo>/noarch/ tree when building a per-arch index, so each arch’s APKINDEX advertises every noarch package as A:noarch. apk’s solver finds the entry from any per-arch index.
  3. Publish regenerates every per-arch APKINDEX after a noarch publish. Without this, the per-arch indexes go stale on every noarch unit rebuild.
  4. cacheValid looks under <repo>/noarch/ when the apk isn’t in the per-arch dir, so noarch passthrough units don’t rebuild on every yoe build invocation.

Symptoms when one of these halves is missing:

  • package mentioned in index not found — usually file in arch dir but PKGINFO says noarch (apk fetches from <base>/noarch/, 404s).
  • <name> (no such package): required by world[<name>] — file in noarch dir but per-arch APKINDEX doesn’t reference it.
  • noarch unit shows [building] every run despite the published apk being unchanged — cacheValid was checking the wrong directory.

Auto-emitted so: provides

For yoe-source-built units, internal/artifact/apk.go walks $DESTDIR, opens every regular file with Go’s debug/elf, reads DT_SONAME, and emits one provides = so:<soname>=<ver>-r<rel> line per shared library. This matches Alpine’s abuild convention and lets Alpine prebuilts that declare depend = so:libcrypto.so.3 resolve cleanly against a yoe-built openssl.

The mirror — auto-emit depend = so:<soname> from DT_NEEDED — is on the roadmap (see docs/roadmap.md’s “Auto-depend from ELF DT_NEEDED”).

Auto-versioned provides

Explicit virtuals listed in a unit’s provides = [...] field are also stamped with =<ver>-r<rel> on emit. So:

unit(name = "openssl", version = "3.4.1", provides = ["libssl3", "libcrypto3"])

emits provides = libssl3=3.4.1-r0 and provides = libcrypto3=3.4.1-r0 in the package’s PKGINFO. Without the version, apk treats the provide as unversioned and refuses to use it to satisfy consumer constraints like depend = libssl3>=3.3.0 (which is what Alpine’s prebuilt python3 ships). If the entry already carries a constraint (=, <, >, ~), it’s emitted verbatim — useful when a unit needs to claim a specific older version’s API contract.

Worked example: why we couldn’t use Alpine’s docker-openrc

Tested end-to-end during the OpenRC switch. Documenting because the same shape of problem will recur with any Alpine package whose dep tree pokes deep enough into module-core’s source-built userland.

The goal: wire dockerd into the OpenRC default runlevel using Alpine’s docker-openrc package (which ships /etc/init.d/docker and a /etc/conf.d/docker config template Alpine maintains).

The dep tree (drawn out from upstream PKGINFOs):

docker-openrc
└── log_proxy
    ├── musl
    └── glib
        ├── pcre2
        ├── libffi
        ├── libintl
        └── libmount
            └── libblkid

log_proxy is a tiny Alpine utility for capturing daemon stdout/stderr to syslog. Glib is needed because log_proxy uses GIO. libmount/libblkid are in glib’s transitive deps because GIO has mount-table integration.

The conflict. Yoe’s source-built util-linux (in module-core) ships libmount.so.1 and libblkid.so.1 directly — it’s a monolithic build. Alpine splits util-linux: libmount and libblkid are separate apks. When apk’s solver tries to install both yoe’s util-linux and Alpine’s libmount/libblkid, it fails because both packages own the same library paths.

First attempt: prefer_modules = {"alpine": {"util-linux": "alpine.main"}}. Forces the Alpine prebuilt instead of yoe’s source-built version (pin syntax names the synthetic module, not the parent — alpine.main rather than alpine). Resolves the library conflict. But Alpine’s util-linux apk is a meta package — it ships nothing on disk; the actual binaries live in util-linux-misc, util-linux-login, the libraries in libuuid/libmount/libblkid, and the headers + unversioned .so symlinks needed at compile time live in util-linux-dev.

After pulling subpackages in via runtime_deps, the next layer: e2fsprogs (yoe-source-built) needs libuuid headers + the unversioned libuuid.so symlink to compile. Those live in util-linux-dev. Adding that pulls libfdisk, liblastlog2, libsmartcols, sqlite-dev — none of which have yoe units. Each one in turn pulls more.

The yak shave. To fully consume Alpine’s util-linux-dev, we’d need units for at least a dozen Alpine subpackages, plus their -dev counterparts, plus careful conflict bookkeeping where yoe-source-built packages still ship competing files. That’s days of work and a much larger Alpine surface to maintain.

The trade. A 30-line yoe-side OpenRC service script (in modules/module-core/units/net/docker-init/) gives us the same boot behaviour with no transitive deps. We give up Alpine’s /etc/conf.d/docker config template and the log_proxy stdout-capture story; for a yoe image those costs are minor.

The lesson generalizes: Alpine’s -dev subpackage convention is fundamentally at odds with yoe’s monolithic source-built libraries. Picking off Alpine packages one at a time is fine; widening the surface to consume Alpine’s whole library-development ecosystem is a significant architecture decision, not a one-off fix.

What’s still rough

Items where the architecture is “works for now” but obviously incomplete.

  • No support for -dev packages. All the architectural reasons in the docker-openrc example. Until yoe has a story for splitting headers out of source-built libraries (or for systematically wrapping Alpine’s -dev ecosystem), pulling new Alpine packages in is a manual review for “does this transitively need any -dev subpackage.”

  • No triggers execution. Alpine apks ship .trigger scripts that fire on path changes (e.g. udev re-runs hot-plug rules when a module is added). The passthrough copy includes them, but yoe’s image assembly doesn’t currently invoke them in any consistent way. apk’s on-target trigger machinery handles them after first boot, but image-build-time triggers (-t in apk) don’t happen.

  • Auto-depend from DT_NEEDED. Counterpart to the auto-so:-provides scan that already runs. Would catch the class of bug where a unit’s RuntimeDeps silently misses a transitive shared-lib dependency. Roadmap item; design in docs/roadmap.md.

  • prefer_modules with subpackage expansion. When you push a monolithic source-built unit (util-linux) to Alpine’s split form, yoe’s resolver follows runtime_deps from the meta package — but build-time deps (unit.Deps) on util-linux don’t pull subpackage destdirs into the build sysroot. Workaround: declare downstream units’ build deps on the subpackages directly. Long-term the resolver should walk runtime_deps for build-deps too, or unit.Deps should accept the same expansion.

  • RepackAPK re-sign vs upstream-keys passthrough. The current path strips Alpine’s signature and re-signs with the project key. An alternate design — keep Alpine’s keys + signature, ship Alpine’s keyring on-target as an additional trust anchor — is captured in docs/specs/2026-05-18-mirror-alpine-keep-keys.md (spec only, not implemented). The trade-off: re-signing means the on-target trust list stays “one key per project” but every passthrough requires cryptographic work; keeping upstream keys removes the re-sign cost but multiplies on-target trust anchors (one per distro the project consumes from). No decision yet on which way to land.

Reference

  • internal/artifact/apk.go — CreateAPK, RepackAPK, ReadAPKArch, scanSONAMEs.
  • internal/feeds/alpine/builtin.go — alpine_feed builtin, SyntheticModule registration, populateBuildFields (sets PassthroughAPK, Source URL, Distro = "alpine", install task with control-file exclusions). The pre-feed wrapper class lives at testdata/.../module-alpine/classes/alpine_pkg.star for legacy / external one-off use.
  • internal/apkindex/materialize.go — MaterializeUnit: APKINDEX entry → *Unit. Wraps multiFeedProviders for cross-feed dep resolution.
  • internal/build/executor.go — passthrough branch in the build loop; cacheValid for the noarch lookup.
  • internal/repo/local.go — Publish (with cross-arch reindex on noarch) and index.go’s GenerateIndex (sibling-noarch scan).
  • internal/feeds/alpine/update.go — yoe update-feeds driver: fetches upstream APKINDEX, verifies RSA-SHA1 against declared keys, atomically writes to the in-tree feed index.
  • docs/catalog.md — resolver-side mechanics (synthetic modules, lazy materialization, working-set sizes, invariants).
  • docs/module-alpine.md — when to reach for module-alpine vs module-core (rubric, not architecture); maintainer playbook for yoe update-feeds.