Overview/xpkg — package builder

xpkg - Architecture

How the project is structured, how a package is produced, and the formats it reads and writes.

Related documents in this folder:

• 01 - Overview and status
• 02 - Usage
• 04 - Future integration

Cargo workspace layout

xpkg is a Cargo workspace (edition 2021) with two crates:

The workspace root Cargo.toml centralises shared dependencies and metadata (version 0.1.0, edition 2021, GPL-3.0-or-later, org equislinux). Notable third-party dependencies: clap (CLI), serde/serde_json/toml, thiserror/anyhow (errors), tracing (logging), ureq (HTTP), sha2 (checksums), flate2/tar/xz2/bzip2/zstd/zip (archives), sequoia-openpgp (signing), tempfile (tests).

Modules in xpkg-core

The build pipeline

xpkg build orchestrates, in order:

1. Parse and validate the recipe (XBUILD or PKGBUILD) and run the recipe-level lint (source-unpinned warnings are reported, never fatal).
2. Apply CLI overrides for builddir/outdir.
3. Set up isolated build directories and environment.
4. Run the build phases: prepare then build then check (optional) then package, executing each recipe phase as shell scripts.
5. Strip ELF binaries (if strip_binaries = true).
6. Create the .xp archive with extended .BUILDINFO provenance (x:recipe_sha256, x:source_commit, x:tool_version).
7. Sign the package (if --sign or sign = true in config).

Rootless packaging

The package() phase writes into a fakeroot context so files are recorded with uid=0/gid=0 without real root privileges. xpkg uses a 3-layer fallback: unshare --user (kernel namespaces, Linux >= 3.8) when available, otherwise the fakeroot tool, otherwise direct execution with tar header rewriting.

Environment

The builder sets PKGDIR, SRCDIR, BUILDDIR, MAKEFLAGS, CFLAGS, CXXFLAGS and LDFLAGS for the phase scripts. The package() phase must install everything into $PKGDIR (never /).

The .xp package format

.xp is an ALPM-compatible compressed tar archive (tar.zst by default). At the archive root it carries the metadata files generated by the metadata module:

package-1.0-1-x86_64.xp (tar.zst)
+-- .PKGINFO       package identity, version, dependencies, sizes
+-- .BUILDINFO     build environment record (packager, builddate, toolchain)
+-- .MTREE          file integrity manifest (hashes, permissions, ownership, symlinks)
+-- .INSTALL        optional pre/post install/upgrade/remove hook scripts
+-- usr/            installed file tree
+-- ...

Optional signing produces an OpenPGP detached signature file next to the archive (package-...-x86_64.xp.sig).

.BUILDINFO appends extended provenance keys after the historical key = value fields, ignored by readers that do not know them:

Repository database format

A repository is a set of .xp packages plus a database index that xpm can query: an ALPM-compatible compressed tar archive (.db.tar.zst by default; .db.tar.gz and .db.tar.xz are auto-detected). Inside, one directory per package holds desc (package metadata, %FILENAME%, %NAME%, %VERSION%, %DESC%, sizes, checksum, ...) and depends (dependency information), in an ALPM-compatible key-value format. The repo module reads/writes these databases and can generate a static repository layout for HTTP hosting.

Next to the database, xpkg maintains history.json (schema 1) with every version still available per package (version, filename, sha256, builddate, optional .sig and source provenance), signed as history.json.sig when a signing key is configured. repo-add --keep N and repo-prune use the index for retention: older versions listed there are swept from disk, but the version exposed by the database is never deleted. The deploy helper does not copy history-referenced versions yet, so publishing flows must keep the old .xp files in the deployed layout themselves.

The XBUILD recipe format

XBUILD is the native TOML recipe format (file XBUILD, TOML v1.0, UTF-8), structured into four top-level sections. See the full XBUILD Specification.

Example:

[package]
name = "hello"
version = "2.12"
release = 1
description = "GNU Hello - the friendly greeter"
url = "https://www.gnu.org/software/hello/"
license = ["GPL-3.0-or-later"]
arch = ["x86_64"]

[dependencies]
depends = ["glibc"]
makedepends = ["gcc", "make"]

[source]
urls = ["https://ftp.gnu.org/gnu/hello/hello-2.12.tar.gz"]
sha256sums = ["cf04af86dc085268c5f4470fbae49b18afbc221b78096aab842d934a76bad0ab"]

[build]
build = """
cd hello-2.12
./configure --prefix=/usr
make
"""
package = """
cd hello-2.12
make DESTDIR=$PKGDIR install
"""

Validation rules applied by the parser: name must follow the naming rules (lowercase ASCII start, lowercase letters/digits/hyphens/underscores, max 128); version non-empty; release >= 1; arch in x86_64, aarch64, i686, armv7h, any; source URL schemes in http, https, ftp, file, git, git+https, git+http; checksum arrays must match the urls length. Errors are collected and reported together, not one by one.

Git sources accept makepkg-style pins (#commit=, #tag=, #branch=); the exact commit of the first pinned source is resolved (local clone HEAD or git ls-remote) and recorded as x:source_commit. A source with neither a checksum nor a pinned commit/tag triggers the source-unpinned warning when the build starts, but never stops it.

When SOURCE_DATE_EPOCH is set to a valid Unix timestamp, it is used for the builddate of .PKGINFO/.BUILDINFO and for the mtime of every tar entry. Without it, the current time is used. This is reproducibility groundwork, not a full binary-reproducibility guarantee.

PKGBUILD compatibility

xpkg build --pkgbuild parses legacy Arch Linux PKGBUILD bash scripts and extracts variables (pkgname, pkgver, pkgrel, depends arrays, source, sha256sums) and functions (prepare, build, check, package) for migration from the Arch ecosystem.

Source handling

Declared sources are downloaded (HTTP/HTTPS/FTP/file via ureq, Git repos via the system git), verified with SHA-256 and/or SHA-512 (each entry index-matched to urls, SKIP bypasses), extracted by extension (tar.gz/tgz, tar.xz/txz, tar.bz2/tbz2, tar.zst/tzst, zip; other files kept as-is), and cached under $XDG_CACHE_HOME/xpkg/sources/ (keyed by a truncated SHA-256 of the URL) to avoid re-downloads. See Source Management.

Configuration model

Configuration lives in ~/.config/xpkg/xpkg.conf (TOML) with [options], [environment] and [lint] sections, loaded at startup and used to build an XpkgConfig. Subcommands clone the loaded config and apply CLI overrides before acting. See etc/xpkg.conf.example for all options.

Self-hosting recipe

The repository carries its own build recipe at packaging/xpkg/XBUILD: it copies the repo tree into the source dir, runs cargo build -p xpkg --release --locked, and installs the binary, license, README and etc/xpkg.conf.example into the package - an example of a [source]-free local recipe.

Edit this page on GitHub