Cairn: A Local-First, Immutable Dependency Manager

Cairn is a package manager for compiled languages (starting with Rust and Go) that never requires a network connection after initial setup. Every package version is a set of content-addressed blobs, hashed with BLAKE3. The local store is a simple object database similar to Git's, with metadata in SQLite. When you cairn install foo, the resolver first checks the local store; if a blob is missing, it broadcasts a request to peers on the LAN using mDNS and a DHT. Peers that have the blob serve it without any central coordination. The 'registry' is just a static, signed text file mapping package names to manifest hashes—fetched once and cached indefinitely. Updates are opt-in and manual; there are no mutable 'latest' tags. The result: a development experience that works on a plane, in a basement, or during a cloud outage. Version history is never deleted, so old builds remain possible. Blob deduplication across versions means minimal local storage overhead. Cairn doesn't replace existing registries—it mirrors them locally and uses P2P to accelerate transfers.

The vision is radical but practical: treat the network as a peer-to-peer cache, not a source of truth. The source of truth is the content hash. This eliminates registry single points of failure, makes CI reproducible offline, and gives developers true freedom from cloud dependencies.

Modern package managers (npm, pip, cargo) depend on centralised cloud registries that fail under load, block regional access, or go down. 'Works offline' is a second-class feature—at best a partial cache. Developers in low-connectivity environments (trains, remote areas, developing nations) are locked out. Even on fast internet, repeated downloads of the same bytes waste bandwidth. P2P solutions like 'npm install from a mirror' are bolted on. Cairn starts from the principle that the remote registry should be a one-time snapshot; everything else is local or peer-sourced.

• Work 100% offline after initial registry snapshot and cached packages.
• Every package version is immutable and content-addressed (blake3 hash).
• LAN peer-to-peer sharing without any central server.
• Never require an internet connection for install, build, or update.
• Verifiable package provenance via TUF-style signing.
• Backward-compatible with existing registries as static mirrors.
• Deduplicate blobs across package versions to minimise disk usage.
• Support for arbitrary compiled languages (via plugins for resolution logic).

• Not a build system or a general dependency resolver (each language brings its own).
• Not intended for cloud-hosted live registries (Cairn treats them as one-time snapshots).
• Not a replacement for CI/CD or runtime package management.
• No automatic dependency upgrades (explicit pinning only).
• No central index of available packages—discovery relies on the snapshot.
• No support for mutable packages or 'latest' tags.

• Rust for core (performance, safety, cross-platform).
• libp2p (rust-libp2p) for P2P networking (mDNS, Kademlia DHT, Bitswap).
• SQLite for local metadata database (via rusqlite).
• BLAKE3 for content-addressing and integrity.
• TUF (The Update Framework) for signing and verification.
• Protobuf for manifest serialization.
• Clap for CLI argument parsing.
• Tokio async runtime for networking.

Cairn's architecture has four layers: the Content-Addressable Store (CAS), the Resolver, the P2P Layer, and the CLI. The CAS is a flat storage directory of blobs keyed by their BLAKE3 hash, plus a SQLite database mapping (package name, version) -> root manifest hash. The Resolver reads the SQLite metadata to find required blobs. If a blob is missing locally, it requests it from the P2P Layer. The P2P Layer uses libp2p: it starts a local node, discovers peers on the LAN via mDNS, and advertises its own blobs via a Bitswap-like protocol. When a blob is needed, Cairn broadcasts a hash and waits for a reply. If no peer responds, the CLI reports the lack as 'cannot resolve; need internet for initial fetch'. Once fetched, the blob is stored in the CAS and never removed. The registry snapshot is a signed JSON file that contains a list of all known packages and their latest manifest hashes. This file is fetched once (via HTTP or file) and then cached. All operations are local; the network is an optimization to avoid manual downloads.

Example flow: cairn install gopkg/yaml.v2@1.0.0 triggers the resolver to check the local SQLite for the root manifest hash. If present, it walks the dependencies, fetching missing blobs via P2P. If not, it queries the registry snapshot for the entry, downloads the manifest, stores it, and then continues. The upgrade path ('cairn update') fetches a new registry snapshot from an optional remote (the user's choice) and updates the local SQLite with new manifest hashes. All old hashes remain in the CAS for reproducibility.

• Adoption requires a critical mass of packages and peers; early days may feel isolated.
• P2P traffic may be blocked by corporate firewalls or IoT networks; fallback to manual imports needed.
• Developers accustomed to mutable 'latest' tags may rebel; cultural shift required.
• Storage growth: without deletion, the CAS grows unbounded. We rely on the user to prune (but old builds need those blobs).
• Malicious package injection: without a central authority, verifying signing keys is harder. TUF helps but trust model must be explicit.
• Legal: some licenses require a registry to take down packages; immutable snapshots conflict with 'right to be forgotten'.

1. How should private (non-public) packages work? A separate namespace or encryption at rest?
2. Should Cairn support multiple registries (snapshots) simultaneously, or is a single global snapshot the only way?
3. How do we handle key rotations for signing without breaking existing verifications?
4. What's the best strategy for reclaiming disk space when users explicitly want to remove a package? (delete blobs or just remove metadata?)
5. Should we allow 'remote' P2P (over internet) or strictly LAN? Internet P2P introduces anonymity and spam considerations.
6. How to handle version conflict resolution when two dependencies require different minor versions of the same library?

We had a working prototype for Go packages that could install a simple project from a local cache and serve blobs between two laptops. But then our startup was acquired, and the acquirer's business model was built on cloud services—they had no interest in an offline-first tool. The code sits in a private repo. The trust model (signing keys, revocation) was never finalised to everyone's satisfaction, so we never felt it was ready for public beta. It's a plan that deserves to exist, but we moved on.

The key insight is that most packages are rarely updated; the long tail of dependencies is almost static. By treating the registry as a snapshot, we eliminate the need for continuous connectivity. The P2P layer is inspired by the IPFS Bitswap protocol but adapted for LAN. The name 'Cairn' suggests a stack of stones that you build and leave as a landmark—immutable, permanent, and communal.