jerboa-drive

jdrive is a Jerboa-native, client-side encrypted drive backed by an S3-compatible object store. Version 2 has one remote layout: the object listing is the complete remote namespace, and every logical file is exactly one object. There is no remote catalog or local sync database.

Quick start

make deps
make run ARGS='help'

export AWS_PROFILE=personal
export AWS_REGION=us-east-1
export JDRIVE_VAULT_PASSWORD='<from-secret-manager>'
make run ARGS='s3 init --profile default --bucket my-private-bucket'
make run ARGS='s3 sync -v ./photos jd:/photos --delete'
unset JDRIVE_VAULT_PASSWORD

When the local path is omitted, s3 sync uses the current directory. For example, jd sync -v -d jd:mac2/ syncs . to jd:mac2/.

Inspect and restore files with s3 ls, s3 cat, s3 get, and s3 cp:

make run ARGS='s3 ls jd:/photos'
make run ARGS='s3 cat jd:/photos/readme.txt'
make run ARGS='s3 get jd:/photos/readme.txt restored.txt'

See the user guide for the full command set.

Installation

Build and install the complete native bundle:

make install
jdrive version

The installed ~/.local/bin/jdrive is a symlink to the bundle launcher under ~/.local/opt/jerboa-drive. Do not copy or rename the intermediate jdrive-bin executable into PATH: it requires the native libraries shipped beside it in the bundle. Running make install also replaces a stale standalone ~/.local/bin/jdrive left by an older or manual installation.

Stateless v2 storage

Each path component is deterministically encrypted with a name key derived from the profile drive key. This makes the remote object key a pure function of the logical path. A sync lists only the encrypted target directory, skips objects whose names do not authenticate, sorts compact (path . size) vectors, and merge-diffs them against the local scan.

File contents begin with an authenticated JDv2 JSON metadata header and are encrypted in independently authenticated 16 MiB frames. Multipart uploads pipeline those frames without buffering whole files. The fixed header and frame overhead let LIST object sizes map back to plaintext sizes, so a normal no-op sync performs LIST requests only. sync --checksum additionally reads the first encrypted frame of each size-equal file and compares its authenticated mtime.

Uploads run through a bounded worker dispatcher. Only the two sorted compact vectors and active transfer buffers scale with tree size. --delete removes remote-only objects in parallel S3 batches. Different directory prefixes have no shared mutable state and can be synchronized concurrently.

Data written by releases before 2.0 is not discovered by this program.

Configuration

Common environment variables are:

• AWS_PROFILE, AWS_REGION, AWS_DEFAULT_REGION, and the standard AWS credential variables supported by jerboa-aws.
• JDRIVE_S3_BUCKET, JDRIVE_S3_PREFIX, JDRIVE_S3_ENDPOINT, and JDRIVE_S3_PATH_STYLE for the object-store target.
• JDRIVE_PROFILE and JDRIVE_STATE_DIR for local profile selection.
• JDRIVE_JOBS for concurrent file transfers; the default is 4.
• JDRIVE_LOW_MEMORY=1 or s3 sync --low-memory to use one transfer worker.
• JDRIVE_HTTP_POOL_MAX, JDRIVE_HTTP_POOL_IDLE_MS, and JDRIVE_HTTP_TIMEOUT_MS for the HTTPS transport.

The HTTPS transport closes a connection after failed TLS I/O and retries the request once on a newly opened connection, whether the failed connection was pooled or already fresh. Persistent failures include the native Rustls I/O error in diagnostics. Set JDRIVE_HTTP_POOL_MAX=0 only when diagnosing endpoint-specific keep-alive behavior.

Endpoint overrides and path-style requests support MinIO, Hetzner Object Storage, and similar S3 services. Use HTTPS outside an explicit loopback test.

Remote paths use jd:. Both jd:photos and jd:/photos name /photos. Progress goes to stderr; the final JSON summary remains on stdout. -d adds object-key, retry, and transfer diagnostics and may expose logical local paths.

Security

The profile drive key is random and stored only in a local scrypt-protected, ChaCha20-Poly1305 vault. Optional YubiKey PIV material can supplement the vault password. File data, file metadata, and every name component are authenticated before use. See the security model for leakage and trust-boundary details.

Development

make test
make security
make binary
make binary-smoke

On macOS and other non-Linux platforms, make binary-smoke installs the bundle into a fresh temporary prefix and runs that installed launcher with dynamic-loader overrides removed. This catches missing or build-tree-only native libraries.

On macOS, make binary also embeds the crypto shim, so the resulting jdrive-bin can be copied directly to ~/.local/bin/jdrive and run without a companion crypto library. YubiKey support remains an optional weak dependency.

The opt-in live test creates and deletes objects in a real bucket:

JDRIVE_S3_INTEGRATION=1 make s3-integration

Use only a disposable prefix.

Repository layout:

• jdrive/s3/namecrypto.ss — deterministic path encryption and object header.
• jdrive/s3/syncv2.ss — streaming LIST, frame I/O, bounded sync, and object operations.
• jdrive/s3/drive.ss — profile vault and CLI-facing façade.
• jdrive/cli.ss — command parsing and dispatch.
• test/ — unit and opt-in S3 integration tests.

Dependencies are fetched at locked revisions into .deps/; build files never depend on adjacent source checkouts. VERSION is authoritative, and every change is merged through a reviewed Forgejo pull request.