Object Storage

RIGOR uses S3-compatible storage for large device artifacts and immutable scientific archives. SQLite remains the local development record store; object bytes are not embedded in Agent context or cross-service events.

Ownership and layout

sdlx-artifacts/
  operations/{operation_id}/{device_id}/{capability_id}/{filename}
  devices/{device_id}/{capability_id}/{operation_id}.{suffix}

sdlx-archives/
  pace/deliveries/{coordination_id}/{iteration_id}/{loop_run_id}/v{version}/
    result-bundle.json
    {delivery-name}.tar.gz
  prism/analyses/{analysis_run_id}/v1/
    analysis.json
    {analysis-name}.tar.gz

LabBridge owns artifact metadata and download proxy behavior. PACE owns delivery records and checksums. PRISM owns committed analysis results and archives. ASCEND stores only allowlisted references and never receives object-store credentials.

Local development defaults to file:// PACE and PRISM archives when storage is disabled. A production deployment must enable S3/MinIO for both services.

Local MinIO

cd /home/flare/sdlx
scripts/minio.sh start
scripts/minio.sh status

Rotate the ignored local secrets and reinitialize the scoped identities without printing credential values. The optional flag also restarts the three runtime units that consume object-store credentials:

scripts/rotate-local-minio-credentials.sh --restart-runtime

The initializer creates separate artifact and archive buckets, disables anonymous access, enables versioning, creates scoped LabBridge/PACE/PRISM users, a read-only backup user, and a distinct sdlx-record-purge identity. The purge identity has no write or broad list permission; it may only read and delete objects inside the owner prefixes used by governed experiment cleanup. The archive bucket is created with Object Lock support. infrastructure.minio.MINIO_ARCHIVE_RETENTION_DAYS: 0 intentionally leaves the duration unset for development; production must set the laboratory-approved duration.

The bundled single-volume MinIO service is a development fixture. Production uses managed S3 or a monitored, redundant MinIO deployment with independent backup and restore verification. MinIO API and Console ports are private infrastructure endpoints and are not public RIGOR routes.

The bundled server and client images are pinned by digest. An intentional upgrade must update both digests and rerun the immutable-write, scoped-identity, outage, recovery, and replay integration gate.

Keep infrastructure.minio.MINIO_DATA_DIR absolute. The rotation command rejects relative bind paths so invoking Compose from another working directory cannot silently select an empty object store.

Service configuration

MinIO server, bucket, retention, root identity, and scoped identity settings live under infrastructure.minio. LabBridge reads its artifact projection; PACE and PRISM read their object_store projections. Credential fields use secret_ref and are resolved only into ignored mode-0600 runtime files. The standard prefixes are pace/deliveries and prism/analyses; PRISM's read prefix includes pace/deliveries. When a module points at the bundled local MinIO endpoint, configuration validation requires its bucket and scoped identity to match the corresponding infrastructure.minio values; this fails before startup instead of surfacing later as an artifact-upload or cleanup 403. The same validation requires every bundled client to use the shared, scoped purge identity for owner-confirmed cleanup.

For AWS S3, use backend=s3, leave the endpoint empty, and prefer the workload identity credential chain over long-lived static keys. Secrets stay in the ignored central secret store or the production deployment secret provider, never in committed configuration.

Immutable commit protocol

PACE and PRISM use deterministic object keys and conditional writes. A retry of the same key and SHA-256 is successful; the same key with different bytes is a hard conflict. The owner commits its database record and outbox event only after the upload has been verified by object size and SHA-256 metadata or content.

PRISM downloads a PACE archive in the worker host, verifies the event checksum, and then mounts extracted input read-only into the no-network analysis sandbox. Object-store credentials are never mounted into the sandbox or exposed to an Agent tool.

Runtime LabBridge, PACE, and PRISM policies grant no delete action. Experiment cleanup invokes each record owner, which verifies its path and checksum before using the separate purge identity. Ordinary record reads, writes, execution, and analysis never receive delete permission. Retention or Object Lock may keep protected historical object versions even after the current logical object and owner metadata are removed.

Release verification

Run the real MinIO policy and conditional-write check from a Docker-enabled account:

cd /home/flare/sdlx
scripts/test-object-storage-minio.sh

Create a portable online backup of every current domain SQLite database and every version in both configured MinIO buckets, then restore it into a separate directory and verify all SQLite integrity checks and SHA-256 values:

scripts/backup-current-runtime.sh /independent-volume/sdlx-backup-YYYYMMDD
python3 scripts/runtime-backup.py verify /independent-volume/sdlx-backup-YYYYMMDD
python3 scripts/runtime-backup.py restore \
  /independent-volume/sdlx-backup-YYYYMMDD \
  /independent-restore-test/sdlx-backup-YYYYMMDD

The backup command requires a new or empty destination and uses SQLite's online backup API, so it never copies a live WAL database byte-for-byte. The restore command likewise requires a new or empty target and never overwrites live state. The local wrapper uses the configured read-only backup identity. It can list and read all versions in the two backup buckets but cannot write or delete objects. Runtime LabBridge, PACE, and PRISM processes continue to use their narrower identities. Production must place the resulting backup on independent, monitored storage.

• PACE emits s3:// Result Bundle and delivery archive references.
• PRISM reads the delivery from its allowed prefix and emits s3:// result and analysis archive references.
• Repeated delivery or analysis commits do not create different objects.
• A checksum mismatch blocks commitment.
• Each service is denied outside its declared prefix.
• The purge identity deletes checksum-verified owner objects but cannot write new objects or delete outside the governed prefixes.
• Storage outage recovery retries the same deterministic key.
• Backup restore preserves the database URI, object bytes, size, and SHA-256.
• Committed archive retention and purge behavior match laboratory policy.