Source preview guide · 0.1.0

Durable jobs and evidence publication

Understand lease ownership, result validation, commit-before-acknowledgement and finding revisions.

The database owns accepted work. A page can reload, an editor can disconnect and a worker can lose its lease without turning the browser into the record of what happened. Durable job state, scanner transport state and published scan evidence remain separate.

Identify a command before executing it

The job repository binds an idempotency key to the client and job kind. Repeating the same command returns the existing job; reusing that identity for a different command is a conflict. The job also has a stable external run identity, bounded payload/checkpoint documents and a revision.

Job states are queued, leased, running, cancelling, completed, failed and cancelled. A successful submission means accepted work. It does not mean that a scanner has run or that complete evidence exists.

claim_next_job requires PostgreSQL. It locks eligible rows without waiting on another worker’s selected row, and handles expired leases and recovery candidates. Job mutations check the job ID, worker identity, attempt, permitted state and an unexpired lease under a lock. Heartbeats renew that exact ownership. A former owner cannot complete a job simply because it once held the lease.

Retryable failure returns a job to the queue only within its attempt and operation-budget limits. A retry is not an unlimited extension of a preparation deadline. See the job repository and job data model.

Publish evidence before allowing cleanup

The scanner seals a normalized result and exposes its manifest and bounded finding pages. The worker does not treat their mere availability as durable publication.

1. Validate the manifest and bind the run, request digest, result digest, artifact-manifest digest and artifact provenance in the durable job checkpoint.
2. Begin one PostgreSQL transaction, lock the parent scan and refuse publication if findings or artifact indexes already exist for it. Fetch each bounded page and validate its identities, cursor, ordinals, byte counts, hashes, canonical values, ordering and unique fingerprints. Redact and insert that page’s admitted findings without committing.
3. Reconcile the complete stream and summary with the manifest. Admit the exact attested artifacts, update revisions and record scanner_persisted=true in that same transaction.
4. Establish that this transaction committed before sending ACK for the exact run, request and result binding.
5. Let the scanner remove the temporary result, then continue scan/gate finalization. A persisted result whose cleanup still needs reconciliation remains a different state from unpublished evidence.

This is an insert-once evidence boundary. A retry must recover its committed checkpoint, not replace existing findings and erase an operator’s review. The code refuses a conflicting publication with scan_evidence_conflict. The controlling implementation is the persistence transaction.

If commit rolls back, partial inserted findings do not become authoritative. If the outcome is uncertain, the worker reconciles durable state before cleanup. After scanner_persisted=true is durable, recovery is ACK-only and does not fetch and insert the collection again. A scanner 404 can count as already-cleaned only on this already-persisted recovery path. Acknowledgement recovery binds that exception.

Bound temporary transport separately from storage

The temporary scanner spool is private tmpfs. It is not a durable evidence store, downloadable artifact or backup. Unacknowledged results are not evicted merely to free a terminal-record slot, so capacity pressure can refuse new work. The time limit still applies. A scanner restart loses the process-local registry and cursor secret and removes recognized old spool files.

Page cursors bind the process, run, request, result and next position. They cannot be reused for another result. The run registry implements this temporary transport contract.

Retain useful evidence without publishing raw reports

Findings retain normalized scanner/rule identity, severity, source location, message, snippet, fingerprint and bounded metadata. They are redacted before scanner publication and again before database insertion. Redaction recognizes known patterns and sensitive structures; it cannot guarantee removal of every possible secret.

The artifact boundary admits bounded sanitized Syft CycloneDX JSON SBOMs. It validates the dedicated path with no-follow operations, binds raw-to-sanitized provenance and indexes the resulting content identity. Download checks size and digest against the index and copies verified bytes into bounded response storage. Raw scanner stdout, stderr, SARIF, JSONV2 and JSONL are not published artifacts.

Read scanner redaction and artifact admission and download. These controls reduce exposure; they do not make arbitrary hostile source safe to execute.

Continue the finding lifecycle after publication

New findings begin open, not suppressed, not gate-exempt, at revision one. The seven review dispositions are open, needs_review, gate_exempt, accepted_risk, ignore, false_positive and fixed. They are selectable review states, not a mandatory linear workflow.

open and needs_review participate in gate counts. gate_exempt and accepted_risk remain unsuppressed but do not count toward the gate. ignore, false_positive and fixed suppress the finding. The five exclusion dispositions require a nonblank reason. Each change checks the expected revision, updates review metadata and refreshes the scan’s finding summary and gate evaluation.

Marking fixed does not inspect an edited file. A new scan produces new evidence and starts its findings open. Matching scanner/fingerprint values across project scans support occurrence history; they do not inherit the older scan’s dispositions. Deleting a finding removes its row and is a separate action with revision and active-job checks.

Finding pages and details are bound to collection revisions so stale views cannot silently overwrite newer decisions. Advisor fields are separate from lifecycle fields. Advice does not automatically suppress a finding, accept risk or change the gate. See lifecycle mutation and the finding workflow.

Treat cancellation and recovery as observable outcomes

A cancellation request is persisted and can win a race with normal completion. Terminal outcomes remain terminal. Scanner-cache operations refuse cancellation during committing or activating, when publication has crossed its protected boundary. Cancelling cannot undo an activated cache generation or recall Advisor data already delivered.

A completed job, complete scan and passing gate are different results. Only complete scan evidence reaches normal policy evaluation; finding dispositions cannot repair missing coverage. The gate implementation retains that distinction.

Durable jobs support reconciliation after interruption. They do not establish full crash or disaster-recovery qualification. The current backup helper is experimental, non-atomic and disabled without an explicit disposable-test opt-in. Follow the recovery guide and preserve exact evidence identities when assessing a failure.