11597679ed
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
console-runner-image / build-runner-image (push) Has been cancelled
wine-csp-build / Build Wine CSP Image (push) Has been cancelled
wine-csp-build / Integration Tests (push) Has been cancelled
wine-csp-build / Security Scan (push) Has been cancelled
wine-csp-build / Generate SBOM (push) Has been cancelled
wine-csp-build / Publish Image (push) Has been cancelled
wine-csp-build / Air-Gap Bundle (push) Has been cancelled
wine-csp-build / Test Summary (push) Has been cancelled
- Added BerkeleyDbReader class to read and extract RPM header blobs from BerkeleyDB hash databases. - Implemented methods to detect BerkeleyDB format and extract values, including handling of page sizes and magic numbers. - Added tests for BerkeleyDbReader to ensure correct functionality and header extraction. feat: Add Yarn PnP data tests - Created YarnPnpDataTests to validate package resolution and data loading from Yarn PnP cache. - Implemented tests for resolved keys, package presence, and loading from cache structure. test: Add egg-info package fixtures for Python tests - Created egg-info package fixtures for testing Python analyzers. - Included PKG-INFO, entry_points.txt, and installed-files.txt for comprehensive coverage. test: Enhance RPM database reader tests - Added tests for RpmDatabaseReader to validate fallback to legacy packages when SQLite is missing. - Implemented helper methods to create legacy package files and RPM headers for testing. test: Implement dual signing tests - Added DualSignTests to validate secondary signature addition when configured. - Created stub implementations for crypto providers and key resolvers to facilitate testing. chore: Update CI script for Playwright Chromium installation - Modified ci-console-exports.sh to ensure deterministic Chromium binary installation for console exports tests. - Added checks for Windows compatibility and environment variable setups for Playwright browsers.
56 lines
1.9 KiB
Bash
56 lines
1.9 KiB
Bash
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# Export Concelier linksets/advisory_chunks from Postgres to a tar.zst bundle.
|
|
# Usage:
|
|
# PGURI=postgres://user:pass@host:5432/db \
|
|
# ./scripts/concelier/export-linksets-tarball.sh out/linksets/linksets-stage-backfill.tar.zst
|
|
#
|
|
# Optional env:
|
|
# PGSCHEMA=public # schema that owns linksets/advisory_chunks
|
|
# LINKSETS_TABLE=linksets # table name for linksets
|
|
# CHUNKS_TABLE=advisory_chunks # table name for advisory chunks
|
|
# TMPDIR=/tmp/export-linksets # working directory (defaults to mktemp)
|
|
|
|
TARGET="${1:-}"
|
|
if [[ -z "${TARGET}" ]]; then
|
|
echo "Usage: PGURI=... $0 out/linksets/linksets-stage-backfill.tar.zst" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [[ -z "${PGURI:-}" ]]; then
|
|
echo "PGURI environment variable is required (postgres://...)" >&2
|
|
exit 1
|
|
fi
|
|
|
|
PGSCHEMA="${PGSCHEMA:-public}"
|
|
LINKSETS_TABLE="${LINKSETS_TABLE:-linksets}"
|
|
CHUNKS_TABLE="${CHUNKS_TABLE:-advisory_chunks}"
|
|
WORKDIR="${TMPDIR:-$(mktemp -d)}"
|
|
|
|
mkdir -p "${WORKDIR}"
|
|
OUTDIR="$(dirname "${TARGET}")"
|
|
mkdir -p "${OUTDIR}"
|
|
|
|
echo "==> Exporting linksets from ${PGSCHEMA}.${LINKSETS_TABLE}"
|
|
psql "${PGURI}" -c "\copy (select row_to_json(t) from ${PGSCHEMA}.${LINKSETS_TABLE} t) to '${WORKDIR}/linksets.ndjson'"
|
|
|
|
echo "==> Exporting advisory_chunks from ${PGSCHEMA}.${CHUNKS_TABLE}"
|
|
psql "${PGURI}" -c "\copy (select row_to_json(t) from ${PGSCHEMA}.${CHUNKS_TABLE} t) to '${WORKDIR}/advisory_chunks.ndjson'"
|
|
|
|
LINKSETS_COUNT="$(wc -l < "${WORKDIR}/linksets.ndjson")"
|
|
CHUNKS_COUNT="$(wc -l < "${WORKDIR}/advisory_chunks.ndjson")"
|
|
|
|
echo "==> Writing manifest.json"
|
|
jq -n --argjson linksets "${LINKSETS_COUNT}" --argjson advisory_chunks "${CHUNKS_COUNT}" \
|
|
'{linksets: $linksets, advisory_chunks: $advisory_chunks}' \
|
|
> "${WORKDIR}/manifest.json"
|
|
|
|
echo "==> Building tarball ${TARGET}"
|
|
tar -I "zstd -19" -cf "${TARGET}" -C "${WORKDIR}" linksets.ndjson advisory_chunks.ndjson manifest.json
|
|
|
|
echo "==> SHA-256"
|
|
sha256sum "${TARGET}"
|
|
|
|
echo "Done. Workdir: ${WORKDIR}"
|