11597679ed
Some checks failed
AOC Guard CI / aoc-guard (push) Has been cancelled
AOC Guard CI / aoc-verify (push) Has been cancelled
Concelier Attestation Tests / attestation-tests (push) Has been cancelled
Docs CI / lint-and-preview (push) Has been cancelled
Policy Lint & Smoke / policy-lint (push) Has been cancelled
Scanner Analyzers / Discover Analyzers (push) Has been cancelled
Scanner Analyzers / Build Analyzers (push) Has been cancelled
Scanner Analyzers / Test Language Analyzers (push) Has been cancelled
Scanner Analyzers / Validate Test Fixtures (push) Has been cancelled
Scanner Analyzers / Verify Deterministic Output (push) Has been cancelled
console-runner-image / build-runner-image (push) Has been cancelled
wine-csp-build / Build Wine CSP Image (push) Has been cancelled
wine-csp-build / Integration Tests (push) Has been cancelled
wine-csp-build / Security Scan (push) Has been cancelled
wine-csp-build / Generate SBOM (push) Has been cancelled
wine-csp-build / Publish Image (push) Has been cancelled
wine-csp-build / Air-Gap Bundle (push) Has been cancelled
wine-csp-build / Test Summary (push) Has been cancelled
- Added BerkeleyDbReader class to read and extract RPM header blobs from BerkeleyDB hash databases. - Implemented methods to detect BerkeleyDB format and extract values, including handling of page sizes and magic numbers. - Added tests for BerkeleyDbReader to ensure correct functionality and header extraction. feat: Add Yarn PnP data tests - Created YarnPnpDataTests to validate package resolution and data loading from Yarn PnP cache. - Implemented tests for resolved keys, package presence, and loading from cache structure. test: Add egg-info package fixtures for Python tests - Created egg-info package fixtures for testing Python analyzers. - Included PKG-INFO, entry_points.txt, and installed-files.txt for comprehensive coverage. test: Enhance RPM database reader tests - Added tests for RpmDatabaseReader to validate fallback to legacy packages when SQLite is missing. - Implemented helper methods to create legacy package files and RPM headers for testing. test: Implement dual signing tests - Added DualSignTests to validate secondary signature addition when configured. - Created stub implementations for crypto providers and key resolvers to facilitate testing. chore: Update CI script for Playwright Chromium installation - Modified ci-console-exports.sh to ensure deterministic Chromium binary installation for console exports tests. - Added checks for Windows compatibility and environment variable setups for Playwright browsers.
75 lines
2.3 KiB
Bash
75 lines
2.3 KiB
Bash
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# Deterministic dataset builder for STORE-AOC-19-005-DEV.
|
|
# Generates linksets-stage-backfill.tar.zst from repo seed data.
|
|
# Usage:
|
|
# ./scripts/concelier/build-store-aoc-19-005-dataset.sh [output_tarball]
|
|
# Default output: out/linksets/linksets-stage-backfill.tar.zst
|
|
|
|
command -v tar >/dev/null || { echo "tar is required" >&2; exit 1; }
|
|
command -v sha256sum >/dev/null || { echo "sha256sum is required" >&2; exit 1; }
|
|
|
|
TAR_COMPRESS=()
|
|
if command -v zstd >/dev/null 2>&1; then
|
|
TAR_COMPRESS=(--zstd)
|
|
else
|
|
echo "zstd not found; building uncompressed tarball (extension kept for compatibility)" >&2
|
|
fi
|
|
|
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
SEED_DIR="${ROOT_DIR}/seed-data/concelier/store-aoc-19-005"
|
|
OUT_DIR="${ROOT_DIR}/out/linksets"
|
|
OUT_PATH="${1:-${OUT_DIR}/linksets-stage-backfill.tar.zst}"
|
|
GEN_TIME="2025-12-07T00:00:00Z"
|
|
|
|
for seed in linksets.ndjson advisory_chunks.ndjson; do
|
|
if [[ ! -f "${SEED_DIR}/${seed}" ]]; then
|
|
echo "Missing seed file: ${SEED_DIR}/${seed}" >&2
|
|
exit 1
|
|
fi
|
|
done
|
|
|
|
WORKDIR="$(mktemp -d)"
|
|
cleanup() { rm -rf "${WORKDIR}"; }
|
|
trap cleanup EXIT
|
|
|
|
cp "${SEED_DIR}/linksets.ndjson" "${WORKDIR}/linksets.ndjson"
|
|
cp "${SEED_DIR}/advisory_chunks.ndjson" "${WORKDIR}/advisory_chunks.ndjson"
|
|
|
|
linksets_sha=$(sha256sum "${WORKDIR}/linksets.ndjson" | awk '{print $1}')
|
|
advisory_sha=$(sha256sum "${WORKDIR}/advisory_chunks.ndjson" | awk '{print $1}')
|
|
linksets_count=$(wc -l < "${WORKDIR}/linksets.ndjson" | tr -d '[:space:]')
|
|
advisory_count=$(wc -l < "${WORKDIR}/advisory_chunks.ndjson" | tr -d '[:space:]')
|
|
|
|
cat >"${WORKDIR}/manifest.json" <<EOF
|
|
{
|
|
"datasetId": "store-aoc-19-005-dev",
|
|
"generatedAt": "${GEN_TIME}",
|
|
"source": "seed-data/concelier/store-aoc-19-005",
|
|
"records": {
|
|
"linksets": ${linksets_count},
|
|
"advisory_chunks": ${advisory_count}
|
|
},
|
|
"sha256": {
|
|
"linksets.ndjson": "${linksets_sha}",
|
|
"advisory_chunks.ndjson": "${advisory_sha}"
|
|
}
|
|
}
|
|
EOF
|
|
|
|
mkdir -p "${OUT_DIR}"
|
|
|
|
tar "${TAR_COMPRESS[@]}" \
|
|
--format=ustar \
|
|
--mtime='1970-01-01 00:00:00Z' \
|
|
--owner=0 --group=0 --numeric-owner \
|
|
-cf "${OUT_PATH}" \
|
|
-C "${WORKDIR}" \
|
|
linksets.ndjson advisory_chunks.ndjson manifest.json
|
|
|
|
sha256sum "${OUT_PATH}" > "${OUT_PATH}.sha256"
|
|
|
|
echo "Wrote ${OUT_PATH}"
|
|
cat "${OUT_PATH}.sha256"
|