git.stella-ops.org/devops/docker/repro-builders/BUILD_ENVIRONMENT.md

Reproducible Build Environment Requirements

Sprint: SPRINT_1227_0002_0001_LB_reproducible_builders Task: T12 — Document build environment requirements

---

Overview

This document describes the environment requirements for running reproducible distro package builds. The build system supports Alpine, Debian, and RHEL package ecosystems.

---

Hardware Requirements

Minimum Requirements

Resource	Minimum	Recommended
CPU	4 cores	8+ cores
RAM	8 GB	16+ GB
Disk	50 GB SSD	200+ GB NVMe
Network	10 Mbps	100+ Mbps

Storage Breakdown

Directory	Purpose	Estimated Size
/var/lib/docker	Docker images and containers	30 GB
/var/cache/stellaops/builds	Build cache	50 GB
/var/cache/stellaops/sources	Source package cache	20 GB
/var/cache/stellaops/artifacts	Output artifacts	50 GB

---

Software Requirements

Host System

Component	Version	Purpose
Docker	24.0+	Container runtime
Docker Compose	2.20+	Multi-container orchestration
.NET SDK	10.0	Worker service runtime
objdump	binutils 2.40+	Binary analysis
readelf	binutils 2.40+	ELF parsing

Container Images

The build system uses the following base images:

Builder	Base Image	Tag
Alpine	alpine	3.19, 3.18
Debian	debian	bookworm, bullseye
RHEL	almalinux	9, 8

---

Environment Variables

Required Variables

# Build configuration
export STELLAOPS_BUILD_CACHE=/var/cache/stellaops/builds
export STELLAOPS_SOURCE_CACHE=/var/cache/stellaops/sources
export STELLAOPS_ARTIFACT_DIR=/var/cache/stellaops/artifacts

# Reproducibility settings
export TZ=UTC
export LC_ALL=C.UTF-8
export SOURCE_DATE_EPOCH=$(date +%s)

# Docker settings
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

Optional Variables

# Parallel build settings
export STELLAOPS_MAX_CONCURRENT_BUILDS=2
export STELLAOPS_BUILD_TIMEOUT=1800  # 30 minutes

# Proxy settings (if behind corporate firewall)
export HTTP_PROXY=http://proxy:8080
export HTTPS_PROXY=http://proxy:8080
export NO_PROXY=localhost,127.0.0.1

---

Builder-Specific Requirements

Alpine Builder

# Required packages in builder image
apk add --no-cache \
    alpine-sdk \
    abuild \
    sudo \
    binutils \
    elfutils \
    build-base

Normalization requirements:

• SOURCE_DATE_EPOCH must be set
• Use abuild -r with reproducible flags
• Archive ordering: --sort=name

Debian Builder

# Required packages in builder image
apt-get install -y \
    build-essential \
    devscripts \
    dpkg-dev \
    fakeroot \
    binutils \
    elfutils \
    debhelper

Normalization requirements:

• Use dpkg-buildpackage -b with reproducible flags
• Set DEB_BUILD_OPTIONS=reproducible
• Apply dh_strip_nondeterminism post-build

RHEL Builder

# Required packages in builder image (AlmaLinux 9)
dnf install -y \
    mock \
    rpm-build \
    rpmdevtools \
    binutils \
    elfutils

Normalization requirements:

• Use mock with --enable-network=false
• Configure mock for deterministic builds
• Set %_buildhost stellaops.build

---

Compiler Flags for Reproducibility

C/C++ Flags

CFLAGS="-fno-record-gcc-switches -fdebug-prefix-map=$(pwd)=/build -grecord-gcc-switches=off"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,--build-id=sha1"

Additional Flags

# Disable date/time macros
-Wdate-time -Werror=date-time

# Normalize paths
-fmacro-prefix-map=$(pwd)=/build
-ffile-prefix-map=$(pwd)=/build

---

Archive Determinism

ar (Static Libraries)

# Use deterministic mode
ar --enable-deterministic-archives crs libfoo.a *.o

# Or set environment variable
export AR_FLAGS=--enable-deterministic-archives

tar (Package Archives)

# Deterministic tar creation
tar --sort=name \
    --mtime="@${SOURCE_DATE_EPOCH}" \
    --owner=0 \
    --group=0 \
    --numeric-owner \
    -cf archive.tar directory/

zip/gzip

# Use gzip -n to avoid timestamp
gzip -n file

# Use mtime for consistent timestamps
touch -d "@${SOURCE_DATE_EPOCH}" file

---

Network Requirements

Outbound Access Required

Destination	Port	Purpose
dl-cdn.alpinelinux.org	443	Alpine packages
deb.debian.org	443	Debian packages
vault.centos.org	443	CentOS/RHEL sources
mirror.almalinux.org	443	AlmaLinux packages
git.*.org	443	Upstream source repos

Air-Gapped Operation

For air-gapped environments:

1. Pre-download source packages
2. Configure local mirrors
3. Set STELLAOPS_OFFLINE_MODE=true
4. Use cached build artifacts

---

Security Considerations

Container Isolation

• Builders run in unprivileged containers
• No host network access
• Read-only source mounts
• Ephemeral containers (destroyed after build)

Signing Keys

• Build outputs are unsigned by default
• DSSE signing requires configured key material
• Keys stored in /etc/stellaops/keys/ or HSM

Build Verification

# Verify reproducibility
sha256sum build1/output/* > checksums1.txt
sha256sum build2/output/* > checksums2.txt
diff checksums1.txt checksums2.txt

---

Troubleshooting

Common Issues

Issue	Cause	Resolution
Build timestamp differs	SOURCE_DATE_EPOCH not set	Export variable before build
Path in debug info	Missing -fdebug-prefix-map	Add to CFLAGS
ar archive differs	Deterministic mode disabled	Use --enable-deterministic-archives
tar ordering differs	Random file order	Use --sort=name

Debugging Reproducibility

# Compare two builds byte-by-byte
diffoscope build1/output/libfoo.so build2/output/libfoo.so

# Check for timestamp differences
objdump -t binary | grep -i time

# Verify no random UUIDs
strings binary | grep -E '[0-9a-f]{8}-[0-9a-f]{4}'

---

Monitoring and Metrics

Key Metrics

Metric	Description	Target
build_reproducibility_rate	% of reproducible builds	> 95%
build_duration_seconds	Time to complete build	< 1800
fingerprint_extraction_rate	Functions per second	> 1000
build_cache_hit_rate	Cache effectiveness	> 80%

Health Checks

# Verify builder containers are ready
docker ps --filter "name=repro-builder"

# Check cache disk usage
df -h /var/cache/stellaops/

# Verify build queue
curl -s http://localhost:9090/metrics | grep stellaops_build

---

References

• Reproducible Builds
• Debian Reproducible Builds
• Alpine Reproducibility
• RPM Reproducibility