git.stella-ops.org/docs/ops/feedser-apple-operations.md

Feedser Apple Security Update Connector Operations

This runbook covers staging and production rollout for the Apple security updates connector (source:vndr-apple:*), including observability checks and fixture maintenance.

1. Prerequisites

• Network egress (or mirrored cache) for https://gdmf.apple.com/v2/pmv and the Apple Support domain (https://support.apple.com/).
• Optional: corporate proxy exclusions for the Apple hosts if outbound traffic is normally filtered.
• Updated configuration (environment variables or feedser.yaml) with an apple section. Example baseline:

feedser:
  sources:
    apple:
      softwareLookupUri: "https://gdmf.apple.com/v2/pmv"
      advisoryBaseUri: "https://support.apple.com/"
      localeSegment: "en-us"
      maxAdvisoriesPerFetch: 25
      initialBackfill: "120.00:00:00"
      modifiedTolerance: "02:00:00"
      failureBackoff: "00:05:00"

ℹ️ softwareLookupUri and advisoryBaseUri must stay absolute and aligned with the HTTP allow-list; Feedser automatically adds both hosts to the connector HttpClient.

2. Staging Smoke Test

1. Deploy the configuration and restart the Feedser workers to ensure the Apple connector options are bound.
2. Trigger a full connector cycle:
   • CLI: stella db jobs run source:vndr-apple:fetch --and-then source:vndr-apple:parse --and-then source:vndr-apple:map
   • REST: POST /jobs/run { "kind": "source:vndr-apple:fetch", "chain": ["source:vndr-apple:parse", "source:vndr-apple:map"] }
3. Validate metrics exported under meter StellaOps.Feedser.Source.Vndr.Apple:
   • apple.fetch.items (documents fetched)
   • apple.fetch.failures
   • apple.fetch.unchanged
   • apple.parse.failures
   • apple.map.affected.count (histogram of affected package counts)
4. Cross-check the shared HTTP counters:
   • feedser.source.http.requests_total{feedser_source="vndr-apple"} should increase for both index and detail phases.
   • feedser.source.http.failures_total{feedser_source="vndr-apple"} should remain flat (0) during a healthy run.
5. Inspect the info logs:
   • Apple software index fetch … processed=X newDocuments=Y
   • Apple advisory parse complete … aliases=… affected=…
   • Mapped Apple advisory … pendingMappings=0
6. Confirm MongoDB state:
   • raw_documents store contains the HT article HTML with metadata (apple.articleId, apple.postingDate).
   • dtos store has schemaVersion="apple.security.update.v1".
   • advisories collection includes keys HTxxxxxx with normalized SemVer rules.
   • source_states entry for apple shows a recent cursor.lastPosted.

3. Production Monitoring

• Dashboards – Add the following expressions to your Feedser Grafana board (OTLP/Prometheus naming assumed):
  • rate(apple_fetch_items_total[15m]) vs rate(feedser_source_http_requests_total{feedser_source="vndr-apple"}[15m])
  • rate(apple_fetch_failures_total[5m]) for error spikes (severity=warning at >0)
  • histogram_quantile(0.95, rate(apple_map_affected_count_bucket[1h])) to watch affected-package fan-out
  • increase(apple_parse_failures_total[6h]) to catch parser drift (alerts at >0)
• Alerts – Page if rate(apple_fetch_items_total[2h]) == 0 during business hours while other connectors are active. This often indicates lookup feed failures or misconfigured allow-lists.
• Logs – Surface warnings Apple document {DocumentId} missing GridFS payload or Apple parse failed—repeated hits imply storage issues or HTML regressions.
• Telemetry pipeline – StellaOps.Feedser.WebService now exports StellaOps.Feedser.Source.Vndr.Apple alongside existing Feedser meters; ensure your OTEL collector or Prometheus scraper includes it.

4. Fixture Maintenance

Regression fixtures live under src/StellaOps.Feedser.Source.Vndr.Apple.Tests/Apple/Fixtures. Refresh them whenever Apple reshapes the HT layout or when new platforms appear.

1. Run the helper script matching your platform:
   • Bash: ./scripts/update-apple-fixtures.sh
   • PowerShell: ./scripts/update-apple-fixtures.ps1
2. Each script exports UPDATE_APPLE_FIXTURES=1, updates the WSLENV passthrough, and touches .update-apple-fixtures so WSL+VS Code test runs observe the flag. The subsequent test execution fetches the live HT articles listed in AppleFixtureManager, sanitises the HTML, and rewrites the .expected.json DTO snapshots.
3. Review the diff for localisation or nav noise. Once satisfied, re-run the tests without the env var (dotnet test src/StellaOps.Feedser.Source.Vndr.Apple.Tests/StellaOps.Feedser.Source.Vndr.Apple.Tests.csproj) to verify determinism.
4. Commit fixture updates together with any parser/mapping changes that motivated them.

5. Known Issues & Follow-up Tasks

• Apple occasionally throttles anonymous requests after bursts. The connector backs off automatically, but persistent apple.fetch.failures spikes might require mirroring the HT content or scheduling wider fetch windows.
• Rapid Security Responses may appear before the general patch notes surface in the lookup JSON. When that happens, the fetch run will log detailFailures>0. Collect sample HTML and refresh fixtures to confirm parser coverage.
• Multi-locale content is still under regression sweep (src/StellaOps.Feedser.Source.Vndr.Apple/TASKS.md). Capture non-en-us snapshots once the fixture tooling stabilises.