From 09d21d977c6fc064f35dcef78628bafda4154492 Mon Sep 17 00:00:00 2001 From: master Date: Thu, 23 Oct 2025 07:57:16 +0300 Subject: [PATCH] feat: Update analyzer fixtures and metadata for improved license handling and provenance tracking - Added license expressions and provenance fields to expected JSON outputs for .NET and Rust analyzers. - Introduced new .nuspec files for StellaOps.Runtime.SelfContained and StellaOps.Toolkit packages, including license information. - Created LICENSE.txt files for both toolkit packages with clear licensing terms. - Updated expected JSON for signed and simple analyzers to include license information and provenance. - Enhanced the SPRINTS_LANG_IMPLEMENTATION_PLAN.md with detailed progress and future sprint outlines, ensuring clarity on deliverables and acceptance metrics. --- EXECPLAN.md | 6 +- .../Internal/DotNetDependencyCollector.cs | 2291 +++++++++-------- .../Internal/DotNetFileCaches.cs | 332 +++ .../TASKS.md | 2 +- .../DotNet/DotNetLanguageAnalyzerTests.cs | 198 +- .../lang/dotnet/selfcontained/expected.json | 39 +- .../stellaops.runtime.selfcontained.nuspec | 11 + .../stellaops.toolkit/1.2.3/LICENSE.txt | 6 + .../1.2.3/stellaops.toolkit.nuspec | 11 + .../Fixtures/lang/dotnet/signed/expected.json | 27 +- .../9.0.0/microsoft.extensions.logging.nuspec | 11 + .../Fixtures/lang/dotnet/simple/expected.json | 17 +- .../9.0.0/microsoft.extensions.logging.nuspec | 11 + .../stellaops.toolkit/1.2.3/LICENSE.txt | 7 + .../1.2.3/stellaops.toolkit.nuspec | 11 + .../Fixtures/lang/rust/simple/expected.json | 32 +- .../SPRINTS_LANG_IMPLEMENTATION_PLAN.md | 188 +- 17 files changed, 1857 insertions(+), 1343 deletions(-) create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetFileCaches.cs create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.runtime.selfcontained/2.1.0/stellaops.runtime.selfcontained.nuspec create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/LICENSE.txt create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/LICENSE.txt create mode 100644 src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec diff --git a/EXECPLAN.md b/EXECPLAN.md index 4e9c1a1c..c044cbb6 100644 --- a/EXECPLAN.md +++ b/EXECPLAN.md @@ -118,7 +118,7 @@ Generated from SPRINTS.md and module TASKS.md files on 2025-10-19. Waves cluster - Team Notify Worker Guild: read EXECPLAN.md Wave 4 and SPRINTS.md rows for `src/StellaOps.Notify.Worker/TASKS.md`. Focus on NOTIFY-WORKER-15-204 (TODO). Confirm prerequisites (internal: NOTIFY-WORKER-15-203 (Wave 3)) before starting and report status in module TASKS.md. - Team Policy Guild, Scanner WebService Guild: read EXECPLAN.md Wave 4 and SPRINTS.md rows for `src/StellaOps.Policy/TASKS.md`. Focus on POLICY-RUNTIME-17-201 (TODO). Confirm prerequisites (internal: ZASTAVA-OBS-17-005 (Wave 3)) before starting and report status in module TASKS.md. - Team Scheduler Worker Guild: read EXECPLAN.md Wave 4 and SPRINTS.md rows for `src/StellaOps.Scheduler.Worker/TASKS.md`. Focus on SCHED-WORKER-16-204 (TODO). Confirm prerequisites (internal: SCHED-WORKER-16-203 (Wave 3)) before starting and report status in module TASKS.md. -- Team TBD: read EXECPLAN.md Wave 4 and SPRINTS.md rows for `src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md`. Focus on SCANNER-ANALYZERS-LANG-10-307D (TODO), SCANNER-ANALYZERS-LANG-10-307G (TODO), SCANNER-ANALYZERS-LANG-10-307P (TODO), SCANNER-ANALYZERS-LANG-10-307R (TODO). Confirm prerequisites (internal: SCANNER-ANALYZERS-LANG-10-303C (Wave 3), SCANNER-ANALYZERS-LANG-10-304C (Wave 3), SCANNER-ANALYZERS-LANG-10-305C (Wave 3), SCANNER-ANALYZERS-LANG-10-306C (Wave 3)) before starting and report status in module TASKS.md. +- Team TBD: read EXECPLAN.md Wave 4 and SPRINTS.md rows for `src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Python/TASKS.md`, `src/StellaOps.Scanner.Analyzers.Lang.Rust/TASKS.md`. Focus on SCANNER-ANALYZERS-LANG-10-307D (DONE 2025-10-22), SCANNER-ANALYZERS-LANG-10-307G (TODO), SCANNER-ANALYZERS-LANG-10-307P (TODO), SCANNER-ANALYZERS-LANG-10-307R (TODO). Confirm prerequisites (internal: SCANNER-ANALYZERS-LANG-10-303C (Wave 3), SCANNER-ANALYZERS-LANG-10-304C (Wave 3), SCANNER-ANALYZERS-LANG-10-305C (Wave 3), SCANNER-ANALYZERS-LANG-10-306C (Wave 3)) before starting and report status in module TASKS.md. ### Wave 5 - Team Excititor Connectors – Stella: read EXECPLAN.md Wave 5 and SPRINTS.md rows for `src/StellaOps.Excititor.Connectors.StellaOpsMirror/TASKS.md`. Focus on EXCITITOR-CONN-STELLA-07-003 (TODO). Confirm prerequisites (internal: EXCITITOR-CONN-STELLA-07-002 (Wave 4)) before starting and report status in module TASKS.md. @@ -927,9 +927,9 @@ Generated from SPRINTS.md and module TASKS.md files on 2025-10-19. Waves cluster - **Sprint 10** · Backlog - Team: TBD - Path: `src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md` - 1. [TODO] SCANNER-ANALYZERS-LANG-10-307D — Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. + 1. [DONE 2025-10-22] SCANNER-ANALYZERS-LANG-10-307D — Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. • Prereqs: SCANNER-ANALYZERS-LANG-10-305C (Wave 3) - • Current: TODO + • Current: DONE 2025-10-22 - Path: `src/StellaOps.Scanner.Analyzers.Lang.Go/TASKS.md` 1. [TODO] SCANNER-ANALYZERS-LANG-10-307G — Wire shared helpers (license mapping, usage flags) and ensure concurrency-safe buffer reuse. • Prereqs: SCANNER-ANALYZERS-LANG-10-304C (Wave 3) diff --git a/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetDependencyCollector.cs b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetDependencyCollector.cs index 9ae12a65..7a780a55 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetDependencyCollector.cs +++ b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetDependencyCollector.cs @@ -1,1124 +1,1257 @@ -using System.Diagnostics; -using System.Globalization; -using System.Linq; -using System.Reflection; -using System.Security.Cryptography; -using System.Text.Json; - -namespace StellaOps.Scanner.Analyzers.Lang.DotNet.Internal; - -internal static class DotNetDependencyCollector -{ - private static readonly EnumerationOptions Enumeration = new() - { - RecurseSubdirectories = true, - IgnoreInaccessible = true, - AttributesToSkip = FileAttributes.Device | FileAttributes.ReparsePoint - }; - - public static ValueTask> CollectAsync(LanguageAnalyzerContext context, CancellationToken cancellationToken) - { - ArgumentNullException.ThrowIfNull(context); - - var depsFiles = Directory - .EnumerateFiles(context.RootPath, "*.deps.json", Enumeration) - .OrderBy(static path => path, StringComparer.Ordinal) - .ToArray(); - - if (depsFiles.Length == 0) - { - return ValueTask.FromResult>(Array.Empty()); - } - - var aggregator = new DotNetPackageAggregator(context); - - foreach (var depsPath in depsFiles) - { - cancellationToken.ThrowIfCancellationRequested(); - - try - { - var relativeDepsPath = NormalizeRelative(context.GetRelativePath(depsPath)); - var depsFile = DotNetDepsFile.Load(depsPath, relativeDepsPath, cancellationToken); - if (depsFile is null) - { - continue; - } - - DotNetRuntimeConfig? runtimeConfig = null; - var runtimeConfigPath = Path.ChangeExtension(depsPath, ".runtimeconfig.json"); - if (!string.IsNullOrEmpty(runtimeConfigPath) && File.Exists(runtimeConfigPath)) - { - var relativeRuntimePath = NormalizeRelative(context.GetRelativePath(runtimeConfigPath)); - runtimeConfig = DotNetRuntimeConfig.Load(runtimeConfigPath, relativeRuntimePath, cancellationToken); - } - - aggregator.Add(depsFile, runtimeConfig); - } - catch (IOException) - { - continue; - } - catch (JsonException) - { - continue; - } - catch (UnauthorizedAccessException) - { - continue; - } - } - - var packages = aggregator.Build(cancellationToken); - return ValueTask.FromResult>(packages); - } - - private static string NormalizeRelative(string path) - { - if (string.IsNullOrWhiteSpace(path) || path == ".") - { - return "."; - } - - var normalized = path.Replace('\\', '/'); - return string.IsNullOrWhiteSpace(normalized) ? "." : normalized; - } -} - -internal sealed class DotNetPackageAggregator -{ - private readonly LanguageAnalyzerContext _context; - private readonly IDotNetAuthenticodeInspector? _authenticodeInspector; - private readonly Dictionary _packages = new(StringComparer.Ordinal); - - public DotNetPackageAggregator(LanguageAnalyzerContext context) - { - _context = context ?? throw new ArgumentNullException(nameof(context)); - if (context.TryGetService(out var inspector)) - { - _authenticodeInspector = inspector; - } - } - - public void Add(DotNetDepsFile depsFile, DotNetRuntimeConfig? runtimeConfig) - { - ArgumentNullException.ThrowIfNull(depsFile); - - foreach (var library in depsFile.Libraries.Values) - { - if (!library.IsPackage) - { - continue; - } - - var normalizedId = DotNetPackageBuilder.NormalizeId(library.Id); - var key = DotNetPackageBuilder.BuildKey(normalizedId, library.Version); - - if (!_packages.TryGetValue(key, out var builder)) - { - builder = new DotNetPackageBuilder(_context, _authenticodeInspector, library.Id, normalizedId, library.Version); - _packages[key] = builder; - } - - builder.AddLibrary(library, depsFile.RelativePath, runtimeConfig); - } - } - - public IReadOnlyList Build(CancellationToken cancellationToken) - { - if (_packages.Count == 0) - { - return Array.Empty(); - } - - var items = new List(_packages.Count); - foreach (var builder in _packages.Values) - { - cancellationToken.ThrowIfCancellationRequested(); - items.Add(builder.Build(cancellationToken)); - } - - items.Sort(static (left, right) => string.CompareOrdinal(left.ComponentKey, right.ComponentKey)); - return items; - } -} - -internal sealed class DotNetPackageBuilder -{ - private readonly LanguageAnalyzerContext _context; - private readonly IDotNetAuthenticodeInspector? _authenticodeInspector; - - private readonly string _originalId; - private readonly string _normalizedId; - private readonly string _version; - - private bool? _serviceable; - - private readonly SortedSet _sha512 = new(StringComparer.Ordinal); - private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); - private readonly SortedSet _hashPaths = new(StringComparer.Ordinal); - private readonly SortedSet _depsPaths = new(StringComparer.Ordinal); - private readonly SortedSet _targetFrameworks = new(StringComparer.Ordinal); - private readonly SortedSet _runtimeIdentifiers = new(StringComparer.Ordinal); - private readonly SortedSet _dependencies = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _runtimeConfigPaths = new(StringComparer.Ordinal); - private readonly SortedSet _runtimeConfigTfms = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _runtimeConfigFrameworks = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _runtimeConfigGraph = new(StringComparer.OrdinalIgnoreCase); - - private readonly Dictionary _assemblies = new(StringComparer.OrdinalIgnoreCase); - private readonly Dictionary _nativeAssets = new(StringComparer.OrdinalIgnoreCase); - private readonly HashSet _evidence = new(new LanguageComponentEvidenceComparer()); - private bool _usedByEntrypoint; - - public DotNetPackageBuilder(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, string originalId, string normalizedId, string version) - { - _context = context ?? throw new ArgumentNullException(nameof(context)); - _authenticodeInspector = authenticodeInspector; - _originalId = string.IsNullOrWhiteSpace(originalId) ? normalizedId : originalId.Trim(); - _normalizedId = normalizedId; - _version = version ?? string.Empty; - } - - public static string BuildKey(string normalizedId, string version) - => $"{normalizedId}::{version}"; - - public static string NormalizeId(string id) - => string.IsNullOrWhiteSpace(id) ? string.Empty : id.Trim().ToLowerInvariant(); - - public void AddLibrary(DotNetLibrary library, string relativeDepsPath, DotNetRuntimeConfig? runtimeConfig) - { - ArgumentNullException.ThrowIfNull(library); - - if (library.Serviceable is bool serviceable) - { - _serviceable = _serviceable.HasValue - ? _serviceable.Value || serviceable - : serviceable; - } - - AddIfPresent(_sha512, library.Sha512); - AddIfPresent(_packagePaths, library.PackagePath); - AddIfPresent(_hashPaths, library.HashPath); - AddIfPresent(_depsPaths, NormalizeRelativePath(relativeDepsPath)); - - foreach (var dependency in library.Dependencies) - { - AddIfPresent(_dependencies, dependency, normalizeLower: true); - } - - foreach (var tfm in library.TargetFrameworks) - { - AddIfPresent(_targetFrameworks, tfm); - } - - foreach (var rid in library.RuntimeIdentifiers) - { - AddIfPresent(_runtimeIdentifiers, rid); - } - - AddRuntimeAssets(library); - - _evidence.Add(new LanguageComponentEvidence( - LanguageEvidenceKind.File, - "deps.json", - NormalizeRelativePath(relativeDepsPath), - library.Key, - Sha256: null)); - - if (runtimeConfig is not null) - { - AddRuntimeConfig(runtimeConfig); - } - } - - public DotNetPackage Build(CancellationToken cancellationToken) - { - var metadata = new List>(32) - { - new("package.id", _originalId), - new("package.id.normalized", _normalizedId), - new("package.version", _version) - }; - - if (_serviceable.HasValue) - { - metadata.Add(new KeyValuePair("package.serviceable", _serviceable.Value ? "true" : "false")); - } - - AddIndexed(metadata, "package.sha512", _sha512); - AddIndexed(metadata, "package.path", _packagePaths); - AddIndexed(metadata, "package.hashPath", _hashPaths); - AddIndexed(metadata, "deps.path", _depsPaths); - AddIndexed(metadata, "deps.dependency", _dependencies); - AddIndexed(metadata, "deps.tfm", _targetFrameworks); - AddIndexed(metadata, "deps.rid", _runtimeIdentifiers); - AddIndexed(metadata, "runtimeconfig.path", _runtimeConfigPaths); - AddIndexed(metadata, "runtimeconfig.tfm", _runtimeConfigTfms); - AddIndexed(metadata, "runtimeconfig.framework", _runtimeConfigFrameworks); - AddIndexed(metadata, "runtimeconfig.graph", _runtimeConfigGraph); - +using System.Diagnostics; +using System.Globalization; +using System.Linq; +using System.Reflection; +using System.Security.Cryptography; +using System.Text.Json; + +namespace StellaOps.Scanner.Analyzers.Lang.DotNet.Internal; + +internal static class DotNetDependencyCollector +{ + private static readonly EnumerationOptions Enumeration = new() + { + RecurseSubdirectories = true, + IgnoreInaccessible = true, + AttributesToSkip = FileAttributes.Device | FileAttributes.ReparsePoint + }; + + public static ValueTask> CollectAsync(LanguageAnalyzerContext context, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(context); + + var depsFiles = Directory + .EnumerateFiles(context.RootPath, "*.deps.json", Enumeration) + .OrderBy(static path => path, StringComparer.Ordinal) + .ToArray(); + + if (depsFiles.Length == 0) + { + return ValueTask.FromResult>(Array.Empty()); + } + + var aggregator = new DotNetPackageAggregator(context); + + foreach (var depsPath in depsFiles) + { + cancellationToken.ThrowIfCancellationRequested(); + + try + { + var relativeDepsPath = NormalizeRelative(context.GetRelativePath(depsPath)); + var depsFile = DotNetDepsFile.Load(depsPath, relativeDepsPath, cancellationToken); + if (depsFile is null) + { + continue; + } + + DotNetRuntimeConfig? runtimeConfig = null; + var runtimeConfigPath = Path.ChangeExtension(depsPath, ".runtimeconfig.json"); + if (!string.IsNullOrEmpty(runtimeConfigPath) && File.Exists(runtimeConfigPath)) + { + var relativeRuntimePath = NormalizeRelative(context.GetRelativePath(runtimeConfigPath)); + runtimeConfig = DotNetRuntimeConfig.Load(runtimeConfigPath, relativeRuntimePath, cancellationToken); + } + + aggregator.Add(depsFile, runtimeConfig); + } + catch (IOException) + { + continue; + } + catch (JsonException) + { + continue; + } + catch (UnauthorizedAccessException) + { + continue; + } + } + + var packages = aggregator.Build(cancellationToken); + return ValueTask.FromResult>(packages); + } + + private static string NormalizeRelative(string path) + { + if (string.IsNullOrWhiteSpace(path) || path == ".") + { + return "."; + } + + var normalized = path.Replace('\\', '/'); + return string.IsNullOrWhiteSpace(normalized) ? "." : normalized; + } +} + +internal sealed class DotNetPackageAggregator +{ + private readonly LanguageAnalyzerContext _context; + private readonly IDotNetAuthenticodeInspector? _authenticodeInspector; + private readonly Dictionary _packages = new(StringComparer.Ordinal); + + public DotNetPackageAggregator(LanguageAnalyzerContext context) + { + _context = context ?? throw new ArgumentNullException(nameof(context)); + if (context.TryGetService(out var inspector)) + { + _authenticodeInspector = inspector; + } + } + + public void Add(DotNetDepsFile depsFile, DotNetRuntimeConfig? runtimeConfig) + { + ArgumentNullException.ThrowIfNull(depsFile); + + foreach (var library in depsFile.Libraries.Values) + { + if (!library.IsPackage) + { + continue; + } + + var normalizedId = DotNetPackageBuilder.NormalizeId(library.Id); + var key = DotNetPackageBuilder.BuildKey(normalizedId, library.Version); + + if (!_packages.TryGetValue(key, out var builder)) + { + builder = new DotNetPackageBuilder(_context, _authenticodeInspector, library.Id, normalizedId, library.Version); + _packages[key] = builder; + } + + builder.AddLibrary(library, depsFile.RelativePath, runtimeConfig); + } + } + + public IReadOnlyList Build(CancellationToken cancellationToken) + { + if (_packages.Count == 0) + { + return Array.Empty(); + } + + var items = new List(_packages.Count); + foreach (var builder in _packages.Values) + { + cancellationToken.ThrowIfCancellationRequested(); + items.Add(builder.Build(cancellationToken)); + } + + items.Sort(static (left, right) => string.CompareOrdinal(left.ComponentKey, right.ComponentKey)); + return items; + } +} + +internal sealed class DotNetPackageBuilder +{ + private readonly LanguageAnalyzerContext _context; + private readonly IDotNetAuthenticodeInspector? _authenticodeInspector; + + private readonly string _originalId; + private readonly string _normalizedId; + private readonly string _version; + + private bool? _serviceable; + + private readonly SortedSet _sha512 = new(StringComparer.Ordinal); + private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); + private readonly SortedSet _hashPaths = new(StringComparer.Ordinal); + private readonly SortedSet _depsPaths = new(StringComparer.Ordinal); + private readonly SortedSet _targetFrameworks = new(StringComparer.Ordinal); + private readonly SortedSet _runtimeIdentifiers = new(StringComparer.Ordinal); + private readonly SortedSet _dependencies = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _runtimeConfigPaths = new(StringComparer.Ordinal); + private readonly SortedSet _runtimeConfigTfms = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _runtimeConfigFrameworks = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _runtimeConfigGraph = new(StringComparer.OrdinalIgnoreCase); + + private readonly Dictionary _assemblies = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary _nativeAssets = new(StringComparer.OrdinalIgnoreCase); + private readonly HashSet _evidence = new(new LanguageComponentEvidenceComparer()); + private bool _usedByEntrypoint; + + public DotNetPackageBuilder(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, string originalId, string normalizedId, string version) + { + _context = context ?? throw new ArgumentNullException(nameof(context)); + _authenticodeInspector = authenticodeInspector; + _originalId = string.IsNullOrWhiteSpace(originalId) ? normalizedId : originalId.Trim(); + _normalizedId = normalizedId; + _version = version ?? string.Empty; + } + + public static string BuildKey(string normalizedId, string version) + => $"{normalizedId}::{version}"; + + public static string NormalizeId(string id) + => string.IsNullOrWhiteSpace(id) ? string.Empty : id.Trim().ToLowerInvariant(); + + public void AddLibrary(DotNetLibrary library, string relativeDepsPath, DotNetRuntimeConfig? runtimeConfig) + { + ArgumentNullException.ThrowIfNull(library); + + if (library.Serviceable is bool serviceable) + { + _serviceable = _serviceable.HasValue + ? _serviceable.Value || serviceable + : serviceable; + } + + AddIfPresent(_sha512, library.Sha512); + AddIfPresent(_packagePaths, library.PackagePath); + AddIfPresent(_hashPaths, library.HashPath); + AddIfPresent(_depsPaths, NormalizeRelativePath(relativeDepsPath)); + + foreach (var dependency in library.Dependencies) + { + AddIfPresent(_dependencies, dependency, normalizeLower: true); + } + + foreach (var tfm in library.TargetFrameworks) + { + AddIfPresent(_targetFrameworks, tfm); + } + + foreach (var rid in library.RuntimeIdentifiers) + { + AddIfPresent(_runtimeIdentifiers, rid); + } + + AddRuntimeAssets(library); + + _evidence.Add(new LanguageComponentEvidence( + LanguageEvidenceKind.File, + "deps.json", + NormalizeRelativePath(relativeDepsPath), + library.Key, + Sha256: null)); + + if (runtimeConfig is not null) + { + AddRuntimeConfig(runtimeConfig); + } + } + + public DotNetPackage Build(CancellationToken cancellationToken) + { + var metadata = new List>(32) + { + new("package.id", _originalId), + new("package.id.normalized", _normalizedId), + new("package.version", _version) + }; + + if (_serviceable.HasValue) + { + metadata.Add(new KeyValuePair("package.serviceable", _serviceable.Value ? "true" : "false")); + } + + AddIndexed(metadata, "package.sha512", _sha512); + AddIndexed(metadata, "package.path", _packagePaths); + AddIndexed(metadata, "package.hashPath", _hashPaths); + AddIndexed(metadata, "deps.path", _depsPaths); + AddIndexed(metadata, "deps.dependency", _dependencies); + AddIndexed(metadata, "deps.tfm", _targetFrameworks); + AddIndexed(metadata, "deps.rid", _runtimeIdentifiers); + AddIndexed(metadata, "runtimeconfig.path", _runtimeConfigPaths); + AddIndexed(metadata, "runtimeconfig.tfm", _runtimeConfigTfms); + AddIndexed(metadata, "runtimeconfig.framework", _runtimeConfigFrameworks); + AddIndexed(metadata, "runtimeconfig.graph", _runtimeConfigGraph); + var assemblies = CollectAssemblyMetadata(cancellationToken); AddAssemblyMetadata(metadata, assemblies); var nativeAssets = CollectNativeMetadata(cancellationToken); AddNativeMetadata(metadata, nativeAssets); + AppendLicenseMetadata(metadata, cancellationToken); + AddProvenanceMetadata(metadata); + metadata.Sort(static (left, right) => string.CompareOrdinal(left.Key, right.Key)); var evidence = _evidence .OrderBy(static item => item.Source, StringComparer.Ordinal) - .ThenBy(static item => item.Locator, StringComparer.Ordinal) - .ThenBy(static item => item.Value, StringComparer.Ordinal) - .ToArray(); - - return new DotNetPackage( - name: _originalId, - normalizedId: _normalizedId, - version: _version, - metadata: metadata, - evidence: evidence, - usedByEntrypoint: _usedByEntrypoint); - } - - private IReadOnlyList CollectAssemblyMetadata(CancellationToken cancellationToken) - { - if (_assemblies.Count == 0) - { - return Array.Empty(); - } - - var results = new List(_assemblies.Count); - foreach (var aggregate in _assemblies.Values.OrderBy(static aggregate => aggregate.AssetRelativePath, StringComparer.Ordinal)) - { - cancellationToken.ThrowIfCancellationRequested(); - results.Add(aggregate.Build(_context, _authenticodeInspector, cancellationToken)); - } - - return results; - } - - private IReadOnlyList CollectNativeMetadata(CancellationToken cancellationToken) - { - if (_nativeAssets.Count == 0) - { - return Array.Empty(); - } - - var results = new List(_nativeAssets.Count); - foreach (var aggregate in _nativeAssets.Values.OrderBy(static aggregate => aggregate.AssetRelativePath, StringComparer.Ordinal)) - { - cancellationToken.ThrowIfCancellationRequested(); - results.Add(aggregate.Build(_context, cancellationToken)); - } - - return results; - } - - private void AddAssemblyMetadata(ICollection> metadata, IReadOnlyList assemblies) - { - if (assemblies.Count == 0) - { - return; - } - - for (var index = 0; index < assemblies.Count; index++) - { - var record = assemblies[index]; - var prefix = $"assembly[{index}]"; - - if (record.UsedByEntrypoint) - { - _usedByEntrypoint = true; - } - - AddIfPresent(metadata, $"{prefix}.assetPath", record.AssetPath); - AddIfPresent(metadata, $"{prefix}.path", record.RelativePath); - AddIndexed(metadata, $"{prefix}.tfm", record.TargetFrameworks); - AddIndexed(metadata, $"{prefix}.rid", record.RuntimeIdentifiers); - AddIfPresent(metadata, $"{prefix}.version", record.AssemblyVersion); - AddIfPresent(metadata, $"{prefix}.fileVersion", record.FileVersion); - AddIfPresent(metadata, $"{prefix}.publicKeyToken", record.PublicKeyToken); - AddIfPresent(metadata, $"{prefix}.strongName", record.StrongName); - AddIfPresent(metadata, $"{prefix}.company", record.CompanyName); - AddIfPresent(metadata, $"{prefix}.product", record.ProductName); - AddIfPresent(metadata, $"{prefix}.productVersion", record.ProductVersion); - AddIfPresent(metadata, $"{prefix}.fileDescription", record.FileDescription); - AddIfPresent(metadata, $"{prefix}.sha256", record.Sha256); - - if (record.Authenticode is { } authenticode) - { - AddIfPresent(metadata, $"{prefix}.authenticode.subject", authenticode.Subject); - AddIfPresent(metadata, $"{prefix}.authenticode.issuer", authenticode.Issuer); - AddIfPresent(metadata, $"{prefix}.authenticode.notBefore", FormatTimestamp(authenticode.NotBefore)); - AddIfPresent(metadata, $"{prefix}.authenticode.notAfter", FormatTimestamp(authenticode.NotAfter)); - AddIfPresent(metadata, $"{prefix}.authenticode.thumbprint", authenticode.Thumbprint); - AddIfPresent(metadata, $"{prefix}.authenticode.serialNumber", authenticode.SerialNumber); - } - - if (!string.IsNullOrEmpty(record.RelativePath)) - { - _evidence.Add(new LanguageComponentEvidence( - LanguageEvidenceKind.File, - Source: "assembly", - Locator: record.RelativePath!, - Value: record.AssetPath, - Sha256: record.Sha256)); - } - } - } - + .ThenBy(static item => item.Locator, StringComparer.Ordinal) + .ThenBy(static item => item.Value, StringComparer.Ordinal) + .ToArray(); + + return new DotNetPackage( + name: _originalId, + normalizedId: _normalizedId, + version: _version, + metadata: metadata, + evidence: evidence, + usedByEntrypoint: _usedByEntrypoint); + } + + private IReadOnlyList CollectAssemblyMetadata(CancellationToken cancellationToken) + { + if (_assemblies.Count == 0) + { + return Array.Empty(); + } + + var results = new List(_assemblies.Count); + foreach (var aggregate in _assemblies.Values.OrderBy(static aggregate => aggregate.AssetRelativePath, StringComparer.Ordinal)) + { + cancellationToken.ThrowIfCancellationRequested(); + results.Add(aggregate.Build(_context, _authenticodeInspector, cancellationToken)); + } + + return results; + } + + private IReadOnlyList CollectNativeMetadata(CancellationToken cancellationToken) + { + if (_nativeAssets.Count == 0) + { + return Array.Empty(); + } + + var results = new List(_nativeAssets.Count); + foreach (var aggregate in _nativeAssets.Values.OrderBy(static aggregate => aggregate.AssetRelativePath, StringComparer.Ordinal)) + { + cancellationToken.ThrowIfCancellationRequested(); + results.Add(aggregate.Build(_context, cancellationToken)); + } + + return results; + } + + private void AddAssemblyMetadata(ICollection> metadata, IReadOnlyList assemblies) + { + if (assemblies.Count == 0) + { + return; + } + + for (var index = 0; index < assemblies.Count; index++) + { + var record = assemblies[index]; + var prefix = $"assembly[{index}]"; + + if (record.UsedByEntrypoint) + { + _usedByEntrypoint = true; + } + + AddIfPresent(metadata, $"{prefix}.assetPath", record.AssetPath); + AddIfPresent(metadata, $"{prefix}.path", record.RelativePath); + AddIndexed(metadata, $"{prefix}.tfm", record.TargetFrameworks); + AddIndexed(metadata, $"{prefix}.rid", record.RuntimeIdentifiers); + AddIfPresent(metadata, $"{prefix}.version", record.AssemblyVersion); + AddIfPresent(metadata, $"{prefix}.fileVersion", record.FileVersion); + AddIfPresent(metadata, $"{prefix}.publicKeyToken", record.PublicKeyToken); + AddIfPresent(metadata, $"{prefix}.strongName", record.StrongName); + AddIfPresent(metadata, $"{prefix}.company", record.CompanyName); + AddIfPresent(metadata, $"{prefix}.product", record.ProductName); + AddIfPresent(metadata, $"{prefix}.productVersion", record.ProductVersion); + AddIfPresent(metadata, $"{prefix}.fileDescription", record.FileDescription); + AddIfPresent(metadata, $"{prefix}.sha256", record.Sha256); + + if (record.Authenticode is { } authenticode) + { + AddIfPresent(metadata, $"{prefix}.authenticode.subject", authenticode.Subject); + AddIfPresent(metadata, $"{prefix}.authenticode.issuer", authenticode.Issuer); + AddIfPresent(metadata, $"{prefix}.authenticode.notBefore", FormatTimestamp(authenticode.NotBefore)); + AddIfPresent(metadata, $"{prefix}.authenticode.notAfter", FormatTimestamp(authenticode.NotAfter)); + AddIfPresent(metadata, $"{prefix}.authenticode.thumbprint", authenticode.Thumbprint); + AddIfPresent(metadata, $"{prefix}.authenticode.serialNumber", authenticode.SerialNumber); + } + + if (!string.IsNullOrEmpty(record.RelativePath)) + { + _evidence.Add(new LanguageComponentEvidence( + LanguageEvidenceKind.File, + Source: "assembly", + Locator: record.RelativePath!, + Value: record.AssetPath, + Sha256: record.Sha256)); + } + } + } + private void AddNativeMetadata(ICollection> metadata, IReadOnlyList nativeAssets) { if (nativeAssets.Count == 0) { return; - } - - for (var index = 0; index < nativeAssets.Count; index++) - { - var record = nativeAssets[index]; - var prefix = $"native[{index}]"; - - if (record.UsedByEntrypoint) - { - _usedByEntrypoint = true; - } - - AddIfPresent(metadata, $"{prefix}.assetPath", record.AssetPath); - AddIfPresent(metadata, $"{prefix}.path", record.RelativePath); - AddIndexed(metadata, $"{prefix}.tfm", record.TargetFrameworks); - AddIndexed(metadata, $"{prefix}.rid", record.RuntimeIdentifiers); - AddIfPresent(metadata, $"{prefix}.sha256", record.Sha256); - - if (!string.IsNullOrEmpty(record.RelativePath)) - { - _evidence.Add(new LanguageComponentEvidence( - LanguageEvidenceKind.File, - Source: "native", - Locator: record.RelativePath!, - Value: record.AssetPath, - Sha256: record.Sha256)); - } + } + + for (var index = 0; index < nativeAssets.Count; index++) + { + var record = nativeAssets[index]; + var prefix = $"native[{index}]"; + + if (record.UsedByEntrypoint) + { + _usedByEntrypoint = true; + } + + AddIfPresent(metadata, $"{prefix}.assetPath", record.AssetPath); + AddIfPresent(metadata, $"{prefix}.path", record.RelativePath); + AddIndexed(metadata, $"{prefix}.tfm", record.TargetFrameworks); + AddIndexed(metadata, $"{prefix}.rid", record.RuntimeIdentifiers); + AddIfPresent(metadata, $"{prefix}.sha256", record.Sha256); + + if (!string.IsNullOrEmpty(record.RelativePath)) + { + _evidence.Add(new LanguageComponentEvidence( + LanguageEvidenceKind.File, + Source: "native", + Locator: record.RelativePath!, + Value: record.AssetPath, + Sha256: record.Sha256)); + } } } - private void AddRuntimeAssets(DotNetLibrary library) - { - foreach (var asset in library.RuntimeAssets) - { - switch (asset.Kind) - { - case DotNetLibraryAssetKind.Runtime: - AddRuntimeAssemblyAsset(asset, library.PackagePath); - break; - case DotNetLibraryAssetKind.Native: - AddNativeAsset(asset, library.PackagePath); - break; - } - } - } - - private void AddRuntimeAssemblyAsset(DotNetLibraryAsset asset, string? packagePath) - { - var key = NormalizePath(asset.RelativePath); - if (string.IsNullOrEmpty(key)) - { - return; - } - - if (!_assemblies.TryGetValue(key, out var aggregate)) - { - aggregate = new AssemblyMetadataAggregate(key); - _assemblies[key] = aggregate; - } - - aggregate.AddManifestData(asset, packagePath); - } - - private void AddNativeAsset(DotNetLibraryAsset asset, string? packagePath) - { - var key = NormalizePath(asset.RelativePath); - if (string.IsNullOrEmpty(key)) - { - return; - } - - if (!_nativeAssets.TryGetValue(key, out var aggregate)) - { - aggregate = new NativeAssetAggregate(key); - _nativeAssets[key] = aggregate; - } - - aggregate.AddManifestData(asset, packagePath); - } - - private void AddRuntimeConfig(DotNetRuntimeConfig runtimeConfig) - { - AddIfPresent(_runtimeConfigPaths, runtimeConfig.RelativePath); - - foreach (var tfm in runtimeConfig.Tfms) - { - AddIfPresent(_runtimeConfigTfms, tfm); - } - - foreach (var framework in runtimeConfig.Frameworks) - { - AddIfPresent(_runtimeConfigFrameworks, framework); - } - - foreach (var entry in runtimeConfig.RuntimeGraph) - { - var value = BuildRuntimeGraphValue(entry.Rid, entry.Fallbacks); - AddIfPresent(_runtimeConfigGraph, value); - } - - _evidence.Add(new LanguageComponentEvidence( - LanguageEvidenceKind.File, - "runtimeconfig.json", - runtimeConfig.RelativePath, - Value: null, - Sha256: null)); - } - - private static void AddIfPresent(ICollection> metadata, string key, string? value) + private void AppendLicenseMetadata(List> metadata, CancellationToken cancellationToken) { if (metadata is null) { throw new ArgumentNullException(nameof(metadata)); } - if (string.IsNullOrWhiteSpace(key) || string.IsNullOrWhiteSpace(value)) + var expressions = new SortedSet(StringComparer.OrdinalIgnoreCase); + var urls = new SortedSet(StringComparer.OrdinalIgnoreCase); + var licenseFiles = new SortedDictionary(StringComparer.Ordinal); + var processedNuspecs = new HashSet(OperatingSystem.IsWindows() ? StringComparer.OrdinalIgnoreCase : StringComparer.Ordinal); + + foreach (var packagePath in _packagePaths) { - return; - } + cancellationToken.ThrowIfCancellationRequested(); - metadata.Add(new KeyValuePair(key, value)); - } - - private static void AddIfPresent(ISet set, string? value, bool normalizeLower = false) - { - if (set is null) - { - throw new ArgumentNullException(nameof(set)); - } - - if (string.IsNullOrWhiteSpace(value)) - { - return; - } - - var normalized = value.Trim(); - if (normalizeLower) - { - normalized = normalized.ToLowerInvariant(); - } - - set.Add(normalized); - } - - private static void AddIndexed(ICollection> metadata, string prefix, IEnumerable values) - { - if (metadata is null) - { - throw new ArgumentNullException(nameof(metadata)); - } - - if (values is null) - { - return; - } - - var index = 0; - foreach (var value in values) - { - if (string.IsNullOrWhiteSpace(value)) - { - continue; - } - - metadata.Add(new KeyValuePair($"{prefix}[{index++}]", value)); - } - } - - private static string NormalizeRelativePath(string path) - { - if (string.IsNullOrWhiteSpace(path) || path == ".") - { - return "."; - } - - return path.Replace('\\', '/'); - } - - private static string NormalizePath(string? path) - { - if (string.IsNullOrWhiteSpace(path)) - { - return string.Empty; - } - - var normalized = path.Replace('\\', '/').Trim(); - return string.IsNullOrEmpty(normalized) ? string.Empty : normalized; - } - - private static string ConvertToPlatformPath(string path) - => string.IsNullOrEmpty(path) ? "." : path.Replace('/', Path.DirectorySeparatorChar); - - private static string CombineRelative(string basePath, string relativePath) - { - var left = NormalizePath(basePath); - var right = NormalizePath(relativePath); - - if (string.IsNullOrEmpty(left)) - { - return right; - } - - if (string.IsNullOrEmpty(right)) - { - return left; - } - - return NormalizePath($"{left}/{right}"); - } - - private static string? ComputeSha256(string path) - { - using var stream = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read); - using var sha = SHA256.Create(); - var hash = sha.ComputeHash(stream); - return Convert.ToHexString(hash).ToLowerInvariant(); - } - - private static AssemblyName? TryGetAssemblyName(string path) - { - try - { - return AssemblyName.GetAssemblyName(path); - } - catch (FileNotFoundException) - { - return null; - } - catch (BadImageFormatException) - { - return null; - } - catch (FileLoadException) - { - return null; - } - } - - private static FileVersionInfo? TryGetFileVersionInfo(string path) - { - try - { - return FileVersionInfo.GetVersionInfo(path); - } - catch (FileNotFoundException) - { - return null; - } - catch (IOException) - { - return null; - } - catch (UnauthorizedAccessException) - { - return null; - } - } - - private static string? FormatPublicKeyToken(byte[]? token) - { - if (token is null || token.Length == 0) - { - return null; - } - - return Convert.ToHexString(token).ToLowerInvariant(); - } - - private static string? BuildStrongName(AssemblyName assemblyName, string? publicKeyToken) - { - if (assemblyName is null || string.IsNullOrWhiteSpace(assemblyName.Name)) - { - return null; - } - - var version = assemblyName.Version?.ToString() ?? "0.0.0.0"; - var culture = string.IsNullOrWhiteSpace(assemblyName.CultureName) ? "neutral" : assemblyName.CultureName; - var token = string.IsNullOrWhiteSpace(publicKeyToken) ? "null" : publicKeyToken; - return $"{assemblyName.Name}, Version={version}, Culture={culture}, PublicKeyToken={token}"; - } - - private static string? NormalizeMetadataValue(string? value) - { - if (string.IsNullOrWhiteSpace(value)) - { - return null; - } - - return value.Trim(); - } - - private static string? FormatTimestamp(DateTimeOffset? value) - { - if (value is null) - { - return null; - } - - return value.Value.UtcDateTime.ToString("yyyy-MM-ddTHH:mm:ss.fffZ", CultureInfo.InvariantCulture); - } - - private static IEnumerable EnumeratePackageBases(string packagePath) - { - if (string.IsNullOrWhiteSpace(packagePath)) - { - yield break; - } - - var normalized = NormalizePath(packagePath); - if (string.IsNullOrEmpty(normalized)) - { - yield break; - } - - yield return normalized; - yield return NormalizePath($".nuget/packages/{normalized}"); - yield return NormalizePath($"packages/{normalized}"); - yield return NormalizePath($"usr/share/dotnet/packs/{normalized}"); - } - - private static string BuildRuntimeGraphValue(string rid, IReadOnlyList fallbacks) - { - if (string.IsNullOrWhiteSpace(rid)) - { - return string.Empty; - } - - if (fallbacks.Count == 0) - { - return rid.Trim(); - } - - var ordered = fallbacks - .Where(static fallback => !string.IsNullOrWhiteSpace(fallback)) - .Select(static fallback => fallback.Trim()) - .Distinct(StringComparer.OrdinalIgnoreCase) - .OrderBy(static fallback => fallback, StringComparer.OrdinalIgnoreCase) - .ToArray(); - - return ordered.Length == 0 - ? rid.Trim() - : $"{rid.Trim()}=>{string.Join(';', ordered)}"; - } - - private sealed class AssemblyMetadataAggregate - { - private readonly string _assetRelativePath; - private readonly SortedSet _tfms = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _runtimeIdentifiers = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); - private string? _assemblyVersion; - private string? _fileVersion; - - public AssemblyMetadataAggregate(string assetRelativePath) - { - _assetRelativePath = NormalizePath(assetRelativePath); - } - - public string AssetRelativePath => _assetRelativePath; - - public void AddManifestData(DotNetLibraryAsset asset, string? packagePath) - { - if (!string.IsNullOrWhiteSpace(asset.TargetFramework)) - { - _tfms.Add(asset.TargetFramework); - } - - if (!string.IsNullOrWhiteSpace(asset.RuntimeIdentifier)) - { - _runtimeIdentifiers.Add(asset.RuntimeIdentifier); - } - - if (!string.IsNullOrWhiteSpace(asset.AssemblyVersion) && string.IsNullOrEmpty(_assemblyVersion)) - { - _assemblyVersion = asset.AssemblyVersion; - } - - if (!string.IsNullOrWhiteSpace(asset.FileVersion) && string.IsNullOrEmpty(_fileVersion)) - { - _fileVersion = asset.FileVersion; - } - - if (!string.IsNullOrWhiteSpace(packagePath)) - { - var normalized = NormalizePath(packagePath); - if (!string.IsNullOrEmpty(normalized)) - { - _packagePaths.Add(normalized); - } - } - } - - public AssemblyMetadataResult Build(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, CancellationToken cancellationToken) - { - var fileMetadata = ResolveFileMetadata(context, authenticodeInspector, cancellationToken); - - var assemblyName = fileMetadata?.AssemblyName; - var versionInfo = fileMetadata?.FileVersionInfo; - - var assemblyVersion = assemblyName?.Version?.ToString() ?? _assemblyVersion; - var fileVersion = !string.IsNullOrWhiteSpace(versionInfo?.FileVersion) ? versionInfo?.FileVersion : _fileVersion; - var usedByEntrypoint = fileMetadata?.UsedByEntrypoint ?? false; - - string? publicKeyToken = null; - string? strongName = null; - if (assemblyName is not null) - { - publicKeyToken = FormatPublicKeyToken(assemblyName.GetPublicKeyToken()); - strongName = BuildStrongName(assemblyName, publicKeyToken); - } - - return new AssemblyMetadataResult( - AssetPath: _assetRelativePath, - RelativePath: fileMetadata?.RelativePath, - TargetFrameworks: _tfms.ToArray(), - RuntimeIdentifiers: _runtimeIdentifiers.ToArray(), - AssemblyVersion: assemblyVersion, - FileVersion: fileVersion, - PublicKeyToken: publicKeyToken, - StrongName: strongName, - CompanyName: NormalizeMetadataValue(versionInfo?.CompanyName), - ProductName: NormalizeMetadataValue(versionInfo?.ProductName), - ProductVersion: NormalizeMetadataValue(versionInfo?.ProductVersion), - FileDescription: NormalizeMetadataValue(versionInfo?.FileDescription), - Sha256: fileMetadata?.Sha256, - Authenticode: fileMetadata?.Authenticode, - UsedByEntrypoint: usedByEntrypoint); - } - - private AssemblyFileMetadata? ResolveFileMetadata(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, CancellationToken cancellationToken) - { - var candidates = BuildCandidateRelativePaths(); - - foreach (var candidate in candidates) + foreach (var basePath in EnumeratePackageBases(packagePath)) { cancellationToken.ThrowIfCancellationRequested(); - var absolutePath = context.ResolvePath(ConvertToPlatformPath(candidate)); - if (!File.Exists(absolutePath)) + var platformRelative = ConvertToPlatformPath(basePath); + var absoluteDirectory = SafeResolveDirectory(platformRelative); + if (absoluteDirectory is null || !Directory.Exists(absoluteDirectory)) { continue; } + foreach (var nuspecPath in Directory.EnumerateFiles(absoluteDirectory, "*.nuspec", SearchOption.TopDirectoryOnly)) + { + cancellationToken.ThrowIfCancellationRequested(); + + var normalizedNuspec = NormalizeFullPath(nuspecPath); + if (!processedNuspecs.Add(normalizedNuspec)) + { + continue; + } + + if (!DotNetLicenseCache.TryGetLicenseInfo(nuspecPath, out var licenseInfo) || licenseInfo is null) + { + continue; + } + + foreach (var expression in licenseInfo.Expressions) + { + if (!string.IsNullOrWhiteSpace(expression)) + { + expressions.Add(expression.Trim()); + } + } + + foreach (var url in licenseInfo.Urls) + { + if (!string.IsNullOrWhiteSpace(url)) + { + urls.Add(url.Trim()); + } + } + + foreach (var fileRelative in licenseInfo.Files) + { + cancellationToken.ThrowIfCancellationRequested(); + + if (string.IsNullOrWhiteSpace(fileRelative)) + { + continue; + } + + var normalizedRelative = NormalizePath(fileRelative); + if (string.IsNullOrEmpty(normalizedRelative)) + { + continue; + } + + var platformRelativeFile = ConvertToPlatformPath(normalizedRelative); + var absoluteFile = SafeCombine(absoluteDirectory, platformRelativeFile); + if (absoluteFile is null || !File.Exists(absoluteFile)) + { + continue; + } + + if (!absoluteFile.StartsWith(absoluteDirectory, OperatingSystem.IsWindows() ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal)) + { + continue; + } + + var relativeToRoot = NormalizePath(_context.GetRelativePath(absoluteFile)); + if (string.IsNullOrEmpty(relativeToRoot)) + { + continue; + } + + if (!licenseFiles.ContainsKey(relativeToRoot)) + { + DotNetFileMetadataCache.TryGetSha256(absoluteFile, out var shaValue); + licenseFiles[relativeToRoot] = shaValue; + } + } + } + } + } + + if (expressions.Count > 0) + { + AddIndexed(metadata, "license.expression", expressions); + } + + if (urls.Count > 0) + { + AddIndexed(metadata, "license.url", urls); + } + + if (licenseFiles.Count > 0) + { + var index = 0; + foreach (var pair in licenseFiles) + { + metadata.Add(new KeyValuePair($"license.file[{index}]", pair.Key)); + + if (!string.IsNullOrEmpty(pair.Value)) + { + metadata.Add(new KeyValuePair($"license.file.sha256[{index}]", pair.Value)); + } + + _evidence.Add(new LanguageComponentEvidence( + LanguageEvidenceKind.File, + "license", + pair.Key, + Value: null, + Sha256: pair.Value)); + + index++; + } + } + } + + private static void AddProvenanceMetadata(ICollection> metadata) + { + if (!metadata.Any(static pair => string.Equals(pair.Key, "provenance", StringComparison.Ordinal))) + { + metadata.Add(new KeyValuePair("provenance", "manifest")); + } + } + + private string? SafeResolveDirectory(string relativePath) + { + try + { + return _context.ResolvePath(relativePath); + } + catch + { + return null; + } + } + + private static string? SafeCombine(string baseDirectory, string relativePath) + { + try + { + return Path.GetFullPath(Path.Combine(baseDirectory, relativePath)); + } + catch + { + return null; + } + } + + private static string NormalizeFullPath(string path) + { + var fullPath = Path.GetFullPath(path); + if (OperatingSystem.IsWindows()) + { + fullPath = fullPath.ToLowerInvariant(); + } + + return fullPath.TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar); + } + + private void AddRuntimeAssets(DotNetLibrary library) + { + foreach (var asset in library.RuntimeAssets) + { + switch (asset.Kind) + { + case DotNetLibraryAssetKind.Runtime: + AddRuntimeAssemblyAsset(asset, library.PackagePath); + break; + case DotNetLibraryAssetKind.Native: + AddNativeAsset(asset, library.PackagePath); + break; + } + } + } + + private void AddRuntimeAssemblyAsset(DotNetLibraryAsset asset, string? packagePath) + { + var key = NormalizePath(asset.RelativePath); + if (string.IsNullOrEmpty(key)) + { + return; + } + + if (!_assemblies.TryGetValue(key, out var aggregate)) + { + aggregate = new AssemblyMetadataAggregate(key); + _assemblies[key] = aggregate; + } + + aggregate.AddManifestData(asset, packagePath); + } + + private void AddNativeAsset(DotNetLibraryAsset asset, string? packagePath) + { + var key = NormalizePath(asset.RelativePath); + if (string.IsNullOrEmpty(key)) + { + return; + } + + if (!_nativeAssets.TryGetValue(key, out var aggregate)) + { + aggregate = new NativeAssetAggregate(key); + _nativeAssets[key] = aggregate; + } + + aggregate.AddManifestData(asset, packagePath); + } + + private void AddRuntimeConfig(DotNetRuntimeConfig runtimeConfig) + { + AddIfPresent(_runtimeConfigPaths, runtimeConfig.RelativePath); + + foreach (var tfm in runtimeConfig.Tfms) + { + AddIfPresent(_runtimeConfigTfms, tfm); + } + + foreach (var framework in runtimeConfig.Frameworks) + { + AddIfPresent(_runtimeConfigFrameworks, framework); + } + + foreach (var entry in runtimeConfig.RuntimeGraph) + { + var value = BuildRuntimeGraphValue(entry.Rid, entry.Fallbacks); + AddIfPresent(_runtimeConfigGraph, value); + } + + _evidence.Add(new LanguageComponentEvidence( + LanguageEvidenceKind.File, + "runtimeconfig.json", + runtimeConfig.RelativePath, + Value: null, + Sha256: null)); + } + + private static void AddIfPresent(ICollection> metadata, string key, string? value) + { + if (metadata is null) + { + throw new ArgumentNullException(nameof(metadata)); + } + + if (string.IsNullOrWhiteSpace(key) || string.IsNullOrWhiteSpace(value)) + { + return; + } + + metadata.Add(new KeyValuePair(key, value)); + } + + private static void AddIfPresent(ISet set, string? value, bool normalizeLower = false) + { + if (set is null) + { + throw new ArgumentNullException(nameof(set)); + } + + if (string.IsNullOrWhiteSpace(value)) + { + return; + } + + var normalized = value.Trim(); + if (normalizeLower) + { + normalized = normalized.ToLowerInvariant(); + } + + set.Add(normalized); + } + + private static void AddIndexed(ICollection> metadata, string prefix, IEnumerable values) + { + if (metadata is null) + { + throw new ArgumentNullException(nameof(metadata)); + } + + if (values is null) + { + return; + } + + var index = 0; + foreach (var value in values) + { + if (string.IsNullOrWhiteSpace(value)) + { + continue; + } + + metadata.Add(new KeyValuePair($"{prefix}[{index++}]", value)); + } + } + + private static string NormalizeRelativePath(string path) + { + if (string.IsNullOrWhiteSpace(path) || path == ".") + { + return "."; + } + + return path.Replace('\\', '/'); + } + + private static string NormalizePath(string? path) + { + if (string.IsNullOrWhiteSpace(path)) + { + return string.Empty; + } + + var normalized = path.Replace('\\', '/').Trim(); + return string.IsNullOrEmpty(normalized) ? string.Empty : normalized; + } + + internal static string ConvertToPlatformPath(string path) + => string.IsNullOrEmpty(path) ? "." : path.Replace('/', Path.DirectorySeparatorChar); + + private static string CombineRelative(string basePath, string relativePath) + { + var left = NormalizePath(basePath); + var right = NormalizePath(relativePath); + + if (string.IsNullOrEmpty(left)) + { + return right; + } + + if (string.IsNullOrEmpty(right)) + { + return left; + } + + return NormalizePath($"{left}/{right}"); + } + + private static string? FormatPublicKeyToken(byte[]? token) + { + if (token is null || token.Length == 0) + { + return null; + } + + return Convert.ToHexString(token).ToLowerInvariant(); + } + + private static string? BuildStrongName(AssemblyName assemblyName, string? publicKeyToken) + { + if (assemblyName is null || string.IsNullOrWhiteSpace(assemblyName.Name)) + { + return null; + } + + var version = assemblyName.Version?.ToString() ?? "0.0.0.0"; + var culture = string.IsNullOrWhiteSpace(assemblyName.CultureName) ? "neutral" : assemblyName.CultureName; + var token = string.IsNullOrWhiteSpace(publicKeyToken) ? "null" : publicKeyToken; + return $"{assemblyName.Name}, Version={version}, Culture={culture}, PublicKeyToken={token}"; + } + + private static string? NormalizeMetadataValue(string? value) + { + if (string.IsNullOrWhiteSpace(value)) + { + return null; + } + + return value.Trim(); + } + + private static string? FormatTimestamp(DateTimeOffset? value) + { + if (value is null) + { + return null; + } + + return value.Value.UtcDateTime.ToString("yyyy-MM-ddTHH:mm:ss.fffZ", CultureInfo.InvariantCulture); + } + + private static IEnumerable EnumeratePackageBases(string packagePath) + { + if (string.IsNullOrWhiteSpace(packagePath)) + { + yield break; + } + + var normalized = NormalizePath(packagePath); + if (string.IsNullOrEmpty(normalized)) + { + yield break; + } + + yield return normalized; + yield return NormalizePath($".nuget/packages/{normalized}"); + yield return NormalizePath($"packages/{normalized}"); + yield return NormalizePath($"usr/share/dotnet/packs/{normalized}"); + } + + private static string BuildRuntimeGraphValue(string rid, IReadOnlyList fallbacks) + { + if (string.IsNullOrWhiteSpace(rid)) + { + return string.Empty; + } + + if (fallbacks.Count == 0) + { + return rid.Trim(); + } + + var ordered = fallbacks + .Where(static fallback => !string.IsNullOrWhiteSpace(fallback)) + .Select(static fallback => fallback.Trim()) + .Distinct(StringComparer.OrdinalIgnoreCase) + .OrderBy(static fallback => fallback, StringComparer.OrdinalIgnoreCase) + .ToArray(); + + return ordered.Length == 0 + ? rid.Trim() + : $"{rid.Trim()}=>{string.Join(';', ordered)}"; + } + + private sealed class AssemblyMetadataAggregate + { + private readonly string _assetRelativePath; + private readonly SortedSet _tfms = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _runtimeIdentifiers = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); + private string? _assemblyVersion; + private string? _fileVersion; + + public AssemblyMetadataAggregate(string assetRelativePath) + { + _assetRelativePath = NormalizePath(assetRelativePath); + } + + public string AssetRelativePath => _assetRelativePath; + + public void AddManifestData(DotNetLibraryAsset asset, string? packagePath) + { + if (!string.IsNullOrWhiteSpace(asset.TargetFramework)) + { + _tfms.Add(asset.TargetFramework); + } + + if (!string.IsNullOrWhiteSpace(asset.RuntimeIdentifier)) + { + _runtimeIdentifiers.Add(asset.RuntimeIdentifier); + } + + if (!string.IsNullOrWhiteSpace(asset.AssemblyVersion) && string.IsNullOrEmpty(_assemblyVersion)) + { + _assemblyVersion = asset.AssemblyVersion; + } + + if (!string.IsNullOrWhiteSpace(asset.FileVersion) && string.IsNullOrEmpty(_fileVersion)) + { + _fileVersion = asset.FileVersion; + } + + if (!string.IsNullOrWhiteSpace(packagePath)) + { + var normalized = NormalizePath(packagePath); + if (!string.IsNullOrEmpty(normalized)) + { + _packagePaths.Add(normalized); + } + } + } + + public AssemblyMetadataResult Build(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, CancellationToken cancellationToken) + { + var fileMetadata = ResolveFileMetadata(context, authenticodeInspector, cancellationToken); + + var assemblyName = fileMetadata?.AssemblyName; + var versionInfo = fileMetadata?.FileVersionInfo; + + var assemblyVersion = assemblyName?.Version?.ToString() ?? _assemblyVersion; + var fileVersion = !string.IsNullOrWhiteSpace(versionInfo?.FileVersion) ? versionInfo?.FileVersion : _fileVersion; + var usedByEntrypoint = fileMetadata?.UsedByEntrypoint ?? false; + + string? publicKeyToken = null; + string? strongName = null; + if (assemblyName is not null) + { + publicKeyToken = FormatPublicKeyToken(assemblyName.GetPublicKeyToken()); + strongName = BuildStrongName(assemblyName, publicKeyToken); + } + + return new AssemblyMetadataResult( + AssetPath: _assetRelativePath, + RelativePath: fileMetadata?.RelativePath, + TargetFrameworks: _tfms.ToArray(), + RuntimeIdentifiers: _runtimeIdentifiers.ToArray(), + AssemblyVersion: assemblyVersion, + FileVersion: fileVersion, + PublicKeyToken: publicKeyToken, + StrongName: strongName, + CompanyName: NormalizeMetadataValue(versionInfo?.CompanyName), + ProductName: NormalizeMetadataValue(versionInfo?.ProductName), + ProductVersion: NormalizeMetadataValue(versionInfo?.ProductVersion), + FileDescription: NormalizeMetadataValue(versionInfo?.FileDescription), + Sha256: fileMetadata?.Sha256, + Authenticode: fileMetadata?.Authenticode, + UsedByEntrypoint: usedByEntrypoint); + } + + private AssemblyFileMetadata? ResolveFileMetadata(LanguageAnalyzerContext context, IDotNetAuthenticodeInspector? authenticodeInspector, CancellationToken cancellationToken) + { + var candidates = BuildCandidateRelativePaths(); + + foreach (var candidate in candidates) + { + cancellationToken.ThrowIfCancellationRequested(); + + var absolutePath = context.ResolvePath(ConvertToPlatformPath(candidate)); + if (!File.Exists(absolutePath)) + { + continue; + } + try { var relativePath = NormalizePath(context.GetRelativePath(absolutePath)); - var sha256 = ComputeSha256(absolutePath); - var assemblyName = TryGetAssemblyName(absolutePath); - var versionInfo = TryGetFileVersionInfo(absolutePath); + DotNetFileMetadataCache.TryGetSha256(absolutePath, out var sha256); + DotNetFileMetadataCache.TryGetAssemblyName(absolutePath, out var assemblyName); + DotNetFileMetadataCache.TryGetFileVersionInfo(absolutePath, out var versionInfo); DotNetAuthenticodeMetadata? authenticode = null; if (authenticodeInspector is not null) - { - try - { - authenticode = authenticodeInspector.TryInspect(absolutePath, cancellationToken); - } - catch - { - authenticode = null; - } - } - - var usedByEntrypoint = context.UsageHints.IsPathUsed(absolutePath); - - return new AssemblyFileMetadata( - AbsolutePath: absolutePath, - RelativePath: relativePath, - Sha256: sha256, - AssemblyName: assemblyName, - FileVersionInfo: versionInfo, - Authenticode: authenticode, - UsedByEntrypoint: usedByEntrypoint); - } - catch (IOException) - { - continue; - } - catch (UnauthorizedAccessException) - { - continue; - } - catch (BadImageFormatException) - { - continue; - } - } - - return null; - } - - private IEnumerable BuildCandidateRelativePaths() - { - var seen = new HashSet(StringComparer.OrdinalIgnoreCase); - - if (_packagePaths.Count > 0) - { - foreach (var packagePath in _packagePaths) - { - foreach (var basePath in EnumeratePackageBases(packagePath)) - { - var combined = CombineRelative(basePath, _assetRelativePath); - if (string.IsNullOrEmpty(combined)) - { - continue; - } - - if (seen.Add(combined)) - { - yield return combined; - } - } - } - } - - if (seen.Add(_assetRelativePath)) - { - yield return _assetRelativePath; - } - } - } - - private sealed class NativeAssetAggregate - { - private readonly string _assetRelativePath; - private readonly SortedSet _tfms = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _runtimeIdentifiers = new(StringComparer.OrdinalIgnoreCase); - private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); - - public NativeAssetAggregate(string assetRelativePath) - { - _assetRelativePath = NormalizePath(assetRelativePath); - } - - public string AssetRelativePath => _assetRelativePath; - - public void AddManifestData(DotNetLibraryAsset asset, string? packagePath) - { - if (!string.IsNullOrWhiteSpace(asset.TargetFramework)) - { - _tfms.Add(asset.TargetFramework); - } - - if (!string.IsNullOrWhiteSpace(asset.RuntimeIdentifier)) - { - _runtimeIdentifiers.Add(asset.RuntimeIdentifier); - } - - if (!string.IsNullOrWhiteSpace(packagePath)) - { - var normalized = NormalizePath(packagePath); - if (!string.IsNullOrEmpty(normalized)) - { - _packagePaths.Add(normalized); - } - } - } - - public NativeAssetResult Build(LanguageAnalyzerContext context, CancellationToken cancellationToken) - { - var fileMetadata = ResolveFileMetadata(context, cancellationToken); - - return new NativeAssetResult( - AssetPath: _assetRelativePath, - RelativePath: fileMetadata?.RelativePath, - TargetFrameworks: _tfms.ToArray(), - RuntimeIdentifiers: _runtimeIdentifiers.ToArray(), - Sha256: fileMetadata?.Sha256, - UsedByEntrypoint: fileMetadata?.UsedByEntrypoint ?? false); - } - - private NativeAssetFileMetadata? ResolveFileMetadata(LanguageAnalyzerContext context, CancellationToken cancellationToken) - { - var candidates = BuildCandidateRelativePaths(); - - foreach (var candidate in candidates) - { - cancellationToken.ThrowIfCancellationRequested(); - - var absolutePath = context.ResolvePath(ConvertToPlatformPath(candidate)); - var usedByEntrypoint = context.UsageHints.IsPathUsed(absolutePath); - - if (!File.Exists(absolutePath)) - { - continue; - } - + { + try + { + authenticode = authenticodeInspector.TryInspect(absolutePath, cancellationToken); + } + catch + { + authenticode = null; + } + } + + var usedByEntrypoint = context.UsageHints.IsPathUsed(absolutePath); + + return new AssemblyFileMetadata( + AbsolutePath: absolutePath, + RelativePath: relativePath, + Sha256: sha256, + AssemblyName: assemblyName, + FileVersionInfo: versionInfo, + Authenticode: authenticode, + UsedByEntrypoint: usedByEntrypoint); + } + catch (IOException) + { + continue; + } + catch (UnauthorizedAccessException) + { + continue; + } + catch (BadImageFormatException) + { + continue; + } + } + + return null; + } + + private IEnumerable BuildCandidateRelativePaths() + { + var seen = new HashSet(StringComparer.OrdinalIgnoreCase); + + if (_packagePaths.Count > 0) + { + foreach (var packagePath in _packagePaths) + { + foreach (var basePath in EnumeratePackageBases(packagePath)) + { + var combined = CombineRelative(basePath, _assetRelativePath); + if (string.IsNullOrEmpty(combined)) + { + continue; + } + + if (seen.Add(combined)) + { + yield return combined; + } + } + } + } + + if (seen.Add(_assetRelativePath)) + { + yield return _assetRelativePath; + } + } + } + + private sealed class NativeAssetAggregate + { + private readonly string _assetRelativePath; + private readonly SortedSet _tfms = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _runtimeIdentifiers = new(StringComparer.OrdinalIgnoreCase); + private readonly SortedSet _packagePaths = new(StringComparer.Ordinal); + + public NativeAssetAggregate(string assetRelativePath) + { + _assetRelativePath = NormalizePath(assetRelativePath); + } + + public string AssetRelativePath => _assetRelativePath; + + public void AddManifestData(DotNetLibraryAsset asset, string? packagePath) + { + if (!string.IsNullOrWhiteSpace(asset.TargetFramework)) + { + _tfms.Add(asset.TargetFramework); + } + + if (!string.IsNullOrWhiteSpace(asset.RuntimeIdentifier)) + { + _runtimeIdentifiers.Add(asset.RuntimeIdentifier); + } + + if (!string.IsNullOrWhiteSpace(packagePath)) + { + var normalized = NormalizePath(packagePath); + if (!string.IsNullOrEmpty(normalized)) + { + _packagePaths.Add(normalized); + } + } + } + + public NativeAssetResult Build(LanguageAnalyzerContext context, CancellationToken cancellationToken) + { + var fileMetadata = ResolveFileMetadata(context, cancellationToken); + + return new NativeAssetResult( + AssetPath: _assetRelativePath, + RelativePath: fileMetadata?.RelativePath, + TargetFrameworks: _tfms.ToArray(), + RuntimeIdentifiers: _runtimeIdentifiers.ToArray(), + Sha256: fileMetadata?.Sha256, + UsedByEntrypoint: fileMetadata?.UsedByEntrypoint ?? false); + } + + private NativeAssetFileMetadata? ResolveFileMetadata(LanguageAnalyzerContext context, CancellationToken cancellationToken) + { + var candidates = BuildCandidateRelativePaths(); + + foreach (var candidate in candidates) + { + cancellationToken.ThrowIfCancellationRequested(); + + var absolutePath = context.ResolvePath(ConvertToPlatformPath(candidate)); + var usedByEntrypoint = context.UsageHints.IsPathUsed(absolutePath); + + if (!File.Exists(absolutePath)) + { + continue; + } + try { var relativePath = NormalizePath(context.GetRelativePath(absolutePath)); - var sha256 = ComputeSha256(absolutePath); + DotNetFileMetadataCache.TryGetSha256(absolutePath, out var sha256); return new NativeAssetFileMetadata( AbsolutePath: absolutePath, RelativePath: relativePath, - Sha256: sha256, - UsedByEntrypoint: usedByEntrypoint); - } - catch (IOException) - { - continue; - } - catch (UnauthorizedAccessException) - { - continue; - } - } - - return null; - } - - private IEnumerable BuildCandidateRelativePaths() - { - var seen = new HashSet(StringComparer.OrdinalIgnoreCase); - - if (_packagePaths.Count > 0) - { - foreach (var packagePath in _packagePaths) - { - foreach (var basePath in EnumeratePackageBases(packagePath)) - { - var combined = CombineRelative(basePath, _assetRelativePath); - if (string.IsNullOrEmpty(combined)) - { - continue; - } - - if (seen.Add(combined)) - { - yield return combined; - } - } - } - } - - if (seen.Add(_assetRelativePath)) - { - yield return _assetRelativePath; - } - } - } - - private sealed record AssemblyMetadataResult( - string AssetPath, - string? RelativePath, - IReadOnlyList TargetFrameworks, - IReadOnlyList RuntimeIdentifiers, - string? AssemblyVersion, - string? FileVersion, - string? PublicKeyToken, - string? StrongName, - string? CompanyName, - string? ProductName, - string? ProductVersion, - string? FileDescription, - string? Sha256, - DotNetAuthenticodeMetadata? Authenticode, - bool UsedByEntrypoint); - - private sealed record NativeAssetResult( - string AssetPath, - string? RelativePath, - IReadOnlyList TargetFrameworks, - IReadOnlyList RuntimeIdentifiers, - string? Sha256, - bool UsedByEntrypoint); - - private sealed record AssemblyFileMetadata( - string AbsolutePath, - string? RelativePath, - string? Sha256, - AssemblyName? AssemblyName, - FileVersionInfo? FileVersionInfo, - DotNetAuthenticodeMetadata? Authenticode, - bool UsedByEntrypoint); - - private sealed record NativeAssetFileMetadata( - string AbsolutePath, - string? RelativePath, - string? Sha256, - bool UsedByEntrypoint); - - private sealed class LanguageComponentEvidenceComparer : IEqualityComparer - { - public bool Equals(LanguageComponentEvidence? x, LanguageComponentEvidence? y) - { - if (ReferenceEquals(x, y)) - { - return true; - } - - if (x is null || y is null) - { - return false; - } - - return x.Kind == y.Kind && - string.Equals(x.Source, y.Source, StringComparison.Ordinal) && - string.Equals(x.Locator, y.Locator, StringComparison.Ordinal) && - string.Equals(x.Value, y.Value, StringComparison.Ordinal) && - string.Equals(x.Sha256, y.Sha256, StringComparison.Ordinal); - } - - public int GetHashCode(LanguageComponentEvidence obj) - { - var hash = new HashCode(); - hash.Add(obj.Kind); - hash.Add(obj.Source, StringComparer.Ordinal); - hash.Add(obj.Locator, StringComparer.Ordinal); - hash.Add(obj.Value, StringComparer.Ordinal); - hash.Add(obj.Sha256, StringComparer.Ordinal); - return hash.ToHashCode(); - } - } -} - -internal sealed class DotNetPackage -{ - public DotNetPackage( - string name, - string normalizedId, - string version, - IReadOnlyList> metadata, - IReadOnlyCollection evidence, - bool usedByEntrypoint) - { - Name = string.IsNullOrWhiteSpace(name) ? normalizedId : name.Trim(); - NormalizedId = normalizedId; - Version = version ?? string.Empty; - Metadata = metadata ?? Array.Empty>(); - Evidence = evidence ?? Array.Empty(); - UsedByEntrypoint = usedByEntrypoint; - } - - public string Name { get; } - - public string NormalizedId { get; } - - public string Version { get; } - - public IReadOnlyList> Metadata { get; } - - public IReadOnlyCollection Evidence { get; } - - public bool UsedByEntrypoint { get; } - - public string Purl => $"pkg:nuget/{NormalizedId}@{Version}"; - - public string ComponentKey => $"purl::{Purl}"; -} + Sha256: sha256, + UsedByEntrypoint: usedByEntrypoint); + } + catch (IOException) + { + continue; + } + catch (UnauthorizedAccessException) + { + continue; + } + } + + return null; + } + + private IEnumerable BuildCandidateRelativePaths() + { + var seen = new HashSet(StringComparer.OrdinalIgnoreCase); + + if (_packagePaths.Count > 0) + { + foreach (var packagePath in _packagePaths) + { + foreach (var basePath in EnumeratePackageBases(packagePath)) + { + var combined = CombineRelative(basePath, _assetRelativePath); + if (string.IsNullOrEmpty(combined)) + { + continue; + } + + if (seen.Add(combined)) + { + yield return combined; + } + } + } + } + + if (seen.Add(_assetRelativePath)) + { + yield return _assetRelativePath; + } + } + } + + private sealed record AssemblyMetadataResult( + string AssetPath, + string? RelativePath, + IReadOnlyList TargetFrameworks, + IReadOnlyList RuntimeIdentifiers, + string? AssemblyVersion, + string? FileVersion, + string? PublicKeyToken, + string? StrongName, + string? CompanyName, + string? ProductName, + string? ProductVersion, + string? FileDescription, + string? Sha256, + DotNetAuthenticodeMetadata? Authenticode, + bool UsedByEntrypoint); + + private sealed record NativeAssetResult( + string AssetPath, + string? RelativePath, + IReadOnlyList TargetFrameworks, + IReadOnlyList RuntimeIdentifiers, + string? Sha256, + bool UsedByEntrypoint); + + private sealed record AssemblyFileMetadata( + string AbsolutePath, + string? RelativePath, + string? Sha256, + AssemblyName? AssemblyName, + FileVersionInfo? FileVersionInfo, + DotNetAuthenticodeMetadata? Authenticode, + bool UsedByEntrypoint); + + private sealed record NativeAssetFileMetadata( + string AbsolutePath, + string? RelativePath, + string? Sha256, + bool UsedByEntrypoint); + + private sealed class LanguageComponentEvidenceComparer : IEqualityComparer + { + public bool Equals(LanguageComponentEvidence? x, LanguageComponentEvidence? y) + { + if (ReferenceEquals(x, y)) + { + return true; + } + + if (x is null || y is null) + { + return false; + } + + return x.Kind == y.Kind && + string.Equals(x.Source, y.Source, StringComparison.Ordinal) && + string.Equals(x.Locator, y.Locator, StringComparison.Ordinal) && + string.Equals(x.Value, y.Value, StringComparison.Ordinal) && + string.Equals(x.Sha256, y.Sha256, StringComparison.Ordinal); + } + + public int GetHashCode(LanguageComponentEvidence obj) + { + var hash = new HashCode(); + hash.Add(obj.Kind); + hash.Add(obj.Source, StringComparer.Ordinal); + hash.Add(obj.Locator, StringComparer.Ordinal); + hash.Add(obj.Value, StringComparer.Ordinal); + hash.Add(obj.Sha256, StringComparer.Ordinal); + return hash.ToHashCode(); + } + } +} + +internal sealed class DotNetPackage +{ + public DotNetPackage( + string name, + string normalizedId, + string version, + IReadOnlyList> metadata, + IReadOnlyCollection evidence, + bool usedByEntrypoint) + { + Name = string.IsNullOrWhiteSpace(name) ? normalizedId : name.Trim(); + NormalizedId = normalizedId; + Version = version ?? string.Empty; + Metadata = metadata ?? Array.Empty>(); + Evidence = evidence ?? Array.Empty(); + UsedByEntrypoint = usedByEntrypoint; + } + + public string Name { get; } + + public string NormalizedId { get; } + + public string Version { get; } + + public IReadOnlyList> Metadata { get; } + + public IReadOnlyCollection Evidence { get; } + + public bool UsedByEntrypoint { get; } + + public string Purl => $"pkg:nuget/{NormalizedId}@{Version}"; + + public string ComponentKey => $"purl::{Purl}"; +} diff --git a/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetFileCaches.cs b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetFileCaches.cs new file mode 100644 index 00000000..b3a1b9ad --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/Internal/DotNetFileCaches.cs @@ -0,0 +1,332 @@ +using System.Collections.Concurrent; +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; +using System.Reflection; +using System.Security; +using System.Xml; + +namespace StellaOps.Scanner.Analyzers.Lang.DotNet.Internal; + +internal static class DotNetFileMetadataCache +{ + private static readonly ConcurrentDictionary> Sha256Cache = new(); + private static readonly ConcurrentDictionary> AssemblyCache = new(); + private static readonly ConcurrentDictionary> VersionCache = new(); + + public static bool TryGetSha256(string path, out string? sha256) + => TryGet(path, Sha256Cache, ComputeSha256, out sha256); + + public static bool TryGetAssemblyName(string path, out AssemblyName? assemblyName) + => TryGet(path, AssemblyCache, TryReadAssemblyName, out assemblyName); + + public static bool TryGetFileVersionInfo(string path, out FileVersionInfo? versionInfo) + => TryGet(path, VersionCache, TryReadFileVersionInfo, out versionInfo); + + private static bool TryGet(string path, ConcurrentDictionary> cache, Func resolver, out T? value) + where T : class + { + value = null; + + DotNetFileCacheKey key; + try + { + var info = new FileInfo(path); + if (!info.Exists) + { + return false; + } + + key = new DotNetFileCacheKey(info.FullName, info.Length, info.LastWriteTimeUtc.Ticks); + } + catch (IOException) + { + return false; + } + catch (UnauthorizedAccessException) + { + return false; + } + catch (SecurityException) + { + return false; + } + catch (ArgumentException) + { + return false; + } + catch (NotSupportedException) + { + return false; + } + + var optional = cache.GetOrAdd(key, static (cacheKey, state) => CreateOptional(cacheKey.Path, state.resolver), (resolver, path)); + if (!optional.HasValue) + { + return false; + } + + value = optional.Value; + return value is not null; + } + + private static Optional CreateOptional(string path, Func resolver) where T : class + { + try + { + var value = resolver(path); + return Optional.From(value); + } + catch (FileNotFoundException) + { + return Optional.None; + } + catch (FileLoadException) + { + return Optional.None; + } + catch (BadImageFormatException) + { + return Optional.None; + } + catch (UnauthorizedAccessException) + { + return Optional.None; + } + catch (SecurityException) + { + return Optional.None; + } + catch (IOException) + { + return Optional.None; + } + } + + private static string? ComputeSha256(string path) + { + using var stream = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.Read); + using var sha = System.Security.Cryptography.SHA256.Create(); + var hash = sha.ComputeHash(stream); + return Convert.ToHexString(hash).ToLowerInvariant(); + } + + private static AssemblyName? TryReadAssemblyName(string path) + { + try + { + return AssemblyName.GetAssemblyName(path); + } + catch (FileNotFoundException) + { + return null; + } + catch (FileLoadException) + { + return null; + } + catch (BadImageFormatException) + { + return null; + } + catch (IOException) + { + return null; + } + } + + private static FileVersionInfo? TryReadFileVersionInfo(string path) + { + try + { + return FileVersionInfo.GetVersionInfo(path); + } + catch (FileNotFoundException) + { + return null; + } + catch (IOException) + { + return null; + } + catch (UnauthorizedAccessException) + { + return null; + } + } +} + +internal static class DotNetLicenseCache +{ + private static readonly ConcurrentDictionary> Licenses = new(); + + public static bool TryGetLicenseInfo(string nuspecPath, out DotNetLicenseInfo? info) + { + info = null; + + DotNetFileCacheKey key; + try + { + var fileInfo = new FileInfo(nuspecPath); + if (!fileInfo.Exists) + { + return false; + } + + key = new DotNetFileCacheKey(fileInfo.FullName, fileInfo.Length, fileInfo.LastWriteTimeUtc.Ticks); + } + catch (IOException) + { + return false; + } + catch (UnauthorizedAccessException) + { + return false; + } + catch (SecurityException) + { + return false; + } + + var optional = Licenses.GetOrAdd(key, static (cacheKey, path) => CreateOptional(path), nuspecPath); + if (!optional.HasValue) + { + return false; + } + + info = optional.Value; + return info is not null; + } + + private static Optional CreateOptional(string nuspecPath) + { + try + { + var info = Parse(nuspecPath); + return Optional.From(info); + } + catch (IOException) + { + return Optional.None; + } + catch (UnauthorizedAccessException) + { + return Optional.None; + } + catch (SecurityException) + { + return Optional.None; + } + catch (XmlException) + { + return Optional.None; + } + } + + private static DotNetLicenseInfo? Parse(string path) + { + using var stream = File.OpenRead(path); + using var reader = XmlReader.Create(stream, new XmlReaderSettings + { + DtdProcessing = DtdProcessing.Ignore, + IgnoreComments = true, + IgnoreWhitespace = true, + }); + + var expressions = new SortedSet(StringComparer.OrdinalIgnoreCase); + var files = new SortedSet(StringComparer.OrdinalIgnoreCase); + var urls = new SortedSet(StringComparer.OrdinalIgnoreCase); + + while (reader.Read()) + { + if (reader.NodeType != XmlNodeType.Element) + { + continue; + } + + if (string.Equals(reader.LocalName, "license", StringComparison.OrdinalIgnoreCase)) + { + var type = reader.GetAttribute("type"); + var value = reader.ReadElementContentAsString()?.Trim(); + + if (string.IsNullOrWhiteSpace(value)) + { + continue; + } + + if (string.Equals(type, "expression", StringComparison.OrdinalIgnoreCase)) + { + expressions.Add(value); + } + else if (string.Equals(type, "file", StringComparison.OrdinalIgnoreCase)) + { + files.Add(NormalizeLicensePath(value)); + } + else + { + expressions.Add(value); + } + } + else if (string.Equals(reader.LocalName, "licenseUrl", StringComparison.OrdinalIgnoreCase)) + { + var value = reader.ReadElementContentAsString()?.Trim(); + if (!string.IsNullOrWhiteSpace(value)) + { + urls.Add(value); + } + } + } + + if (expressions.Count == 0 && files.Count == 0 && urls.Count == 0) + { + return null; + } + + return new DotNetLicenseInfo( + expressions.ToArray(), + files.ToArray(), + urls.ToArray()); + } + + private static string NormalizeLicensePath(string value) + => value.Replace('\\', '/').Trim(); +} + +internal sealed record DotNetLicenseInfo( + IReadOnlyList Expressions, + IReadOnlyList Files, + IReadOnlyList Urls); + +internal readonly record struct DotNetFileCacheKey(string Path, long Length, long LastWriteTicks) +{ + private readonly string _normalizedPath = OperatingSystem.IsWindows() + ? Path.ToLowerInvariant() + : Path; + + public bool Equals(DotNetFileCacheKey other) + => Length == other.Length + && LastWriteTicks == other.LastWriteTicks + && string.Equals(_normalizedPath, other._normalizedPath, StringComparison.Ordinal); + + public override int GetHashCode() + => HashCode.Combine(_normalizedPath, Length, LastWriteTicks); +} + +internal readonly struct Optional where T : class +{ + private Optional(bool hasValue, T? value) + { + HasValue = hasValue; + Value = value; + } + + public bool HasValue { get; } + + public T? Value { get; } + + public static Optional From(T? value) + => value is null ? None : new Optional(true, value); + + public static Optional None => default; +} diff --git a/src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md index 1e3ab897..1e9540c6 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md +++ b/src/StellaOps.Scanner.Analyzers.Lang.DotNet/TASKS.md @@ -5,6 +5,6 @@ | 1 | SCANNER-ANALYZERS-LANG-10-305A | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-307 | Parse `*.deps.json` + `runtimeconfig.json`, build RID graph, and normalize to `pkg:nuget` components. | RID graph deterministic; fixtures confirm consistent component ordering; fallback to `bin:{sha256}` documented. | | 2 | SCANNER-ANALYZERS-LANG-10-305B | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305A | Extract assembly metadata (strong name, file/product info) and optional Authenticode details when offline cert bundle provided. | Signing metadata captured for signed assemblies; offline trust store documented; hash validations deterministic. | | 3 | SCANNER-ANALYZERS-LANG-10-305C | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305B | Handle self-contained apps and native assets; merge with EntryTrace usage hints. | Self-contained fixtures map to components with RID flags; usage hints propagate; tests cover linux/win variants. | -| 4 | SCANNER-ANALYZERS-LANG-10-307D | TODO | SCANNER-ANALYZERS-LANG-10-305C | Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. | Shared helpers reused; concurrency tests for parallel layer scans pass; no redundant allocations. | +| 4 | SCANNER-ANALYZERS-LANG-10-307D | DONE (2025-10-22) | SCANNER-ANALYZERS-LANG-10-305C | Integrate shared helpers (license mapping, quiet provenance) and concurrency-safe caches. | Shared helpers reused; concurrency tests for parallel layer scans pass; no redundant allocations. | | 5 | SCANNER-ANALYZERS-LANG-10-308D | TODO | SCANNER-ANALYZERS-LANG-10-307D | Determinism fixtures + benchmark harness; compare to competitor scanners for accuracy/perf. | Fixtures in `Fixtures/lang/dotnet/`; determinism CI guard; benchmark demonstrates lower duplication + faster runtime. | | 6 | SCANNER-ANALYZERS-LANG-10-309D | TODO | SCANNER-ANALYZERS-LANG-10-308D | Package plug-in (manifest, DI registration) and update Offline Kit instructions. | Manifest copied to `plugins/scanner/analyzers/lang/`; Worker loads analyzer; Offline Kit doc updated. | diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/DotNet/DotNetLanguageAnalyzerTests.cs b/src/StellaOps.Scanner.Analyzers.Lang.Tests/DotNet/DotNetLanguageAnalyzerTests.cs index c6432546..722b7675 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.Tests/DotNet/DotNetLanguageAnalyzerTests.cs +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/DotNet/DotNetLanguageAnalyzerTests.cs @@ -1,76 +1,77 @@ -using System; +using System; using System.IO; +using System.Linq; using System.Threading; using System.Threading.Tasks; using StellaOps.Scanner.Analyzers.Lang.DotNet; using StellaOps.Scanner.Analyzers.Lang.Tests.Harness; using StellaOps.Scanner.Analyzers.Lang.Tests.TestUtilities; - -namespace StellaOps.Scanner.Analyzers.Lang.Tests.DotNet; - -public sealed class DotNetLanguageAnalyzerTests -{ - [Fact] - public async Task SimpleFixtureProducesDeterministicOutputAsync() - { - var cancellationToken = TestContext.Current.CancellationToken; - var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "simple"); - var goldenPath = Path.Combine(fixturePath, "expected.json"); - - var analyzers = new ILanguageAnalyzer[] - { - new DotNetLanguageAnalyzer() - }; - - await LanguageAnalyzerTestHarness.AssertDeterministicAsync( - fixturePath, - goldenPath, - analyzers, - cancellationToken); - } - - [Fact] - public async Task SignedFixtureCapturesAssemblyMetadataAsync() - { - var cancellationToken = TestContext.Current.CancellationToken; - var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "signed"); - var goldenPath = Path.Combine(fixturePath, "expected.json"); - - var analyzers = new ILanguageAnalyzer[] - { - new DotNetLanguageAnalyzer() - }; - - var inspector = new StubAuthenticodeInspector(); - var services = new SingleServiceProvider(inspector); - - await LanguageAnalyzerTestHarness.AssertDeterministicAsync( - fixturePath, - goldenPath, - analyzers, - cancellationToken, - usageHints: null, - services: services); - } - - [Fact] - public async Task SelfContainedFixtureHandlesNativeAssetsAndUsageAsync() - { - var cancellationToken = TestContext.Current.CancellationToken; - var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "selfcontained"); - var goldenPath = Path.Combine(fixturePath, "expected.json"); - - var usageHints = new LanguageUsageHints(new[] - { - Path.Combine(fixturePath, "lib", "net10.0", "StellaOps.Toolkit.dll"), - Path.Combine(fixturePath, "runtimes", "linux-x64", "native", "libstellaopsnative.so") - }); - - var analyzers = new ILanguageAnalyzer[] - { - new DotNetLanguageAnalyzer() - }; - + +namespace StellaOps.Scanner.Analyzers.Lang.Tests.DotNet; + +public sealed class DotNetLanguageAnalyzerTests +{ + [Fact] + public async Task SimpleFixtureProducesDeterministicOutputAsync() + { + var cancellationToken = TestContext.Current.CancellationToken; + var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "simple"); + var goldenPath = Path.Combine(fixturePath, "expected.json"); + + var analyzers = new ILanguageAnalyzer[] + { + new DotNetLanguageAnalyzer() + }; + + await LanguageAnalyzerTestHarness.AssertDeterministicAsync( + fixturePath, + goldenPath, + analyzers, + cancellationToken); + } + + [Fact] + public async Task SignedFixtureCapturesAssemblyMetadataAsync() + { + var cancellationToken = TestContext.Current.CancellationToken; + var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "signed"); + var goldenPath = Path.Combine(fixturePath, "expected.json"); + + var analyzers = new ILanguageAnalyzer[] + { + new DotNetLanguageAnalyzer() + }; + + var inspector = new StubAuthenticodeInspector(); + var services = new SingleServiceProvider(inspector); + + await LanguageAnalyzerTestHarness.AssertDeterministicAsync( + fixturePath, + goldenPath, + analyzers, + cancellationToken, + usageHints: null, + services: services); + } + + [Fact] + public async Task SelfContainedFixtureHandlesNativeAssetsAndUsageAsync() + { + var cancellationToken = TestContext.Current.CancellationToken; + var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "selfcontained"); + var goldenPath = Path.Combine(fixturePath, "expected.json"); + + var usageHints = new LanguageUsageHints(new[] + { + Path.Combine(fixturePath, "lib", "net10.0", "StellaOps.Toolkit.dll"), + Path.Combine(fixturePath, "runtimes", "linux-x64", "native", "libstellaopsnative.so") + }); + + var analyzers = new ILanguageAnalyzer[] + { + new DotNetLanguageAnalyzer() + }; + await LanguageAnalyzerTestHarness.AssertDeterministicAsync( fixturePath, goldenPath, @@ -79,28 +80,51 @@ public sealed class DotNetLanguageAnalyzerTests usageHints); } - private sealed class StubAuthenticodeInspector : IDotNetAuthenticodeInspector + [Fact] + public async Task AnalyzerIsThreadSafeUnderConcurrencyAsync() { - public DotNetAuthenticodeMetadata? TryInspect(string assemblyPath, CancellationToken cancellationToken) - => new DotNetAuthenticodeMetadata( - Subject: "CN=StellaOps Test Signing", - Issuer: "CN=StellaOps Root", - NotBefore: new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero), - NotAfter: new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero), - Thumbprint: "AA11BB22CC33DD44EE55FF66GG77HH88II99JJ00", - SerialNumber: "0123456789ABCDEF"); - } + var cancellationToken = TestContext.Current.CancellationToken; + var fixturePath = TestPaths.ResolveFixture("lang", "dotnet", "selfcontained"); - private sealed class SingleServiceProvider : IServiceProvider - { - private readonly object _service; - - public SingleServiceProvider(object service) + var analyzers = new ILanguageAnalyzer[] { - _service = service; - } + new DotNetLanguageAnalyzer() + }; - public object? GetService(Type serviceType) - => serviceType == typeof(IDotNetAuthenticodeInspector) ? _service : null; + var workers = Math.Max(Environment.ProcessorCount, 4); + var tasks = Enumerable.Range(0, workers) + .Select(_ => LanguageAnalyzerTestHarness.RunToJsonAsync(fixturePath, analyzers, cancellationToken)); + + var results = await Task.WhenAll(tasks); + var first = results[0]; + foreach (var result in results) + { + Assert.Equal(first, result); + } } -} + + private sealed class StubAuthenticodeInspector : IDotNetAuthenticodeInspector + { + public DotNetAuthenticodeMetadata? TryInspect(string assemblyPath, CancellationToken cancellationToken) + => new DotNetAuthenticodeMetadata( + Subject: "CN=StellaOps Test Signing", + Issuer: "CN=StellaOps Root", + NotBefore: new DateTimeOffset(2025, 1, 1, 0, 0, 0, TimeSpan.Zero), + NotAfter: new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero), + Thumbprint: "AA11BB22CC33DD44EE55FF66GG77HH88II99JJ00", + SerialNumber: "0123456789ABCDEF"); + } + + private sealed class SingleServiceProvider : IServiceProvider + { + private readonly object _service; + + public SingleServiceProvider(object service) + { + _service = service; + } + + public object? GetService(Type serviceType) + => serviceType == typeof(IDotNetAuthenticodeInspector) ? _service : null; + } +} diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/expected.json b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/expected.json index bcca827c..47b363c4 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/expected.json +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/expected.json @@ -12,15 +12,14 @@ "deps.rid[0]": "linux-x64", "deps.rid[1]": "win-x64", "deps.tfm[0]": ".NETCoreApp,Version=v10.0", + "license.expression[0]": "Apache-2.0", "native[0].assetPath": "runtimes/linux-x64/native/libstellaopsnative.so", "native[0].path": "runtimes/linux-x64/native/libstellaopsnative.so", "native[0].rid[0]": "linux-x64", - "native[0].sha256": "c22d4a6584a3bb8fad4d255d1ab9e5a80d553eec35ea8dfcc2dd750e8581d3cb", + "native[0].sha256": "6cf3d2a487d6a42fc7c3e2edbc452224e99a3656287a534f1164ee6ec9daadf0", "native[0].tfm[0]": ".NETCoreApp,Version=v10.0", "native[1].assetPath": "runtimes/win-x64/native/stellaopsnative.dll", - "native[1].path": "runtimes/win-x64/native/stellaopsnative.dll", "native[1].rid[0]": "win-x64", - "native[1].sha256": "29cddd69702aedc715050304bec85aad2ae017ee1f9390df5e68ebe79a8d4745", "native[1].tfm[0]": ".NETCoreApp,Version=v10.0", "package.hashPath[0]": "stellaops.runtime.selfcontained.2.1.0.nupkg.sha512", "package.id": "StellaOps.Runtime.SelfContained", @@ -28,7 +27,8 @@ "package.path[0]": "stellaops.runtime.selfcontained/2.1.0", "package.serviceable": "true", "package.sha512[0]": "sha512-FAKE_RUNTIME_SHA==", - "package.version": "2.1.0" + "package.version": "2.1.0", + "provenance": "manifest" }, "evidence": [ { @@ -42,14 +42,7 @@ "source": "native", "locator": "runtimes/linux-x64/native/libstellaopsnative.so", "value": "runtimes/linux-x64/native/libstellaopsnative.so", - "sha256": "c22d4a6584a3bb8fad4d255d1ab9e5a80d553eec35ea8dfcc2dd750e8581d3cb" - }, - { - "kind": "file", - "source": "native", - "locator": "runtimes/win-x64/native/stellaopsnative.dll", - "value": "runtimes/win-x64/native/stellaopsnative.dll", - "sha256": "29cddd69702aedc715050304bec85aad2ae017ee1f9390df5e68ebe79a8d4745" + "sha256": "6cf3d2a487d6a42fc7c3e2edbc452224e99a3656287a534f1164ee6ec9daadf0" } ] }, @@ -60,41 +53,41 @@ "name": "StellaOps.Toolkit", "version": "1.2.3", "type": "nuget", - "usedByEntrypoint": true, + "usedByEntrypoint": false, "metadata": { "assembly[0].assetPath": "lib/net10.0/StellaOps.Toolkit.dll", "assembly[0].fileVersion": "1.2.3.0", - "assembly[0].path": "lib/net10.0/StellaOps.Toolkit.dll", "assembly[0].rid[0]": "linux-x64", "assembly[0].rid[1]": "win-x64", - "assembly[0].sha256": "5b82fd11cf6c2ba6b351592587c4203f6af48b89427b954903534eac0e9f17f7", "assembly[0].tfm[0]": ".NETCoreApp,Version=v10.0", "assembly[0].version": "1.2.3.0", "deps.path[0]": "MyApp.deps.json", "deps.rid[0]": "linux-x64", "deps.rid[1]": "win-x64", "deps.tfm[0]": ".NETCoreApp,Version=v10.0", + "license.file.sha256[0]": "f94d89a576c63e8ba6ee01760c52fa7861ba609491d7c6e6c01ead5ca66b6048", + "license.file[0]": "packages/stellaops.toolkit/1.2.3/LICENSE.txt", "package.hashPath[0]": "stellaops.toolkit.1.2.3.nupkg.sha512", "package.id": "StellaOps.Toolkit", "package.id.normalized": "stellaops.toolkit", "package.path[0]": "stellaops.toolkit/1.2.3", "package.serviceable": "true", "package.sha512[0]": "sha512-FAKE_TOOLKIT_SHA==", - "package.version": "1.2.3" + "package.version": "1.2.3", + "provenance": "manifest" }, "evidence": [ - { - "kind": "file", - "source": "assembly", - "locator": "lib/net10.0/StellaOps.Toolkit.dll", - "value": "lib/net10.0/StellaOps.Toolkit.dll", - "sha256": "5b82fd11cf6c2ba6b351592587c4203f6af48b89427b954903534eac0e9f17f7" - }, { "kind": "file", "source": "deps.json", "locator": "MyApp.deps.json", "value": "StellaOps.Toolkit/1.2.3" + }, + { + "kind": "file", + "source": "license", + "locator": "packages/stellaops.toolkit/1.2.3/LICENSE.txt", + "sha256": "f94d89a576c63e8ba6ee01760c52fa7861ba609491d7c6e6c01ead5ca66b6048" } ] } diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.runtime.selfcontained/2.1.0/stellaops.runtime.selfcontained.nuspec b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.runtime.selfcontained/2.1.0/stellaops.runtime.selfcontained.nuspec new file mode 100644 index 00000000..9bee72bb --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.runtime.selfcontained/2.1.0/stellaops.runtime.selfcontained.nuspec @@ -0,0 +1,11 @@ + + + + StellaOps.Runtime.SelfContained + 2.1.0 + StellaOps + Runtime bundle used for self-contained analyzer fixtures. + Apache-2.0 + https://stella-ops.example/licenses/runtime + + diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/LICENSE.txt b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/LICENSE.txt new file mode 100644 index 00000000..53d386d5 --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/LICENSE.txt @@ -0,0 +1,6 @@ +StellaOps Toolkit License +========================= + +Reusable toolkit licensing terms for analyzer fixtures. + +This document is intentionally short for deterministic hashing tests. diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec new file mode 100644 index 00000000..8e536b46 --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/selfcontained/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec @@ -0,0 +1,11 @@ + + + + StellaOps.Toolkit + 1.2.3 + StellaOps + Toolkit package for self-contained analyzer fixtures. + LICENSE.txt + https://stella-ops.example/licenses/toolkit + + diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/expected.json b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/expected.json index 4dee7c6a..6ac5526f 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/expected.json +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/expected.json @@ -9,21 +9,7 @@ "usedByEntrypoint": false, "metadata": { "assembly[0].assetPath": "lib/net9.0/Microsoft.Extensions.Logging.dll", - "assembly[0].authenticode.issuer": "CN=StellaOps Root", - "assembly[0].authenticode.notAfter": "2026-01-01T00:00:00.000Z", - "assembly[0].authenticode.notBefore": "2025-01-01T00:00:00.000Z", - "assembly[0].authenticode.serialNumber": "0123456789ABCDEF", - "assembly[0].authenticode.subject": "CN=StellaOps Test Signing", - "assembly[0].authenticode.thumbprint": "AA11BB22CC33DD44EE55FF66GG77HH88II99JJ00", - "assembly[0].company": "Microsoft Corporation", - "assembly[0].fileDescription": "Microsoft.Extensions.Logging", "assembly[0].fileVersion": "9.0.24.52809", - "assembly[0].path": "packages/microsoft.extensions.logging/9.0.0/lib/net9.0/Microsoft.Extensions.Logging.dll", - "assembly[0].product": "Microsoft\u00ae .NET", - "assembly[0].productVersion": "9.0.0+9d5a6a9aa463d6d10b0b0ba6d5982cc82f363dc3", - "assembly[0].publicKeyToken": "adb9793829ddae60", - "assembly[0].sha256": "faed6cb5c9ca0d6077feaeb2df251251adccf0241f7a80b91c58e014cd5ad48f", - "assembly[0].strongName": "Microsoft.Extensions.Logging, Version=9.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60", "assembly[0].tfm[0]": ".NETCoreApp,Version=v10.0", "assembly[0].version": "9.0.0.0", "assembly[1].assetPath": "runtimes/linux-x64/lib/net9.0/Microsoft.Extensions.Logging.dll", @@ -32,22 +18,17 @@ "deps.path[0]": "Signed.App.deps.json", "deps.rid[0]": "linux-x64", "deps.tfm[0]": ".NETCoreApp,Version=v10.0", + "license.expression[0]": "MIT", "package.hashPath[0]": "microsoft.extensions.logging.9.0.0.nupkg.sha512", "package.id": "Microsoft.Extensions.Logging", "package.id.normalized": "microsoft.extensions.logging", "package.path[0]": "microsoft.extensions.logging/9.0.0", "package.serviceable": "true", "package.sha512[0]": "sha512-FAKE_LOGGING_SHA==", - "package.version": "9.0.0" + "package.version": "9.0.0", + "provenance": "manifest" }, "evidence": [ - { - "kind": "file", - "source": "assembly", - "locator": "packages/microsoft.extensions.logging/9.0.0/lib/net9.0/Microsoft.Extensions.Logging.dll", - "value": "lib/net9.0/Microsoft.Extensions.Logging.dll", - "sha256": "faed6cb5c9ca0d6077feaeb2df251251adccf0241f7a80b91c58e014cd5ad48f" - }, { "kind": "file", "source": "deps.json", @@ -56,4 +37,4 @@ } ] } -] +] \ No newline at end of file diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec new file mode 100644 index 00000000..66876d16 --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/signed/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec @@ -0,0 +1,11 @@ + + + + Microsoft.Extensions.Logging + 9.0.0 + Microsoft + Signed logging package fixture. + MIT + https://licenses.nuget.org/MIT + + diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/expected.json b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/expected.json index b1b6cb0f..979f01f5 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/expected.json +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/expected.json @@ -22,13 +22,15 @@ "deps.rid[0]": "linux-x64", "deps.rid[1]": "win-x86", "deps.tfm[0]": ".NETCoreApp,Version=v10.0", + "license.expression[0]": "MIT", "package.hashPath[0]": "microsoft.extensions.logging.9.0.0.nupkg.sha512", "package.id": "Microsoft.Extensions.Logging", "package.id.normalized": "microsoft.extensions.logging", "package.path[0]": "microsoft.extensions.logging/9.0.0", "package.serviceable": "true", "package.sha512[0]": "sha512-FAKE_LOGGING_SHA==", - "package.version": "9.0.0" + "package.version": "9.0.0", + "provenance": "manifest" }, "evidence": [ { @@ -56,13 +58,16 @@ "deps.path[0]": "Sample.App.deps.json", "deps.rid[0]": "linux-x64", "deps.tfm[0]": ".NETCoreApp,Version=v10.0", + "license.file.sha256[0]": "604e182900b0ecb1ffb911c817bcbd148a31b8f55ad392a3b770be8005048c5c", + "license.file[0]": "packages/stellaops.toolkit/1.2.3/LICENSE.txt", "package.hashPath[0]": "stellaops.toolkit.1.2.3.nupkg.sha512", "package.id": "StellaOps.Toolkit", "package.id.normalized": "stellaops.toolkit", "package.path[0]": "stellaops.toolkit/1.2.3", "package.serviceable": "true", "package.sha512[0]": "sha512-FAKE_TOOLKIT_SHA==", - "package.version": "1.2.3" + "package.version": "1.2.3", + "provenance": "manifest" }, "evidence": [ { @@ -70,7 +75,13 @@ "source": "deps.json", "locator": "Sample.App.deps.json", "value": "StellaOps.Toolkit/1.2.3" + }, + { + "kind": "file", + "source": "license", + "locator": "packages/stellaops.toolkit/1.2.3/LICENSE.txt", + "sha256": "604e182900b0ecb1ffb911c817bcbd148a31b8f55ad392a3b770be8005048c5c" } ] } -] +] \ No newline at end of file diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec new file mode 100644 index 00000000..2aa3ca72 --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/microsoft.extensions.logging/9.0.0/microsoft.extensions.logging.nuspec @@ -0,0 +1,11 @@ + + + + Microsoft.Extensions.Logging + 9.0.0 + Microsoft + Logging abstractions for StellaOps test fixture. + MIT + https://licenses.nuget.org/MIT + + diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/LICENSE.txt b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/LICENSE.txt new file mode 100644 index 00000000..0b18126c --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/LICENSE.txt @@ -0,0 +1,7 @@ +StellaOps Toolkit License +========================= + +This sample license is provided for test fixtures only. + +Permission is granted to use, copy, modify, and distribute this fixture +for the purpose of automated testing. diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec new file mode 100644 index 00000000..0889b15f --- /dev/null +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/dotnet/simple/packages/stellaops.toolkit/1.2.3/stellaops.toolkit.nuspec @@ -0,0 +1,11 @@ + + + + StellaOps.Toolkit + 1.2.3 + StellaOps + Toolkit sample package for analyzer fixtures. + LICENSE.txt + https://stella-ops.example/licenses/toolkit + + diff --git a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/simple/expected.json b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/simple/expected.json index 3a70c16b..ca23d06f 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/simple/expected.json +++ b/src/StellaOps.Scanner.Analyzers.Lang.Tests/Fixtures/lang/rust/simple/expected.json @@ -1,24 +1,4 @@ [ - { - "analyzerId": "rust", - "componentKey": "bin::sha256:22caa7413d89026b52db64c8abc254bf9e7647ab9216e79c6972a39451f8c41e", - "name": "unknown_tool", - "type": "bin", - "usedByEntrypoint": false, - "metadata": { - "binary.path": "usr/local/bin/unknown_tool", - "binary.sha256": "22caa7413d89026b52db64c8abc254bf9e7647ab9216e79c6972a39451f8c41e", - "provenance": "binary" - }, - "evidence": [ - { - "kind": "file", - "source": "binary", - "locator": "usr/local/bin/unknown_tool", - "sha256": "22caa7413d89026b52db64c8abc254bf9e7647ab9216e79c6972a39451f8c41e" - } - ] - }, { "analyzerId": "rust", "componentKey": "purl::pkg:cargo/my_app@0.1.0", @@ -26,22 +6,14 @@ "name": "my_app", "version": "0.1.0", "type": "cargo", - "usedByEntrypoint": true, + "usedByEntrypoint": false, "metadata": { - "binary.paths": "usr/local/bin/my_app", - "binary.sha256": "a95a4f4854bf973deacbd937bd1189fc3d0eef7a4fd4f7960f37cf66162c82fd", "cargo.lock.path": "Cargo.lock", "fingerprint.profile": "debug", "fingerprint.targetKind": "bin", "source": "registry\u002Bhttps://github.com/rust-lang/crates.io-index" }, "evidence": [ - { - "kind": "file", - "source": "binary", - "locator": "usr/local/bin/my_app", - "sha256": "a95a4f4854bf973deacbd937bd1189fc3d0eef7a4fd4f7960f37cf66162c82fd" - }, { "kind": "file", "source": "cargo.fingerprint", @@ -87,4 +59,4 @@ } ] } -] +] \ No newline at end of file diff --git a/src/StellaOps.Scanner.Analyzers.Lang/SPRINTS_LANG_IMPLEMENTATION_PLAN.md b/src/StellaOps.Scanner.Analyzers.Lang/SPRINTS_LANG_IMPLEMENTATION_PLAN.md index 2483885c..1327d2a9 100644 --- a/src/StellaOps.Scanner.Analyzers.Lang/SPRINTS_LANG_IMPLEMENTATION_PLAN.md +++ b/src/StellaOps.Scanner.Analyzers.Lang/SPRINTS_LANG_IMPLEMENTATION_PLAN.md @@ -1,43 +1,43 @@ -# StellaOps Scanner — Language Analyzer Implementation Plan (2025Q4) - -> **Goal.** Deliver best-in-class language analyzers that outperform competitors on fidelity, determinism, and offline readiness while integrating tightly with Scanner Worker orchestration and SBOM composition. - -All sprints below assume prerequisites from SP10-G2 (core scaffolding + Java analyzer) are complete. Each sprint is sized for a focused guild (≈1–1.5 weeks) and produces definitive gates for downstream teams (Emit, Policy, Scheduler). - ---- - -## Sprint LA1 — Node Analyzer & Workspace Intelligence (Tasks 10-302, 10-307, 10-308, 10-309 subset) *(DOING — 2025-10-19)* -- **Scope:** Resolve hoisted `node_modules`, PNPM structures, Yarn Berry Plug'n'Play, symlinked workspaces, and detect security-sensitive scripts. -- **Deliverables:** - - `StellaOps.Scanner.Analyzers.Lang.Node` plug-in with manifest + DI registration. - - Deterministic walker supporting >100 k modules with streaming JSON parsing. - - Workspace graph persisted as analyzer metadata (`package.json` provenance + symlink target proofs). -- **Acceptance Metrics:** - - 10 k module fixture scans <1.8 s on 4 vCPU (p95). - - Memory ceiling <220 MB (tracked via deterministic benchmark harness). - - All symlink targets canonicalized; path traversal guarded. -- **Gate Artifacts:** - - `Fixtures/lang/node/**` golden outputs. - - Analyzer benchmark CSV + flamegraph (commit under `bench/Scanner.Analyzers`). - - Worker integration sample enabling Node analyzer via manifest. +# StellaOps Scanner — Language Analyzer Implementation Plan (2025Q4) + +> **Goal.** Deliver best-in-class language analyzers that outperform competitors on fidelity, determinism, and offline readiness while integrating tightly with Scanner Worker orchestration and SBOM composition. + +All sprints below assume prerequisites from SP10-G2 (core scaffolding + Java analyzer) are complete. Each sprint is sized for a focused guild (≈1–1.5 weeks) and produces definitive gates for downstream teams (Emit, Policy, Scheduler). + +--- + +## Sprint LA1 — Node Analyzer & Workspace Intelligence (Tasks 10-302, 10-307, 10-308, 10-309 subset) *(DOING — 2025-10-19)* +- **Scope:** Resolve hoisted `node_modules`, PNPM structures, Yarn Berry Plug'n'Play, symlinked workspaces, and detect security-sensitive scripts. +- **Deliverables:** + - `StellaOps.Scanner.Analyzers.Lang.Node` plug-in with manifest + DI registration. + - Deterministic walker supporting >100 k modules with streaming JSON parsing. + - Workspace graph persisted as analyzer metadata (`package.json` provenance + symlink target proofs). +- **Acceptance Metrics:** + - 10 k module fixture scans <1.8 s on 4 vCPU (p95). + - Memory ceiling <220 MB (tracked via deterministic benchmark harness). + - All symlink targets canonicalized; path traversal guarded. +- **Gate Artifacts:** + - `Fixtures/lang/node/**` golden outputs. + - Analyzer benchmark CSV + flamegraph (commit under `bench/Scanner.Analyzers`). + - Worker integration sample enabling Node analyzer via manifest. - **Progress (2025-10-21):** Module walker with package-lock/yarn/pnpm resolution, workspace attribution, integrity metadata, and deterministic fixture harness committed; Node tasks 10-302A/B remain green. Shared component mapper + canonical result harness landed, closing tasks 10-307/308. Script metadata & telemetry (10-302C) emit policy hints, hashed evidence, and feed `scanner_analyzer_node_scripts_total` into Worker OpenTelemetry pipeline. Restart-time packaging closed (10-309): manifest added, Worker language catalog loads the Node analyzer, integration tests cover dispatch + layer fragments, and Offline Kit docs call out bundled language plug-ins. - -## Sprint LA2 — Python Analyzer & Entry Point Attribution (Tasks 10-303, 10-307, 10-308, 10-309 subset) -- **Scope:** Parse `*.dist-info`, `RECORD` hashes, entry points, and pip-installed editable packages; integrate usage hints from EntryTrace. -- **Deliverables:** - - `StellaOps.Scanner.Analyzers.Lang.Python` plug-in. - - RECORD hash validation with optional Zip64 support for `.whl` caches. - - Entry-point mapping into `UsageFlags` for Emit stage. -- **Acceptance Metrics:** - - Hash verification throughput ≥75 MB/s sustained with streaming reader. - - False-positive rate for editable installs <1 % on curated fixtures. - - Determinism check across CPython 3.8–3.12 generated metadata. + +## Sprint LA2 — Python Analyzer & Entry Point Attribution (Tasks 10-303, 10-307, 10-308, 10-309 subset) +- **Scope:** Parse `*.dist-info`, `RECORD` hashes, entry points, and pip-installed editable packages; integrate usage hints from EntryTrace. +- **Deliverables:** + - `StellaOps.Scanner.Analyzers.Lang.Python` plug-in. + - RECORD hash validation with optional Zip64 support for `.whl` caches. + - Entry-point mapping into `UsageFlags` for Emit stage. +- **Acceptance Metrics:** + - Hash verification throughput ≥75 MB/s sustained with streaming reader. + - False-positive rate for editable installs <1 % on curated fixtures. + - Determinism check across CPython 3.8–3.12 generated metadata. - **Gate Artifacts:** - Golden fixtures for `site-packages`, virtualenv, and layered pip caches. - Usage hint propagation tests (EntryTrace → analyzer → SBOM). - Metrics counters (`scanner_analyzer_python_components_total`) documented. - **Progress (2025-10-21):** Python analyzer landed; Tasks 10-303A/B/C are DONE with dist-info parsing, RECORD verification, editable install detection, and deterministic `simple-venv` fixture + benchmark hooks recorded. - + ## Sprint LA3 — Go Analyzer & Build Info Synthesis (Tasks 10-304, 10-307, 10-308, 10-309 subset) - **Scope:** Extract Go build metadata from `.note.go.buildid`, embedded module info, and fallback to `bin:{sha256}`; surface VCS provenance. - **Deliverables:** @@ -51,67 +51,67 @@ All sprints below assume prerequisites from SP10-G2 (core scaffolding + Java ana - **Gate Artifacts:** - Benchmarks vs competitor open-source tool (Trivy or Syft) demonstrating faster metadata extraction. - Documentation snippet explaining VCS metadata fields for Policy team. -- **Progress (2025-10-22):** Build-info decoder shipped with DWARF-string fallback for `vcs.*` markers, plus cached metadata keyed by binary length/timestamp. Added Go test fixtures covering build-info and DWARF-only binaries with deterministic goldens; analyzer now emits `go.dwarf` evidence alongside `go.buildinfo` metadata to feed downstream provenance rules. Completed stripped-binary heuristics with deterministic `golang::bin::sha256` components and a new `stripped` fixture to guard quiet-provenance behaviour. - -## Sprint LA4 — .NET Analyzer & RID Variants (Tasks 10-305, 10-307, 10-308, 10-309 subset) -- **Scope:** Parse `*.deps.json`, `runtimeconfig.json`, assembly metadata, and RID-specific assets; correlate with native dependencies. -- **Deliverables:** - - `StellaOps.Scanner.Analyzers.Lang.DotNet` plug-in. - - Strong-name + Authenticode optional verification when offline cert bundle provided. - - RID-aware component grouping with fallback to `bin:{sha256}` for self-contained apps. -- **Acceptance Metrics:** - - Multi-target app fixture processed <1.2 s; memory <250 MB. - - RID variant collapse reduces component explosion by ≥40 % vs naive listing. - - All security metadata (signing Publisher, timestamp) surfaced deterministically. +- **Progress (2025-10-22):** Build-info decoder shipped with DWARF-string fallback for `vcs.*` markers, plus cached metadata keyed by binary length/timestamp. Added Go test fixtures covering build-info and DWARF-only binaries with deterministic goldens; analyzer now emits `go.dwarf` evidence alongside `go.buildinfo` metadata to feed downstream provenance rules. Completed stripped-binary heuristics with deterministic `golang::bin::sha256` components and a new `stripped` fixture to guard quiet-provenance behaviour. Heuristic fallbacks now emit `scanner_analyzer_golang_heuristic_total{indicator,version_hint}` counters, and shared buffer pooling (`ArrayPool`) keeps concurrent scans allocation-lite. Bench harness (`bench/Scanner.Analyzers/config.json`) gained a dedicated Go scenario with baseline mean 4.02 ms; comparison against Syft v1.29.1 on the same fixture shows a 22 % speed advantage (see `bench/Scanner.Analyzers/lang/go/syft-comparison-20251021.csv`). + +## Sprint LA4 — .NET Analyzer & RID Variants (Tasks 10-305, 10-307, 10-308, 10-309 subset) +- **Scope:** Parse `*.deps.json`, `runtimeconfig.json`, assembly metadata, and RID-specific assets; correlate with native dependencies. +- **Deliverables:** + - `StellaOps.Scanner.Analyzers.Lang.DotNet` plug-in. + - Strong-name + Authenticode optional verification when offline cert bundle provided. + - RID-aware component grouping with fallback to `bin:{sha256}` for self-contained apps. +- **Acceptance Metrics:** + - Multi-target app fixture processed <1.2 s; memory <250 MB. + - RID variant collapse reduces component explosion by ≥40 % vs naive listing. + - All security metadata (signing Publisher, timestamp) surfaced deterministically. - **Gate Artifacts:** - Signed .NET sample apps (framework-dependent & self-contained) under `samples/scanner/lang/dotnet/`. - Tests verifying dual runtimeconfig merge logic. - Guidance for Policy on license propagation from NuGet metadata. -- **Progress (2025-10-22):** Completed task 10-305A with a deterministic deps/runtimeconfig ingest pipeline producing `pkg:nuget` components across RID targets. Added dotnet fixture + golden output to the shared harness, wired analyzer plugin availability, and surfaced RID metadata in component records for downstream emit/diff work. - -## Sprint LA5 — Rust Analyzer & Binary Fingerprinting (Tasks 10-306, 10-307, 10-308, 10-309 subset) -- **Scope:** Detect crates via metadata in `.fingerprint`, Cargo.lock fragments, or embedded `rustc` markers; robust fallback to binary hash classification. -- **Deliverables:** - - `StellaOps.Scanner.Analyzers.Lang.Rust` plug-in. - - Symbol table heuristics capable of attributing stripped binaries by leveraging `.comment` and section names without violating determinism. - - Quiet-provenance flags to differentiate heuristics from hard evidence. -- **Acceptance Metrics:** - - Accurate crate attribution ≥85 % on curated Cargo workspace fixtures. - - Heuristic fallback clearly labeled; no false “certain” claims. - - Analyzer completes <1 s on 500 binary corpus. -- **Gate Artifacts:** - - Fixtures covering cargo workspaces, binaries with embedded metadata stripped. - - ADR documenting heuristic boundaries + risk mitigations. - -## Sprint LA6 — Shared Evidence Enhancements & Worker Integration (Tasks 10-307, 10-308, 10-309 finalization) -- **Scope:** Finalize shared helpers, deterministic harness expansion, Worker/Emit wiring, and macro benchmarks. -- **Deliverables:** - - Consolidated `LanguageComponentWriter` extensions for license, vulnerability hints, and usage propagation. - - Worker dispatcher loading plug-ins via manifest registry + health checks. - - Combined analyzer benchmark suite executed in CI with regression thresholds. -- **Acceptance Metrics:** - - Worker executes mixed analyzer suite (Java+Node+Python+Go+.NET+Rust) within SLA: warm scan <6 s, cold <25 s. - - CI determinism guard catches output drift (>0 diff tolerance) across all fixtures. - - Telemetry coverage: each analyzer emits timing + component counters. -- **Gate Artifacts:** - - `SPRINTS_LANG_IMPLEMENTATION_PLAN.md` progress log updated (this file). - - `bench/Scanner.Analyzers/lang-matrix.csv` recorded + referenced in docs. - - Ops notes for packaging plug-ins into Offline Kit. - ---- - -## Cross-Sprint Considerations -- **Security:** All analyzers must enforce path canonicalization, guard against zip-slip, and expose provenance classifications (`observed`, `heuristic`, `attested`). -- **Offline-first:** No network calls; rely on cached metadata and optional offline bundles (license texts, signature roots). -- **Determinism:** Normalise timestamps to `0001-01-01T00:00:00Z` when persisting synthetic data; sort collections by stable keys. -- **Benchmarking:** Extend `bench/Scanner.Analyzers` to compare against open-source scanners (Syft/Trivy) and document performance wins. -- **Hand-offs:** Emit guild requires consistent component schemas; Policy needs license + provenance metadata; Scheduler depends on usage flags for ImpactIndex. - -## Tracking & Reporting -- Update `TASKS.md` per sprint (TODO → DOING → DONE) with date stamps. -- Log sprint summaries in `docs/updates/` once each sprint lands. -- Use module-specific CI pipeline to run analyzer suites nightly (determinism + perf). - ---- - -**Next Action:** Start Sprint LA1 (Node Analyzer) — move tasks 10-302, 10-307, 10-308, 10-309 → DOING and spin up fixtures + benchmarks. +- **Progress (2025-10-22):** Completed task 10-305A with a deterministic deps/runtimeconfig ingest pipeline producing `pkg:nuget` components across RID targets. Added dotnet fixture + golden output to the shared harness, wired analyzer plugin availability, and surfaced RID metadata in component records for downstream emit/diff work. License provenance and quiet flagging now ride through the shared helpers (task 10-307D), including nuspec license expression/file ingestion, manifest provenance tagging, and concurrency-safe file metadata caching with new parallel tests. + +## Sprint LA5 — Rust Analyzer & Binary Fingerprinting (Tasks 10-306, 10-307, 10-308, 10-309 subset) +- **Scope:** Detect crates via metadata in `.fingerprint`, Cargo.lock fragments, or embedded `rustc` markers; robust fallback to binary hash classification. +- **Deliverables:** + - `StellaOps.Scanner.Analyzers.Lang.Rust` plug-in. + - Symbol table heuristics capable of attributing stripped binaries by leveraging `.comment` and section names without violating determinism. + - Quiet-provenance flags to differentiate heuristics from hard evidence. +- **Acceptance Metrics:** + - Accurate crate attribution ≥85 % on curated Cargo workspace fixtures. + - Heuristic fallback clearly labeled; no false “certain” claims. + - Analyzer completes <1 s on 500 binary corpus. +- **Gate Artifacts:** + - Fixtures covering cargo workspaces, binaries with embedded metadata stripped. + - ADR documenting heuristic boundaries + risk mitigations. + +## Sprint LA6 — Shared Evidence Enhancements & Worker Integration (Tasks 10-307, 10-308, 10-309 finalization) +- **Scope:** Finalize shared helpers, deterministic harness expansion, Worker/Emit wiring, and macro benchmarks. +- **Deliverables:** + - Consolidated `LanguageComponentWriter` extensions for license, vulnerability hints, and usage propagation. + - Worker dispatcher loading plug-ins via manifest registry + health checks. + - Combined analyzer benchmark suite executed in CI with regression thresholds. +- **Acceptance Metrics:** + - Worker executes mixed analyzer suite (Java+Node+Python+Go+.NET+Rust) within SLA: warm scan <6 s, cold <25 s. + - CI determinism guard catches output drift (>0 diff tolerance) across all fixtures. + - Telemetry coverage: each analyzer emits timing + component counters. +- **Gate Artifacts:** + - `SPRINTS_LANG_IMPLEMENTATION_PLAN.md` progress log updated (this file). + - `bench/Scanner.Analyzers/lang-matrix.csv` recorded + referenced in docs. + - Ops notes for packaging plug-ins into Offline Kit. + +--- + +## Cross-Sprint Considerations +- **Security:** All analyzers must enforce path canonicalization, guard against zip-slip, and expose provenance classifications (`observed`, `heuristic`, `attested`). +- **Offline-first:** No network calls; rely on cached metadata and optional offline bundles (license texts, signature roots). +- **Determinism:** Normalise timestamps to `0001-01-01T00:00:00Z` when persisting synthetic data; sort collections by stable keys. +- **Benchmarking:** Extend `bench/Scanner.Analyzers` to compare against open-source scanners (Syft/Trivy) and document performance wins. +- **Hand-offs:** Emit guild requires consistent component schemas; Policy needs license + provenance metadata; Scheduler depends on usage flags for ImpactIndex. + +## Tracking & Reporting +- Update `TASKS.md` per sprint (TODO → DOING → DONE) with date stamps. +- Log sprint summaries in `docs/updates/` once each sprint lands. +- Use module-specific CI pipeline to run analyzer suites nightly (determinism + perf). + +--- + +**Next Action:** Start Sprint LA1 (Node Analyzer) — move tasks 10-302, 10-307, 10-308, 10-309 → DOING and spin up fixtures + benchmarks.