Rename Concelier Source modules to Connector
This commit is contained in:
29
src/StellaOps.Concelier.Connector.Jvn/AGENTS.md
Normal file
29
src/StellaOps.Concelier.Connector.Jvn/AGENTS.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# AGENTS
|
||||
## Role
|
||||
Japan JVN/MyJVN connector; national CERT enrichment with strong identifiers (JVNDB) and vendor status; authoritative only where concrete package evidence exists; otherwise enriches text, severity, references, and aliases.
|
||||
## Scope
|
||||
- Fetch JVNRSS (overview) and VULDEF (detail) via MyJVN API; window by dateFirstPublished/dateLastUpdated; paginate; respect rate limits.
|
||||
- Validate XML or JSON payloads; normalize titles, CVEs, JVNDB ids, vendor status, categories; map references and severity text; attach jp_flags.
|
||||
- Persist raw docs with sha256 and headers; manage source_state cursor; idempotent parse/map.
|
||||
## Participants
|
||||
- Source.Common (HTTP, pagination, XML or XSD validators, retries/backoff).
|
||||
- Storage.Mongo (document, dto, advisory, alias, affected (when concrete), reference, jp_flags, source_state).
|
||||
- Models (canonical Advisory/Affected/Provenance).
|
||||
- Core/WebService (jobs: source:jvn:fetch|parse|map).
|
||||
- Merge engine applies enrichment precedence (does not override distro or PSIRT ranges unless JVN gives explicit package truth).
|
||||
## Interfaces & contracts
|
||||
- Aliases include JVNDB-YYYY-NNNNN and CVE ids; scheme "JVNDB".
|
||||
- jp_flags: { jvndb_id, jvn_category, vendor_status }.
|
||||
- References typed: advisory/vendor/bulletin; URLs normalized and deduped.
|
||||
- Affected only when VULDEF gives concrete coordinates; otherwise omit.
|
||||
- Provenance: method=parser; kind=api; value=endpoint plus query window; recordedAt=fetched time.
|
||||
## In/Out of scope
|
||||
In: JVN/MyJVN ingestion, aliases, jp_flags, enrichment mapping, watermarking.
|
||||
Out: overriding distro or PSIRT ranges without concrete evidence; scraping unofficial mirrors.
|
||||
## Observability & security expectations
|
||||
- Metrics: SourceDiagnostics emits `concelier.source.http.*` counters/histograms tagged `concelier.source=jvn`, enabling dashboards to track fetch requests, item counts, parse failures, and enrichment/map activity (including jp_flags) via tag filters.
|
||||
- Logs: window bounds, jvndb ids processed, vendor_status distribution; redact API keys.
|
||||
## Tests
|
||||
- Author and review coverage in `../StellaOps.Concelier.Connector.Jvn.Tests`.
|
||||
- Shared fixtures (e.g., `MongoIntegrationFixture`, `ConnectorTestHarness`) live in `../StellaOps.Concelier.Testing`.
|
||||
- Keep fixtures deterministic; match new cases to real-world advisories or regression scenarios.
|
||||
@@ -0,0 +1,80 @@
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Configuration;
|
||||
|
||||
/// <summary>
|
||||
/// Options controlling the JVN connector fetch cadence and HTTP client configuration.
|
||||
/// </summary>
|
||||
public sealed class JvnOptions
|
||||
{
|
||||
public static string HttpClientName => "source.jvn";
|
||||
|
||||
/// <summary>
|
||||
/// Base endpoint for the MyJVN API.
|
||||
/// </summary>
|
||||
public Uri BaseEndpoint { get; set; } = new("https://jvndb.jvn.jp/myjvn", UriKind.Absolute);
|
||||
|
||||
/// <summary>
|
||||
/// Size of each fetch window applied to dateFirstPublished/dateLastUpdated queries.
|
||||
/// </summary>
|
||||
public TimeSpan WindowSize { get; set; } = TimeSpan.FromDays(7);
|
||||
|
||||
/// <summary>
|
||||
/// Overlap applied between consecutive windows to ensure late-arriving updates are captured.
|
||||
/// </summary>
|
||||
public TimeSpan WindowOverlap { get; set; } = TimeSpan.FromDays(1);
|
||||
|
||||
/// <summary>
|
||||
/// Number of overview records requested per page (MyJVN max is 50).
|
||||
/// </summary>
|
||||
public int PageSize { get; set; } = 50;
|
||||
|
||||
/// <summary>
|
||||
/// Optional delay enforced between HTTP requests to respect service rate limits.
|
||||
/// </summary>
|
||||
public TimeSpan RequestDelay { get; set; } = TimeSpan.FromMilliseconds(500);
|
||||
|
||||
/// <summary>
|
||||
/// Maximum number of overview pages the connector will request in a single fetch cycle.
|
||||
/// </summary>
|
||||
public int MaxOverviewPagesPerFetch { get; set; } = 20;
|
||||
|
||||
[MemberNotNull(nameof(BaseEndpoint))]
|
||||
public void Validate()
|
||||
{
|
||||
if (BaseEndpoint is null || !BaseEndpoint.IsAbsoluteUri)
|
||||
{
|
||||
throw new InvalidOperationException("JVN options require an absolute BaseEndpoint.");
|
||||
}
|
||||
|
||||
if (WindowSize <= TimeSpan.Zero)
|
||||
{
|
||||
throw new InvalidOperationException("WindowSize must be greater than zero.");
|
||||
}
|
||||
|
||||
if (WindowOverlap < TimeSpan.Zero)
|
||||
{
|
||||
throw new InvalidOperationException("WindowOverlap cannot be negative.");
|
||||
}
|
||||
|
||||
if (WindowOverlap >= WindowSize)
|
||||
{
|
||||
throw new InvalidOperationException("WindowOverlap must be smaller than WindowSize.");
|
||||
}
|
||||
|
||||
if (PageSize is < 1 or > 50)
|
||||
{
|
||||
throw new InvalidOperationException("PageSize must be between 1 and 50 to satisfy MyJVN limits.");
|
||||
}
|
||||
|
||||
if (RequestDelay < TimeSpan.Zero)
|
||||
{
|
||||
throw new InvalidOperationException("RequestDelay cannot be negative.");
|
||||
}
|
||||
|
||||
if (MaxOverviewPagesPerFetch <= 0)
|
||||
{
|
||||
throw new InvalidOperationException("MaxOverviewPagesPerFetch must be positive.");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,418 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using StellaOps.Concelier.Models;
|
||||
using StellaOps.Concelier.Connector.Common;
|
||||
using StellaOps.Concelier.Normalization.Cvss;
|
||||
using StellaOps.Concelier.Normalization.Identifiers;
|
||||
using StellaOps.Concelier.Normalization.Text;
|
||||
using StellaOps.Concelier.Storage.Mongo.Documents;
|
||||
using StellaOps.Concelier.Storage.Mongo.Dtos;
|
||||
using StellaOps.Concelier.Storage.Mongo.JpFlags;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal static class JvnAdvisoryMapper
|
||||
{
|
||||
private static readonly string[] SeverityOrder = { "none", "low", "medium", "high", "critical" };
|
||||
|
||||
public static (Advisory Advisory, JpFlagRecord Flag) Map(
|
||||
JvnDetailDto detail,
|
||||
DocumentRecord document,
|
||||
DtoRecord dtoRecord,
|
||||
TimeProvider timeProvider)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(detail);
|
||||
ArgumentNullException.ThrowIfNull(document);
|
||||
ArgumentNullException.ThrowIfNull(dtoRecord);
|
||||
ArgumentNullException.ThrowIfNull(timeProvider);
|
||||
|
||||
var recordedAt = dtoRecord.ValidatedAt;
|
||||
var fetchProvenance = new AdvisoryProvenance(JvnConnectorPlugin.SourceName, "document", document.Uri, document.FetchedAt);
|
||||
var mappingProvenance = new AdvisoryProvenance(JvnConnectorPlugin.SourceName, "mapping", detail.VulnerabilityId, recordedAt);
|
||||
|
||||
var aliases = BuildAliases(detail);
|
||||
var references = BuildReferences(detail, recordedAt);
|
||||
var affectedPackages = BuildAffected(detail, recordedAt);
|
||||
var cvssMetrics = BuildCvss(detail, recordedAt, out var severity);
|
||||
|
||||
var description = DescriptionNormalizer.Normalize(new[]
|
||||
{
|
||||
new LocalizedText(detail.Overview, detail.Language),
|
||||
});
|
||||
|
||||
var language = description.Language;
|
||||
var summary = string.IsNullOrEmpty(description.Text) ? null : description.Text;
|
||||
|
||||
var provenance = new[] { fetchProvenance, mappingProvenance };
|
||||
|
||||
var advisory = new Advisory(
|
||||
detail.VulnerabilityId,
|
||||
detail.Title,
|
||||
summary,
|
||||
language,
|
||||
detail.DateFirstPublished,
|
||||
detail.DateLastUpdated,
|
||||
severity,
|
||||
exploitKnown: false,
|
||||
aliases,
|
||||
references,
|
||||
affectedPackages,
|
||||
cvssMetrics,
|
||||
provenance);
|
||||
|
||||
var vendorStatus = detail.VendorStatuses.Length == 0
|
||||
? null
|
||||
: string.Join(",", detail.VendorStatuses.OrderBy(static status => status, StringComparer.Ordinal));
|
||||
|
||||
var flag = new JpFlagRecord(
|
||||
detail.VulnerabilityId,
|
||||
JvnConnectorPlugin.SourceName,
|
||||
detail.JvnCategory,
|
||||
vendorStatus,
|
||||
timeProvider.GetUtcNow());
|
||||
|
||||
return (advisory, flag);
|
||||
}
|
||||
|
||||
private static IEnumerable<string> BuildAliases(JvnDetailDto detail)
|
||||
{
|
||||
var aliases = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
detail.VulnerabilityId,
|
||||
};
|
||||
|
||||
foreach (var cve in detail.CveIds)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(cve))
|
||||
{
|
||||
aliases.Add(cve);
|
||||
}
|
||||
}
|
||||
|
||||
return aliases;
|
||||
}
|
||||
|
||||
private static IEnumerable<AdvisoryReference> BuildReferences(JvnDetailDto detail, DateTimeOffset recordedAt)
|
||||
{
|
||||
var references = new List<AdvisoryReference>();
|
||||
|
||||
foreach (var reference in detail.References)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(reference.Url))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
string? kind = reference.Type?.ToLowerInvariant() switch
|
||||
{
|
||||
"vendor" => "vendor",
|
||||
"advisory" => "advisory",
|
||||
"cwe" => "weakness",
|
||||
_ => null,
|
||||
};
|
||||
|
||||
string? sourceTag = !string.IsNullOrWhiteSpace(reference.Id) ? reference.Id : reference.Type;
|
||||
string? summary = reference.Name;
|
||||
|
||||
try
|
||||
{
|
||||
references.Add(new AdvisoryReference(
|
||||
reference.Url,
|
||||
kind,
|
||||
sourceTag,
|
||||
summary,
|
||||
new AdvisoryProvenance(JvnConnectorPlugin.SourceName, "reference", reference.Url, recordedAt)));
|
||||
}
|
||||
catch (ArgumentException)
|
||||
{
|
||||
// Ignore malformed URLs that slipped through validation.
|
||||
}
|
||||
}
|
||||
|
||||
if (references.Count == 0)
|
||||
{
|
||||
return references;
|
||||
}
|
||||
|
||||
var map = new Dictionary<string, AdvisoryReference>(StringComparer.OrdinalIgnoreCase);
|
||||
foreach (var reference in references)
|
||||
{
|
||||
if (!map.TryGetValue(reference.Url, out var existing))
|
||||
{
|
||||
map[reference.Url] = reference;
|
||||
continue;
|
||||
}
|
||||
|
||||
map[reference.Url] = MergeReferences(existing, reference);
|
||||
}
|
||||
|
||||
var deduped = map.Values.ToList();
|
||||
deduped.Sort(CompareReferences);
|
||||
return deduped;
|
||||
}
|
||||
|
||||
private static IEnumerable<AffectedPackage> BuildAffected(JvnDetailDto detail, DateTimeOffset recordedAt)
|
||||
{
|
||||
var packages = new List<AffectedPackage>();
|
||||
|
||||
foreach (var product in detail.Affected)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(product.Cpe))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(product.Status) && !product.Status.StartsWith("vulnerable", StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!IdentifierNormalizer.TryNormalizeCpe(product.Cpe, out var cpe))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var provenance = new List<AdvisoryProvenance>
|
||||
{
|
||||
new AdvisoryProvenance(JvnConnectorPlugin.SourceName, "affected", cpe!, recordedAt),
|
||||
};
|
||||
|
||||
var attributeParts = new List<string>(capacity: 2);
|
||||
if (!string.IsNullOrWhiteSpace(product.CpeVendor))
|
||||
{
|
||||
attributeParts.Add($"vendor={product.CpeVendor}");
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(product.CpeProduct))
|
||||
{
|
||||
attributeParts.Add($"product={product.CpeProduct}");
|
||||
}
|
||||
|
||||
if (attributeParts.Count > 0)
|
||||
{
|
||||
provenance.Add(new AdvisoryProvenance(
|
||||
JvnConnectorPlugin.SourceName,
|
||||
"cpe-attributes",
|
||||
string.Join(";", attributeParts),
|
||||
recordedAt));
|
||||
}
|
||||
|
||||
var platform = product.Vendor ?? product.CpeVendor;
|
||||
|
||||
var versionRanges = BuildVersionRanges(product, recordedAt, provenance[0]);
|
||||
|
||||
packages.Add(new AffectedPackage(
|
||||
AffectedPackageTypes.Cpe,
|
||||
cpe!,
|
||||
platform: platform,
|
||||
versionRanges: versionRanges,
|
||||
statuses: Array.Empty<AffectedPackageStatus>(),
|
||||
provenance: provenance.ToArray()));
|
||||
}
|
||||
|
||||
return packages;
|
||||
}
|
||||
|
||||
private static IReadOnlyList<AffectedVersionRange> BuildVersionRanges(JvnAffectedProductDto product, DateTimeOffset recordedAt, AdvisoryProvenance provenance)
|
||||
{
|
||||
var extensions = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
|
||||
if (!string.IsNullOrWhiteSpace(product.Version))
|
||||
{
|
||||
extensions["jvn.version"] = product.Version!;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(product.Build))
|
||||
{
|
||||
extensions["jvn.build"] = product.Build!;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(product.Description))
|
||||
{
|
||||
extensions["jvn.description"] = product.Description!;
|
||||
}
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(product.Status))
|
||||
{
|
||||
extensions["jvn.status"] = product.Status!;
|
||||
}
|
||||
|
||||
if (extensions.Count == 0)
|
||||
{
|
||||
return Array.Empty<AffectedVersionRange>();
|
||||
}
|
||||
|
||||
var primitives = new RangePrimitives(
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
extensions);
|
||||
|
||||
var expression = product.Version;
|
||||
var range = new AffectedVersionRange(
|
||||
rangeKind: "cpe",
|
||||
introducedVersion: null,
|
||||
fixedVersion: null,
|
||||
lastAffectedVersion: null,
|
||||
rangeExpression: string.IsNullOrWhiteSpace(expression) ? null : expression,
|
||||
provenance: provenance,
|
||||
primitives: primitives);
|
||||
|
||||
return new[] { range };
|
||||
}
|
||||
|
||||
private static IReadOnlyList<CvssMetric> BuildCvss(JvnDetailDto detail, DateTimeOffset recordedAt, out string? severity)
|
||||
{
|
||||
var metrics = new List<CvssMetric>();
|
||||
severity = null;
|
||||
var bestRank = -1;
|
||||
|
||||
foreach (var cvss in detail.Cvss)
|
||||
{
|
||||
if (!CvssMetricNormalizer.TryNormalize(cvss.Version, cvss.Vector, cvss.Score, cvss.Severity, out var normalized))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var provenance = new AdvisoryProvenance(JvnConnectorPlugin.SourceName, "cvss", cvss.Type, recordedAt);
|
||||
metrics.Add(normalized.ToModel(provenance));
|
||||
|
||||
var rank = Array.IndexOf(SeverityOrder, normalized.BaseSeverity);
|
||||
if (rank > bestRank)
|
||||
{
|
||||
bestRank = rank;
|
||||
severity = normalized.BaseSeverity;
|
||||
}
|
||||
}
|
||||
|
||||
return metrics;
|
||||
}
|
||||
|
||||
private static int CompareReferences(AdvisoryReference? left, AdvisoryReference? right)
|
||||
{
|
||||
if (ReferenceEquals(left, right))
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (left is null)
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (right is null)
|
||||
{
|
||||
return -1;
|
||||
}
|
||||
|
||||
var compare = StringComparer.OrdinalIgnoreCase.Compare(left.Url, right.Url);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = CompareNullable(left.Kind, right.Kind);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = CompareNullable(left.SourceTag, right.SourceTag);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = CompareNullable(left.Summary, right.Summary);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = StringComparer.Ordinal.Compare(left.Provenance.Source, right.Provenance.Source);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = StringComparer.Ordinal.Compare(left.Provenance.Kind, right.Provenance.Kind);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
compare = CompareNullable(left.Provenance.Value, right.Provenance.Value);
|
||||
if (compare != 0)
|
||||
{
|
||||
return compare;
|
||||
}
|
||||
|
||||
return left.Provenance.RecordedAt.CompareTo(right.Provenance.RecordedAt);
|
||||
}
|
||||
|
||||
private static int CompareNullable(string? left, string? right)
|
||||
{
|
||||
if (left is null && right is null)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (left is null)
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
if (right is null)
|
||||
{
|
||||
return -1;
|
||||
}
|
||||
|
||||
return StringComparer.Ordinal.Compare(left, right);
|
||||
}
|
||||
|
||||
private static AdvisoryReference MergeReferences(AdvisoryReference existing, AdvisoryReference candidate)
|
||||
{
|
||||
var kind = existing.Kind ?? candidate.Kind;
|
||||
var sourceTag = existing.SourceTag ?? candidate.SourceTag;
|
||||
var summary = ChoosePreferredSummary(existing.Summary, candidate.Summary);
|
||||
var provenance = existing.Provenance.RecordedAt <= candidate.Provenance.RecordedAt
|
||||
? existing.Provenance
|
||||
: candidate.Provenance;
|
||||
|
||||
if (kind == existing.Kind
|
||||
&& sourceTag == existing.SourceTag
|
||||
&& summary == existing.Summary
|
||||
&& provenance == existing.Provenance)
|
||||
{
|
||||
return existing;
|
||||
}
|
||||
|
||||
if (kind == candidate.Kind
|
||||
&& sourceTag == candidate.SourceTag
|
||||
&& summary == candidate.Summary
|
||||
&& provenance == candidate.Provenance)
|
||||
{
|
||||
return candidate;
|
||||
}
|
||||
|
||||
return new AdvisoryReference(existing.Url, kind, sourceTag, summary, provenance);
|
||||
}
|
||||
|
||||
private static string? ChoosePreferredSummary(string? left, string? right)
|
||||
{
|
||||
var leftValue = string.IsNullOrWhiteSpace(left) ? null : left;
|
||||
var rightValue = string.IsNullOrWhiteSpace(right) ? null : right;
|
||||
|
||||
if (leftValue is null)
|
||||
{
|
||||
return rightValue;
|
||||
}
|
||||
|
||||
if (rightValue is null)
|
||||
{
|
||||
return leftValue;
|
||||
}
|
||||
|
||||
return leftValue.Length >= rightValue.Length ? leftValue : rightValue;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal static class JvnConstants
|
||||
{
|
||||
public const string DtoSchemaVersion = "jvn.vuldef.3.2";
|
||||
|
||||
public const string VuldefNamespace = "http://jvn.jp/vuldef/";
|
||||
public const string StatusNamespace = "http://jvndb.jvn.jp/myjvn/Status";
|
||||
public const string ModSecNamespace = "http://jvn.jp/rss/mod_sec/3.0/";
|
||||
}
|
||||
106
src/StellaOps.Concelier.Connector.Jvn/Internal/JvnCursor.cs
Normal file
106
src/StellaOps.Concelier.Connector.Jvn/Internal/JvnCursor.cs
Normal file
@@ -0,0 +1,106 @@
|
||||
using System.Linq;
|
||||
using MongoDB.Bson;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal sealed record JvnCursor(
|
||||
DateTimeOffset? WindowStart,
|
||||
DateTimeOffset? WindowEnd,
|
||||
DateTimeOffset? LastCompletedWindowEnd,
|
||||
IReadOnlyCollection<Guid> PendingDocuments,
|
||||
IReadOnlyCollection<Guid> PendingMappings)
|
||||
{
|
||||
public static JvnCursor Empty { get; } = new(null, null, null, Array.Empty<Guid>(), Array.Empty<Guid>());
|
||||
|
||||
public BsonDocument ToBsonDocument()
|
||||
{
|
||||
var document = new BsonDocument();
|
||||
|
||||
if (WindowStart.HasValue)
|
||||
{
|
||||
document["windowStart"] = WindowStart.Value.UtcDateTime;
|
||||
}
|
||||
|
||||
if (WindowEnd.HasValue)
|
||||
{
|
||||
document["windowEnd"] = WindowEnd.Value.UtcDateTime;
|
||||
}
|
||||
|
||||
if (LastCompletedWindowEnd.HasValue)
|
||||
{
|
||||
document["lastCompletedWindowEnd"] = LastCompletedWindowEnd.Value.UtcDateTime;
|
||||
}
|
||||
|
||||
document["pendingDocuments"] = new BsonArray(PendingDocuments.Select(static id => id.ToString()));
|
||||
document["pendingMappings"] = new BsonArray(PendingMappings.Select(static id => id.ToString()));
|
||||
return document;
|
||||
}
|
||||
|
||||
public static JvnCursor FromBson(BsonDocument? document)
|
||||
{
|
||||
if (document is null || document.ElementCount == 0)
|
||||
{
|
||||
return Empty;
|
||||
}
|
||||
|
||||
DateTimeOffset? windowStart = TryGetDateTime(document, "windowStart");
|
||||
DateTimeOffset? windowEnd = TryGetDateTime(document, "windowEnd");
|
||||
DateTimeOffset? lastCompleted = TryGetDateTime(document, "lastCompletedWindowEnd");
|
||||
|
||||
var pendingDocuments = ReadGuidArray(document, "pendingDocuments");
|
||||
var pendingMappings = ReadGuidArray(document, "pendingMappings");
|
||||
|
||||
return new JvnCursor(windowStart, windowEnd, lastCompleted, pendingDocuments, pendingMappings);
|
||||
}
|
||||
|
||||
public JvnCursor WithWindow(DateTimeOffset start, DateTimeOffset end)
|
||||
=> this with { WindowStart = start, WindowEnd = end };
|
||||
|
||||
public JvnCursor WithCompletedWindow(DateTimeOffset end)
|
||||
=> this with { LastCompletedWindowEnd = end };
|
||||
|
||||
public JvnCursor WithPendingDocuments(IEnumerable<Guid> pending)
|
||||
=> this with { PendingDocuments = pending?.Distinct().ToArray() ?? Array.Empty<Guid>() };
|
||||
|
||||
public JvnCursor WithPendingMappings(IEnumerable<Guid> pending)
|
||||
=> this with { PendingMappings = pending?.Distinct().ToArray() ?? Array.Empty<Guid>() };
|
||||
|
||||
private static DateTimeOffset? TryGetDateTime(BsonDocument document, string field)
|
||||
{
|
||||
if (!document.TryGetValue(field, out var value))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return value.BsonType switch
|
||||
{
|
||||
BsonType.DateTime => DateTime.SpecifyKind(value.ToUniversalTime(), DateTimeKind.Utc),
|
||||
BsonType.String when DateTimeOffset.TryParse(value.AsString, out var parsed) => parsed.ToUniversalTime(),
|
||||
_ => null,
|
||||
};
|
||||
}
|
||||
|
||||
private static IReadOnlyCollection<Guid> ReadGuidArray(BsonDocument document, string field)
|
||||
{
|
||||
if (!document.TryGetValue(field, out var value) || value is not BsonArray array)
|
||||
{
|
||||
return Array.Empty<Guid>();
|
||||
}
|
||||
|
||||
var results = new List<Guid>(array.Count);
|
||||
foreach (var element in array)
|
||||
{
|
||||
if (element is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (element.BsonType == BsonType.String && Guid.TryParse(element.AsString, out var guid))
|
||||
{
|
||||
results.Add(guid);
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
using System.Collections.Immutable;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal sealed record JvnDetailDto(
|
||||
string VulnerabilityId,
|
||||
string Title,
|
||||
string? Overview,
|
||||
string? Language,
|
||||
DateTimeOffset? DateFirstPublished,
|
||||
DateTimeOffset? DateLastUpdated,
|
||||
DateTimeOffset? DatePublic,
|
||||
ImmutableArray<JvnCvssDto> Cvss,
|
||||
ImmutableArray<JvnAffectedProductDto> Affected,
|
||||
ImmutableArray<JvnReferenceDto> References,
|
||||
ImmutableArray<JvnHistoryEntryDto> History,
|
||||
ImmutableArray<string> CweIds,
|
||||
ImmutableArray<string> CveIds,
|
||||
string? AdvisoryUrl,
|
||||
string? JvnCategory,
|
||||
ImmutableArray<string> VendorStatuses)
|
||||
{
|
||||
public static JvnDetailDto Empty { get; } = new(
|
||||
"unknown",
|
||||
"unknown",
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
null,
|
||||
ImmutableArray<JvnCvssDto>.Empty,
|
||||
ImmutableArray<JvnAffectedProductDto>.Empty,
|
||||
ImmutableArray<JvnReferenceDto>.Empty,
|
||||
ImmutableArray<JvnHistoryEntryDto>.Empty,
|
||||
ImmutableArray<string>.Empty,
|
||||
ImmutableArray<string>.Empty,
|
||||
null,
|
||||
null,
|
||||
ImmutableArray<string>.Empty);
|
||||
}
|
||||
|
||||
internal sealed record JvnCvssDto(
|
||||
string Version,
|
||||
string Type,
|
||||
string Severity,
|
||||
double Score,
|
||||
string? Vector);
|
||||
|
||||
internal sealed record JvnAffectedProductDto(
|
||||
string? Vendor,
|
||||
string? Product,
|
||||
string? Cpe,
|
||||
string? CpeVendor,
|
||||
string? CpeProduct,
|
||||
string? Version,
|
||||
string? Build,
|
||||
string? Description,
|
||||
string? Status);
|
||||
|
||||
internal sealed record JvnReferenceDto(
|
||||
string Type,
|
||||
string Id,
|
||||
string? Name,
|
||||
string Url);
|
||||
|
||||
internal sealed record JvnHistoryEntryDto(
|
||||
string? Number,
|
||||
DateTimeOffset? Timestamp,
|
||||
string? Description);
|
||||
@@ -0,0 +1,268 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Collections.Immutable;
|
||||
using System.Globalization;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Xml;
|
||||
using System.Xml.Linq;
|
||||
using System.Xml.Schema;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal static class JvnDetailParser
|
||||
{
|
||||
private static readonly XNamespace Vuldef = JvnConstants.VuldefNamespace;
|
||||
private static readonly XNamespace Status = JvnConstants.StatusNamespace;
|
||||
|
||||
public static JvnDetailDto Parse(byte[] payload, string? documentUri)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(payload);
|
||||
|
||||
using var stream = new MemoryStream(payload, writable: false);
|
||||
var settings = new XmlReaderSettings
|
||||
{
|
||||
DtdProcessing = DtdProcessing.Prohibit,
|
||||
IgnoreComments = true,
|
||||
IgnoreProcessingInstructions = true,
|
||||
IgnoreWhitespace = true,
|
||||
};
|
||||
|
||||
using var reader = XmlReader.Create(stream, settings);
|
||||
var document = XDocument.Load(reader, LoadOptions.None);
|
||||
Validate(document, documentUri);
|
||||
return Extract(document, documentUri);
|
||||
}
|
||||
|
||||
private static void Validate(XDocument document, string? documentUri)
|
||||
{
|
||||
void Handler(object? sender, ValidationEventArgs args)
|
||||
{
|
||||
throw new JvnSchemaValidationException(
|
||||
$"JVN schema validation failed for {documentUri ?? "<unknown>"}: {args.Message}",
|
||||
args.Exception ?? new XmlSchemaValidationException(args.Message));
|
||||
}
|
||||
|
||||
document.Validate(JvnSchemaProvider.SchemaSet, Handler, addSchemaInfo: true);
|
||||
}
|
||||
|
||||
private static JvnDetailDto Extract(XDocument document, string? documentUri)
|
||||
{
|
||||
var root = document.Root ?? throw new InvalidOperationException("JVN VULDEF document missing root element.");
|
||||
|
||||
var vulinfo = root.Element(Vuldef + "Vulinfo") ?? throw new InvalidOperationException("Vulinfo element missing.");
|
||||
var vulinfoId = Clean(vulinfo.Element(Vuldef + "VulinfoID")?.Value)
|
||||
?? throw new InvalidOperationException("VulinfoID element missing.");
|
||||
|
||||
var data = vulinfo.Element(Vuldef + "VulinfoData") ?? throw new InvalidOperationException("VulinfoData element missing.");
|
||||
var title = Clean(data.Element(Vuldef + "Title")?.Value) ?? vulinfoId;
|
||||
var overview = Clean(data.Element(Vuldef + "VulinfoDescription")?.Element(Vuldef + "Overview")?.Value);
|
||||
|
||||
var dateFirstPublished = ParseDate(data.Element(Vuldef + "DateFirstPublished")?.Value);
|
||||
var dateLastUpdated = ParseDate(data.Element(Vuldef + "DateLastUpdated")?.Value);
|
||||
var datePublic = ParseDate(data.Element(Vuldef + "DatePublic")?.Value);
|
||||
|
||||
var cvssEntries = ParseCvss(data.Element(Vuldef + "Impact"));
|
||||
var affected = ParseAffected(data.Element(Vuldef + "Affected"));
|
||||
var references = ParseReferences(data.Element(Vuldef + "Related"));
|
||||
var history = ParseHistory(data.Element(Vuldef + "History"));
|
||||
|
||||
var cweIds = references.Where(r => string.Equals(r.Type, "cwe", StringComparison.OrdinalIgnoreCase))
|
||||
.Select(r => r.Id)
|
||||
.Where(static id => !string.IsNullOrWhiteSpace(id))
|
||||
.Distinct(StringComparer.OrdinalIgnoreCase)
|
||||
.Select(static id => id!)
|
||||
.ToImmutableArray();
|
||||
|
||||
var cveIds = references.Where(r => string.Equals(r.Type, "advisory", StringComparison.OrdinalIgnoreCase)
|
||||
&& !string.IsNullOrWhiteSpace(r.Id)
|
||||
&& r.Id.StartsWith("CVE-", StringComparison.OrdinalIgnoreCase))
|
||||
.Select(r => r.Id)
|
||||
.Where(static id => !string.IsNullOrWhiteSpace(id))
|
||||
.Distinct(StringComparer.OrdinalIgnoreCase)
|
||||
.Select(static id => id!)
|
||||
.ToImmutableArray();
|
||||
|
||||
var language = Clean(root.Attribute(XNamespace.Xml + "lang")?.Value);
|
||||
|
||||
var statusElement = root.Element(Status + "Status");
|
||||
var jvnCategory = Clean(statusElement?.Attribute("category")?.Value);
|
||||
|
||||
var vendorStatuses = affected
|
||||
.Select(a => a.Status)
|
||||
.Where(static status => !string.IsNullOrWhiteSpace(status))
|
||||
.Select(static status => status!.ToLowerInvariant())
|
||||
.Distinct(StringComparer.Ordinal)
|
||||
.ToImmutableArray();
|
||||
|
||||
return new JvnDetailDto(
|
||||
vulinfoId,
|
||||
title,
|
||||
overview,
|
||||
language,
|
||||
dateFirstPublished,
|
||||
dateLastUpdated,
|
||||
datePublic,
|
||||
cvssEntries,
|
||||
affected,
|
||||
references,
|
||||
history,
|
||||
cweIds,
|
||||
cveIds,
|
||||
Clean(documentUri),
|
||||
jvnCategory,
|
||||
vendorStatuses);
|
||||
}
|
||||
|
||||
private static ImmutableArray<JvnCvssDto> ParseCvss(XElement? impactElement)
|
||||
{
|
||||
if (impactElement is null)
|
||||
{
|
||||
return ImmutableArray<JvnCvssDto>.Empty;
|
||||
}
|
||||
|
||||
var results = new List<JvnCvssDto>();
|
||||
foreach (var cvssElement in impactElement.Elements(Vuldef + "Cvss"))
|
||||
{
|
||||
var version = Clean(cvssElement.Attribute("version")?.Value) ?? "";
|
||||
var severityElement = cvssElement.Element(Vuldef + "Severity");
|
||||
var severity = Clean(severityElement?.Value) ?? Clean(cvssElement.Attribute("severity")?.Value) ?? string.Empty;
|
||||
var type = Clean(severityElement?.Attribute("type")?.Value) ?? Clean(cvssElement.Attribute("type")?.Value) ?? "base";
|
||||
var scoreText = Clean(cvssElement.Element(Vuldef + "Base")?.Value)
|
||||
?? Clean(cvssElement.Attribute("score")?.Value)
|
||||
?? "0";
|
||||
if (!double.TryParse(scoreText, NumberStyles.Float, CultureInfo.InvariantCulture, out var score))
|
||||
{
|
||||
score = 0d;
|
||||
}
|
||||
|
||||
var vector = Clean(cvssElement.Element(Vuldef + "Vector")?.Value)
|
||||
?? Clean(cvssElement.Attribute("vector")?.Value);
|
||||
|
||||
results.Add(new JvnCvssDto(
|
||||
version,
|
||||
type,
|
||||
severity,
|
||||
score,
|
||||
vector));
|
||||
}
|
||||
|
||||
return results.ToImmutableArray();
|
||||
}
|
||||
|
||||
private static ImmutableArray<JvnAffectedProductDto> ParseAffected(XElement? affectedElement)
|
||||
{
|
||||
if (affectedElement is null)
|
||||
{
|
||||
return ImmutableArray<JvnAffectedProductDto>.Empty;
|
||||
}
|
||||
|
||||
var results = new List<JvnAffectedProductDto>();
|
||||
foreach (var item in affectedElement.Elements(Vuldef + "AffectedItem"))
|
||||
{
|
||||
var vendor = Clean(item.Element(Vuldef + "Name")?.Value);
|
||||
var product = Clean(item.Element(Vuldef + "ProductName")?.Value);
|
||||
var cpeElement = item.Element(Vuldef + "Cpe");
|
||||
var cpe = Clean(cpeElement?.Value);
|
||||
var cpeVendor = Clean(cpeElement?.Attribute("vendor")?.Value);
|
||||
var cpeProduct = Clean(cpeElement?.Attribute("product")?.Value);
|
||||
var version = Clean(ReadConcatenated(item.Elements(Vuldef + "VersionNumber")));
|
||||
var build = Clean(ReadConcatenated(item.Elements(Vuldef + "BuildNumber")));
|
||||
var description = Clean(ReadConcatenated(item.Elements(Vuldef + "Description")));
|
||||
var status = Clean(item.Attribute("affectedstatus")?.Value);
|
||||
|
||||
results.Add(new JvnAffectedProductDto(vendor, product, cpe, cpeVendor, cpeProduct, version, build, description, status));
|
||||
}
|
||||
|
||||
return results.ToImmutableArray();
|
||||
}
|
||||
|
||||
private static ImmutableArray<JvnReferenceDto> ParseReferences(XElement? relatedElement)
|
||||
{
|
||||
if (relatedElement is null)
|
||||
{
|
||||
return ImmutableArray<JvnReferenceDto>.Empty;
|
||||
}
|
||||
|
||||
var results = new List<JvnReferenceDto>();
|
||||
foreach (var item in relatedElement.Elements(Vuldef + "RelatedItem"))
|
||||
{
|
||||
var type = Clean(item.Attribute("type")?.Value) ?? string.Empty;
|
||||
var id = Clean(item.Element(Vuldef + "VulinfoID")?.Value) ?? string.Empty;
|
||||
var name = Clean(item.Element(Vuldef + "Name")?.Value);
|
||||
var url = Clean(item.Element(Vuldef + "URL")?.Value);
|
||||
|
||||
if (string.IsNullOrWhiteSpace(url))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!Uri.TryCreate(url, UriKind.Absolute, out var uri) || (uri.Scheme is not "http" and not "https"))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
results.Add(new JvnReferenceDto(type, id, name, uri.ToString()));
|
||||
}
|
||||
|
||||
return results.ToImmutableArray();
|
||||
}
|
||||
|
||||
private static ImmutableArray<JvnHistoryEntryDto> ParseHistory(XElement? historyElement)
|
||||
{
|
||||
if (historyElement is null)
|
||||
{
|
||||
return ImmutableArray<JvnHistoryEntryDto>.Empty;
|
||||
}
|
||||
|
||||
var results = new List<JvnHistoryEntryDto>();
|
||||
foreach (var item in historyElement.Elements(Vuldef + "HistoryItem"))
|
||||
{
|
||||
var number = Clean(item.Element(Vuldef + "HistoryNo")?.Value);
|
||||
var timestamp = ParseDate(item.Element(Vuldef + "DateTime")?.Value);
|
||||
var description = Clean(item.Element(Vuldef + "Description")?.Value);
|
||||
results.Add(new JvnHistoryEntryDto(number, timestamp, description));
|
||||
}
|
||||
|
||||
return results.ToImmutableArray();
|
||||
}
|
||||
|
||||
private static DateTimeOffset? ParseDate(string? value)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(value))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return DateTimeOffset.TryParse(value, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out var parsed)
|
||||
? parsed.ToUniversalTime()
|
||||
: null;
|
||||
}
|
||||
|
||||
private static string? Clean(string? value)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(value))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return value.Trim();
|
||||
}
|
||||
|
||||
private static string? ReadConcatenated(IEnumerable<XElement> elements)
|
||||
{
|
||||
var builder = new List<string>();
|
||||
foreach (var element in elements)
|
||||
{
|
||||
var text = element?.Value;
|
||||
if (string.IsNullOrWhiteSpace(text))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
builder.Add(text.Trim());
|
||||
}
|
||||
|
||||
return builder.Count == 0 ? null : string.Join("; ", builder);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal sealed record JvnOverviewItem(
|
||||
string VulnerabilityId,
|
||||
Uri DetailUri,
|
||||
string Title,
|
||||
DateTimeOffset? DateFirstPublished,
|
||||
DateTimeOffset? DateLastUpdated);
|
||||
@@ -0,0 +1,7 @@
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal sealed record JvnOverviewPage(
|
||||
IReadOnlyList<JvnOverviewItem> Items,
|
||||
int TotalResults,
|
||||
int ReturnedCount,
|
||||
int FirstResultIndex);
|
||||
@@ -0,0 +1,167 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.IO;
|
||||
using System.Net;
|
||||
using System.Reflection;
|
||||
using System.Threading;
|
||||
using System.Xml;
|
||||
using System.Xml.Schema;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal static class JvnSchemaProvider
|
||||
{
|
||||
private static readonly Lazy<(XmlSchemaSet SchemaSet, EmbeddedResourceXmlResolver Resolver)> Cached = new(
|
||||
LoadSchemas,
|
||||
LazyThreadSafetyMode.ExecutionAndPublication);
|
||||
|
||||
public static XmlSchemaSet SchemaSet => Cached.Value.SchemaSet;
|
||||
|
||||
private static (XmlSchemaSet SchemaSet, EmbeddedResourceXmlResolver Resolver) LoadSchemas()
|
||||
{
|
||||
var assembly = typeof(JvnSchemaProvider).GetTypeInfo().Assembly;
|
||||
var resourceMap = CreateResourceMap();
|
||||
var resolver = new EmbeddedResourceXmlResolver(assembly, resourceMap);
|
||||
|
||||
var schemaSet = new XmlSchemaSet
|
||||
{
|
||||
XmlResolver = resolver,
|
||||
};
|
||||
|
||||
AddSchema(schemaSet, resolver, "https://jvndb.jvn.jp/schema/vuldef_3.2.xsd");
|
||||
AddSchema(schemaSet, resolver, "https://jvndb.jvn.jp/schema/mod_sec_3.0.xsd");
|
||||
AddSchema(schemaSet, resolver, "https://jvndb.jvn.jp/schema/status_3.3.xsd");
|
||||
AddSchema(schemaSet, resolver, "https://jvndb.jvn.jp/schema/tlp_marking.xsd");
|
||||
AddSchema(schemaSet, resolver, "https://jvndb.jvn.jp/schema/data_marking.xsd");
|
||||
|
||||
schemaSet.Compile();
|
||||
return (schemaSet, resolver);
|
||||
}
|
||||
|
||||
private static void AddSchema(XmlSchemaSet set, EmbeddedResourceXmlResolver resolver, string uri)
|
||||
{
|
||||
using var stream = resolver.OpenStream(uri);
|
||||
using var reader = XmlReader.Create(stream, new XmlReaderSettings { XmlResolver = resolver }, uri);
|
||||
set.Add(null, reader);
|
||||
}
|
||||
|
||||
private static Dictionary<string, string> CreateResourceMap()
|
||||
{
|
||||
var baseNamespace = typeof(JvnSchemaProvider).Namespace ?? "StellaOps.Concelier.Connector.Jvn.Internal";
|
||||
var prefix = baseNamespace.Replace(".Internal", string.Empty, StringComparison.Ordinal);
|
||||
|
||||
return new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["https://jvndb.jvn.jp/schema/vuldef_3.2.xsd"] = $"{prefix}.Schemas.vuldef_3.2.xsd",
|
||||
["vuldef_3.2.xsd"] = $"{prefix}.Schemas.vuldef_3.2.xsd",
|
||||
["https://jvndb.jvn.jp/schema/mod_sec_3.0.xsd"] = $"{prefix}.Schemas.mod_sec_3.0.xsd",
|
||||
["mod_sec_3.0.xsd"] = $"{prefix}.Schemas.mod_sec_3.0.xsd",
|
||||
["https://jvndb.jvn.jp/schema/status_3.3.xsd"] = $"{prefix}.Schemas.status_3.3.xsd",
|
||||
["status_3.3.xsd"] = $"{prefix}.Schemas.status_3.3.xsd",
|
||||
["https://jvndb.jvn.jp/schema/tlp_marking.xsd"] = $"{prefix}.Schemas.tlp_marking.xsd",
|
||||
["tlp_marking.xsd"] = $"{prefix}.Schemas.tlp_marking.xsd",
|
||||
["https://jvndb.jvn.jp/schema/data_marking.xsd"] = $"{prefix}.Schemas.data_marking.xsd",
|
||||
["data_marking.xsd"] = $"{prefix}.Schemas.data_marking.xsd",
|
||||
["https://www.w3.org/2001/xml.xsd"] = $"{prefix}.Schemas.xml.xsd",
|
||||
["xml.xsd"] = $"{prefix}.Schemas.xml.xsd",
|
||||
};
|
||||
}
|
||||
|
||||
private sealed class EmbeddedResourceXmlResolver : XmlResolver
|
||||
{
|
||||
private readonly Assembly _assembly;
|
||||
private readonly Dictionary<string, string> _resourceMap;
|
||||
|
||||
public EmbeddedResourceXmlResolver(Assembly assembly, Dictionary<string, string> resourceMap)
|
||||
{
|
||||
_assembly = assembly ?? throw new ArgumentNullException(nameof(assembly));
|
||||
_resourceMap = resourceMap ?? throw new ArgumentNullException(nameof(resourceMap));
|
||||
}
|
||||
|
||||
public override ICredentials? Credentials
|
||||
{
|
||||
set { }
|
||||
}
|
||||
|
||||
public Stream OpenStream(string uriOrName)
|
||||
{
|
||||
var resourceName = ResolveResourceName(uriOrName)
|
||||
?? throw new FileNotFoundException($"Schema resource '{uriOrName}' not found in manifest.");
|
||||
|
||||
var stream = _assembly.GetManifestResourceStream(resourceName);
|
||||
if (stream is null)
|
||||
{
|
||||
throw new FileNotFoundException($"Embedded schema '{resourceName}' could not be opened.");
|
||||
}
|
||||
|
||||
return stream;
|
||||
}
|
||||
|
||||
public override object? GetEntity(Uri absoluteUri, string? role, Type? ofObjectToReturn)
|
||||
{
|
||||
if (absoluteUri is null)
|
||||
{
|
||||
throw new ArgumentNullException(nameof(absoluteUri));
|
||||
}
|
||||
|
||||
var resourceName = ResolveResourceName(absoluteUri.AbsoluteUri)
|
||||
?? ResolveResourceName(absoluteUri.AbsolutePath.TrimStart('/'))
|
||||
?? ResolveResourceName(Path.GetFileName(absoluteUri.AbsolutePath))
|
||||
?? throw new FileNotFoundException($"Schema resource for '{absoluteUri}' not found.");
|
||||
|
||||
var stream = _assembly.GetManifestResourceStream(resourceName);
|
||||
if (stream is null)
|
||||
{
|
||||
throw new FileNotFoundException($"Embedded schema '{resourceName}' could not be opened.");
|
||||
}
|
||||
|
||||
return stream;
|
||||
}
|
||||
|
||||
public override Uri ResolveUri(Uri? baseUri, string? relativeUri)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(relativeUri))
|
||||
{
|
||||
return base.ResolveUri(baseUri, relativeUri);
|
||||
}
|
||||
|
||||
if (Uri.TryCreate(relativeUri, UriKind.Absolute, out var absolute))
|
||||
{
|
||||
return absolute;
|
||||
}
|
||||
|
||||
if (baseUri is not null && Uri.TryCreate(baseUri, relativeUri, out var combined))
|
||||
{
|
||||
return combined;
|
||||
}
|
||||
|
||||
if (_resourceMap.ContainsKey(relativeUri))
|
||||
{
|
||||
return new Uri($"embedded:///{relativeUri}", UriKind.Absolute);
|
||||
}
|
||||
|
||||
return base.ResolveUri(baseUri, relativeUri);
|
||||
}
|
||||
|
||||
private string? ResolveResourceName(string? key)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(key))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
if (_resourceMap.TryGetValue(key, out var resource))
|
||||
{
|
||||
return resource;
|
||||
}
|
||||
|
||||
var fileName = Path.GetFileName(key);
|
||||
if (!string.IsNullOrEmpty(fileName) && _resourceMap.TryGetValue(fileName, out resource))
|
||||
{
|
||||
return resource;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
using System;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
internal sealed class JvnSchemaValidationException : Exception
|
||||
{
|
||||
public JvnSchemaValidationException(string message)
|
||||
: base(message)
|
||||
{
|
||||
}
|
||||
|
||||
public JvnSchemaValidationException(string message, Exception innerException)
|
||||
: base(message, innerException)
|
||||
{
|
||||
}
|
||||
}
|
||||
240
src/StellaOps.Concelier.Connector.Jvn/Internal/MyJvnClient.cs
Normal file
240
src/StellaOps.Concelier.Connector.Jvn/Internal/MyJvnClient.cs
Normal file
@@ -0,0 +1,240 @@
|
||||
using System.Collections.Generic;
|
||||
using System.Globalization;
|
||||
using System.Linq;
|
||||
using System.Net;
|
||||
using System.Net.Http;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using System.Xml;
|
||||
using System.Xml.Linq;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using StellaOps.Concelier.Connector.Jvn.Configuration;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
public sealed class MyJvnClient
|
||||
{
|
||||
private static readonly XNamespace RssNamespace = "http://purl.org/rss/1.0/";
|
||||
private static readonly XNamespace DcTermsNamespace = "http://purl.org/dc/terms/";
|
||||
private static readonly XNamespace SecNamespace = "http://jvn.jp/rss/mod_sec/3.0/";
|
||||
private static readonly XNamespace RdfNamespace = "http://www.w3.org/1999/02/22-rdf-syntax-ns#";
|
||||
private static readonly XNamespace StatusNamespace = "http://jvndb.jvn.jp/myjvn/Status";
|
||||
|
||||
private static readonly TimeSpan TokyoOffset = TimeSpan.FromHours(9);
|
||||
|
||||
private readonly IHttpClientFactory _httpClientFactory;
|
||||
private readonly JvnOptions _options;
|
||||
private readonly ILogger<MyJvnClient> _logger;
|
||||
|
||||
public MyJvnClient(IHttpClientFactory httpClientFactory, IOptions<JvnOptions> options, ILogger<MyJvnClient> logger)
|
||||
{
|
||||
_httpClientFactory = httpClientFactory ?? throw new ArgumentNullException(nameof(httpClientFactory));
|
||||
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value ?? throw new ArgumentNullException(nameof(options));
|
||||
_options.Validate();
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
internal async Task<IReadOnlyList<JvnOverviewItem>> GetOverviewAsync(DateTimeOffset windowStart, DateTimeOffset windowEnd, CancellationToken cancellationToken)
|
||||
{
|
||||
if (windowEnd <= windowStart)
|
||||
{
|
||||
throw new ArgumentException("windowEnd must be greater than windowStart", nameof(windowEnd));
|
||||
}
|
||||
|
||||
var items = new List<JvnOverviewItem>();
|
||||
var client = _httpClientFactory.CreateClient(JvnOptions.HttpClientName);
|
||||
|
||||
var startItem = 1;
|
||||
var pagesFetched = 0;
|
||||
|
||||
while (pagesFetched < _options.MaxOverviewPagesPerFetch)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
|
||||
var requestUri = BuildOverviewUri(windowStart, windowEnd, startItem);
|
||||
using var response = await client.GetAsync(requestUri, cancellationToken).ConfigureAwait(false);
|
||||
response.EnsureSuccessStatusCode();
|
||||
|
||||
await using var contentStream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false);
|
||||
using var reader = XmlReader.Create(contentStream, new XmlReaderSettings { Async = true, IgnoreWhitespace = true, IgnoreComments = true });
|
||||
var document = await XDocument.LoadAsync(reader, LoadOptions.None, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
var page = ParseOverviewPage(document);
|
||||
if (page.Items.Count == 0)
|
||||
{
|
||||
_logger.LogDebug("JVN overview page starting at {StartItem} returned zero results", startItem);
|
||||
break;
|
||||
}
|
||||
|
||||
items.AddRange(page.Items);
|
||||
pagesFetched++;
|
||||
|
||||
if (page.ReturnedCount < _options.PageSize || startItem + _options.PageSize > page.TotalResults)
|
||||
{
|
||||
break;
|
||||
}
|
||||
|
||||
startItem += _options.PageSize;
|
||||
|
||||
if (_options.RequestDelay > TimeSpan.Zero)
|
||||
{
|
||||
try
|
||||
{
|
||||
await Task.Delay(_options.RequestDelay, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (TaskCanceledException)
|
||||
{
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
private Uri BuildOverviewUri(DateTimeOffset windowStart, DateTimeOffset windowEnd, int startItem)
|
||||
{
|
||||
var (startYear, startMonth, startDay) = ToTokyoDateParts(windowStart);
|
||||
var (endYear, endMonth, endDay) = ToTokyoDateParts(windowEnd);
|
||||
|
||||
var parameters = new[]
|
||||
{
|
||||
new KeyValuePair<string, string>("method", "getVulnOverviewList"),
|
||||
new KeyValuePair<string, string>("feed", "hnd"),
|
||||
new KeyValuePair<string, string>("lang", "en"),
|
||||
new KeyValuePair<string, string>("rangeDatePublished", "n"),
|
||||
new KeyValuePair<string, string>("rangeDatePublic", "n"),
|
||||
new KeyValuePair<string, string>("rangeDateFirstPublished", "n"),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedStartY", startYear),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedStartM", startMonth),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedStartD", startDay),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedEndY", endYear),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedEndM", endMonth),
|
||||
new KeyValuePair<string, string>("dateFirstPublishedEndD", endDay),
|
||||
new KeyValuePair<string, string>("startItem", startItem.ToString(CultureInfo.InvariantCulture)),
|
||||
new KeyValuePair<string, string>("maxCountItem", _options.PageSize.ToString(CultureInfo.InvariantCulture)),
|
||||
};
|
||||
|
||||
var query = BuildQueryString(parameters);
|
||||
|
||||
var builder = new UriBuilder(_options.BaseEndpoint)
|
||||
{
|
||||
Query = query,
|
||||
};
|
||||
return builder.Uri;
|
||||
}
|
||||
|
||||
private static (string Year, string Month, string Day) ToTokyoDateParts(DateTimeOffset timestamp)
|
||||
{
|
||||
var local = timestamp.ToOffset(TokyoOffset).Date;
|
||||
return (
|
||||
local.Year.ToString("D4", CultureInfo.InvariantCulture),
|
||||
local.Month.ToString("D2", CultureInfo.InvariantCulture),
|
||||
local.Day.ToString("D2", CultureInfo.InvariantCulture));
|
||||
}
|
||||
|
||||
private static JvnOverviewPage ParseOverviewPage(XDocument document)
|
||||
{
|
||||
var items = new List<JvnOverviewItem>();
|
||||
|
||||
foreach (var item in document.Descendants(RssNamespace + "item"))
|
||||
{
|
||||
var identifier = item.Element(SecNamespace + "identifier")?.Value?.Trim();
|
||||
if (string.IsNullOrWhiteSpace(identifier))
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
Uri? detailUri = null;
|
||||
var linkValue = item.Element(RssNamespace + "link")?.Value?.Trim();
|
||||
if (!string.IsNullOrWhiteSpace(linkValue))
|
||||
{
|
||||
Uri.TryCreate(linkValue, UriKind.Absolute, out detailUri);
|
||||
}
|
||||
|
||||
if (detailUri is null)
|
||||
{
|
||||
var aboutValue = item.Attribute(RdfNamespace + "about")?.Value?.Trim();
|
||||
if (!string.IsNullOrWhiteSpace(aboutValue))
|
||||
{
|
||||
Uri.TryCreate(aboutValue, UriKind.Absolute, out detailUri);
|
||||
}
|
||||
}
|
||||
|
||||
if (detailUri is null)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
var title = item.Element(RssNamespace + "title")?.Value?.Trim();
|
||||
if (string.IsNullOrWhiteSpace(title))
|
||||
{
|
||||
title = identifier;
|
||||
}
|
||||
|
||||
var firstPublished = TryParseDate(item.Element(DcTermsNamespace + "issued")?.Value);
|
||||
var lastUpdated = TryParseDate(item.Element(DcTermsNamespace + "modified")?.Value);
|
||||
|
||||
items.Add(new JvnOverviewItem(identifier, detailUri, title!, firstPublished, lastUpdated));
|
||||
}
|
||||
|
||||
var statusElement = document.Root?.Element(StatusNamespace + "Status")
|
||||
?? document.Descendants(StatusNamespace + "Status").FirstOrDefault();
|
||||
|
||||
var totalResults = TryParseInt(statusElement?.Attribute("totalRes")?.Value) ?? items.Count;
|
||||
var returned = TryParseInt(statusElement?.Attribute("totalResRet")?.Value) ?? items.Count;
|
||||
var firstResult = TryParseInt(statusElement?.Attribute("firstRes")?.Value) ?? 1;
|
||||
|
||||
return new JvnOverviewPage(items, totalResults, returned, firstResult);
|
||||
}
|
||||
|
||||
private static DateTimeOffset? TryParseDate(string? value)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(value))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return DateTimeOffset.TryParse(value, CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal, out var parsed)
|
||||
? parsed.ToUniversalTime()
|
||||
: null;
|
||||
}
|
||||
|
||||
private static int? TryParseInt(string? value)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(value))
|
||||
{
|
||||
return null;
|
||||
}
|
||||
|
||||
return int.TryParse(value, NumberStyles.Integer, CultureInfo.InvariantCulture, out var parsed) ? parsed : null;
|
||||
}
|
||||
|
||||
internal Uri BuildDetailUri(string vulnerabilityId)
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrEmpty(vulnerabilityId);
|
||||
|
||||
var query = BuildQueryString(new[]
|
||||
{
|
||||
new KeyValuePair<string, string>("method", "getVulnDetailInfo"),
|
||||
new KeyValuePair<string, string>("feed", "hnd"),
|
||||
new KeyValuePair<string, string>("lang", "en"),
|
||||
new KeyValuePair<string, string>("vulnId", vulnerabilityId.Trim()),
|
||||
});
|
||||
var builder = new UriBuilder(_options.BaseEndpoint)
|
||||
{
|
||||
Query = query,
|
||||
};
|
||||
|
||||
return builder.Uri;
|
||||
}
|
||||
|
||||
private static string BuildQueryString(IEnumerable<KeyValuePair<string, string>> parameters)
|
||||
{
|
||||
return string.Join(
|
||||
"&",
|
||||
parameters.Select(parameter =>
|
||||
$"{WebUtility.UrlEncode(parameter.Key)}={WebUtility.UrlEncode(parameter.Value)}"));
|
||||
}
|
||||
}
|
||||
46
src/StellaOps.Concelier.Connector.Jvn/Jobs.cs
Normal file
46
src/StellaOps.Concelier.Connector.Jvn/Jobs.cs
Normal file
@@ -0,0 +1,46 @@
|
||||
using System;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using StellaOps.Concelier.Core.Jobs;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn;
|
||||
|
||||
internal static class JvnJobKinds
|
||||
{
|
||||
public const string Fetch = "source:jvn:fetch";
|
||||
public const string Parse = "source:jvn:parse";
|
||||
public const string Map = "source:jvn:map";
|
||||
}
|
||||
|
||||
internal sealed class JvnFetchJob : IJob
|
||||
{
|
||||
private readonly JvnConnector _connector;
|
||||
|
||||
public JvnFetchJob(JvnConnector connector)
|
||||
=> _connector = connector ?? throw new ArgumentNullException(nameof(connector));
|
||||
|
||||
public Task ExecuteAsync(JobExecutionContext context, CancellationToken cancellationToken)
|
||||
=> _connector.FetchAsync(context.Services, cancellationToken);
|
||||
}
|
||||
|
||||
internal sealed class JvnParseJob : IJob
|
||||
{
|
||||
private readonly JvnConnector _connector;
|
||||
|
||||
public JvnParseJob(JvnConnector connector)
|
||||
=> _connector = connector ?? throw new ArgumentNullException(nameof(connector));
|
||||
|
||||
public Task ExecuteAsync(JobExecutionContext context, CancellationToken cancellationToken)
|
||||
=> _connector.ParseAsync(context.Services, cancellationToken);
|
||||
}
|
||||
|
||||
internal sealed class JvnMapJob : IJob
|
||||
{
|
||||
private readonly JvnConnector _connector;
|
||||
|
||||
public JvnMapJob(JvnConnector connector)
|
||||
=> _connector = connector ?? throw new ArgumentNullException(nameof(connector));
|
||||
|
||||
public Task ExecuteAsync(JobExecutionContext context, CancellationToken cancellationToken)
|
||||
=> _connector.MapAsync(context.Services, cancellationToken);
|
||||
}
|
||||
325
src/StellaOps.Concelier.Connector.Jvn/JvnConnector.cs
Normal file
325
src/StellaOps.Concelier.Connector.Jvn/JvnConnector.cs
Normal file
@@ -0,0 +1,325 @@
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text.Json;
|
||||
using Microsoft.Extensions.Logging;
|
||||
using Microsoft.Extensions.Options;
|
||||
using MongoDB.Bson;
|
||||
using StellaOps.Concelier.Models;
|
||||
using StellaOps.Concelier.Connector.Common;
|
||||
using StellaOps.Concelier.Connector.Common.Fetch;
|
||||
using StellaOps.Concelier.Connector.Jvn.Configuration;
|
||||
using StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
using StellaOps.Concelier.Storage.Mongo;
|
||||
using StellaOps.Concelier.Storage.Mongo.Advisories;
|
||||
using StellaOps.Concelier.Storage.Mongo.Documents;
|
||||
using StellaOps.Concelier.Storage.Mongo.Dtos;
|
||||
using StellaOps.Concelier.Storage.Mongo.JpFlags;
|
||||
using StellaOps.Plugin;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn;
|
||||
|
||||
public sealed class JvnConnector : IFeedConnector
|
||||
{
|
||||
private static readonly JsonSerializerOptions SerializerOptions = new(JsonSerializerDefaults.General)
|
||||
{
|
||||
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
|
||||
WriteIndented = false,
|
||||
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingNull,
|
||||
};
|
||||
|
||||
private readonly MyJvnClient _client;
|
||||
private readonly SourceFetchService _fetchService;
|
||||
private readonly RawDocumentStorage _rawDocumentStorage;
|
||||
private readonly IDocumentStore _documentStore;
|
||||
private readonly IDtoStore _dtoStore;
|
||||
private readonly IAdvisoryStore _advisoryStore;
|
||||
private readonly IJpFlagStore _jpFlagStore;
|
||||
private readonly ISourceStateRepository _stateRepository;
|
||||
private readonly TimeProvider _timeProvider;
|
||||
private readonly JvnOptions _options;
|
||||
private readonly ILogger<JvnConnector> _logger;
|
||||
|
||||
public JvnConnector(
|
||||
MyJvnClient client,
|
||||
SourceFetchService fetchService,
|
||||
RawDocumentStorage rawDocumentStorage,
|
||||
IDocumentStore documentStore,
|
||||
IDtoStore dtoStore,
|
||||
IAdvisoryStore advisoryStore,
|
||||
IJpFlagStore jpFlagStore,
|
||||
ISourceStateRepository stateRepository,
|
||||
IOptions<JvnOptions> options,
|
||||
TimeProvider? timeProvider,
|
||||
ILogger<JvnConnector> logger)
|
||||
{
|
||||
_client = client ?? throw new ArgumentNullException(nameof(client));
|
||||
_fetchService = fetchService ?? throw new ArgumentNullException(nameof(fetchService));
|
||||
_rawDocumentStorage = rawDocumentStorage ?? throw new ArgumentNullException(nameof(rawDocumentStorage));
|
||||
_documentStore = documentStore ?? throw new ArgumentNullException(nameof(documentStore));
|
||||
_dtoStore = dtoStore ?? throw new ArgumentNullException(nameof(dtoStore));
|
||||
_advisoryStore = advisoryStore ?? throw new ArgumentNullException(nameof(advisoryStore));
|
||||
_jpFlagStore = jpFlagStore ?? throw new ArgumentNullException(nameof(jpFlagStore));
|
||||
_stateRepository = stateRepository ?? throw new ArgumentNullException(nameof(stateRepository));
|
||||
_options = (options ?? throw new ArgumentNullException(nameof(options))).Value ?? throw new ArgumentNullException(nameof(options));
|
||||
_options.Validate();
|
||||
_timeProvider = timeProvider ?? TimeProvider.System;
|
||||
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
|
||||
}
|
||||
|
||||
public string SourceName => JvnConnectorPlugin.SourceName;
|
||||
|
||||
public async Task FetchAsync(IServiceProvider services, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
|
||||
var cursor = await GetCursorAsync(cancellationToken).ConfigureAwait(false);
|
||||
var now = _timeProvider.GetUtcNow();
|
||||
|
||||
var windowEnd = now;
|
||||
var defaultWindowStart = windowEnd - _options.WindowSize;
|
||||
|
||||
var windowStart = cursor.LastCompletedWindowEnd.HasValue
|
||||
? cursor.LastCompletedWindowEnd.Value - _options.WindowOverlap
|
||||
: defaultWindowStart;
|
||||
|
||||
if (windowStart < defaultWindowStart)
|
||||
{
|
||||
windowStart = defaultWindowStart;
|
||||
}
|
||||
|
||||
if (windowStart >= windowEnd)
|
||||
{
|
||||
windowStart = windowEnd - TimeSpan.FromHours(1);
|
||||
}
|
||||
|
||||
_logger.LogInformation("JVN fetch window {WindowStart:o} - {WindowEnd:o}", windowStart, windowEnd);
|
||||
|
||||
IReadOnlyList<JvnOverviewItem> overviewItems;
|
||||
try
|
||||
{
|
||||
overviewItems = await _client.GetOverviewAsync(windowStart, windowEnd, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to retrieve JVN overview between {Start:o} and {End:o}", windowStart, windowEnd);
|
||||
await _stateRepository.MarkFailureAsync(SourceName, now, TimeSpan.FromMinutes(5), ex.Message, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
_logger.LogInformation("JVN overview returned {Count} items", overviewItems.Count);
|
||||
|
||||
var pendingDocuments = cursor.PendingDocuments.ToHashSet();
|
||||
|
||||
foreach (var item in overviewItems)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
|
||||
var detailUri = _client.BuildDetailUri(item.VulnerabilityId);
|
||||
var metadata = new Dictionary<string, string>(StringComparer.Ordinal)
|
||||
{
|
||||
["jvn.vulnId"] = item.VulnerabilityId,
|
||||
["jvn.detailUrl"] = detailUri.ToString(),
|
||||
};
|
||||
|
||||
if (item.DateFirstPublished.HasValue)
|
||||
{
|
||||
metadata["jvn.firstPublished"] = item.DateFirstPublished.Value.ToString("O");
|
||||
}
|
||||
|
||||
if (item.DateLastUpdated.HasValue)
|
||||
{
|
||||
metadata["jvn.lastUpdated"] = item.DateLastUpdated.Value.ToString("O");
|
||||
}
|
||||
|
||||
var result = await _fetchService.FetchAsync(
|
||||
new SourceFetchRequest(JvnOptions.HttpClientName, SourceName, detailUri)
|
||||
{
|
||||
Metadata = metadata
|
||||
},
|
||||
cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (!result.IsSuccess || result.Document is null)
|
||||
{
|
||||
if (!result.IsNotModified)
|
||||
{
|
||||
_logger.LogWarning("JVN fetch for {Uri} returned status {Status}", detailUri, result.StatusCode);
|
||||
}
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
_logger.LogDebug("JVN fetched document {DocumentId}", result.Document.Id);
|
||||
pendingDocuments.Add(result.Document.Id);
|
||||
}
|
||||
|
||||
var updatedCursor = cursor
|
||||
.WithWindow(windowStart, windowEnd)
|
||||
.WithCompletedWindow(windowEnd)
|
||||
.WithPendingDocuments(pendingDocuments);
|
||||
|
||||
await UpdateCursorAsync(updatedCursor, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
public async Task ParseAsync(IServiceProvider services, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
|
||||
var cursor = await GetCursorAsync(cancellationToken).ConfigureAwait(false);
|
||||
_logger.LogDebug("JVN parse pending documents: {PendingCount}", cursor.PendingDocuments.Count);
|
||||
Console.WriteLine($"JVN parse pending count: {cursor.PendingDocuments.Count}");
|
||||
if (cursor.PendingDocuments.Count == 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var remainingDocuments = cursor.PendingDocuments.ToList();
|
||||
var pendingMappings = cursor.PendingMappings.ToList();
|
||||
|
||||
foreach (var documentId in cursor.PendingDocuments)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
_logger.LogDebug("JVN parsing document {DocumentId}", documentId);
|
||||
Console.WriteLine($"JVN parsing document {documentId}");
|
||||
|
||||
var document = await _documentStore.FindAsync(documentId, cancellationToken).ConfigureAwait(false);
|
||||
if (document is null)
|
||||
{
|
||||
_logger.LogWarning("JVN document {DocumentId} no longer exists; skipping", documentId);
|
||||
remainingDocuments.Remove(documentId);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!document.GridFsId.HasValue)
|
||||
{
|
||||
_logger.LogWarning("JVN document {DocumentId} is missing GridFS content; marking as failed", documentId);
|
||||
await _documentStore.UpdateStatusAsync(document.Id, DocumentStatuses.Failed, cancellationToken).ConfigureAwait(false);
|
||||
remainingDocuments.Remove(documentId);
|
||||
continue;
|
||||
}
|
||||
|
||||
byte[] rawBytes;
|
||||
try
|
||||
{
|
||||
rawBytes = await _rawDocumentStorage.DownloadAsync(document.GridFsId.Value, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Unable to download raw JVN document {DocumentId}", document.Id);
|
||||
throw;
|
||||
}
|
||||
|
||||
JvnDetailDto detail;
|
||||
try
|
||||
{
|
||||
detail = JvnDetailParser.Parse(rawBytes, document.Uri);
|
||||
}
|
||||
catch (JvnSchemaValidationException ex)
|
||||
{
|
||||
Console.WriteLine($"JVN schema validation exception: {ex.Message}");
|
||||
_logger.LogWarning(ex, "JVN schema validation failed for document {DocumentId} ({Uri})", document.Id, document.Uri);
|
||||
await _documentStore.UpdateStatusAsync(document.Id, DocumentStatuses.Failed, cancellationToken).ConfigureAwait(false);
|
||||
remainingDocuments.Remove(documentId);
|
||||
throw;
|
||||
}
|
||||
|
||||
var sanitizedJson = JsonSerializer.Serialize(detail, SerializerOptions);
|
||||
var payload = BsonDocument.Parse(sanitizedJson);
|
||||
var dtoRecord = new DtoRecord(
|
||||
Guid.NewGuid(),
|
||||
document.Id,
|
||||
SourceName,
|
||||
JvnConstants.DtoSchemaVersion,
|
||||
payload,
|
||||
_timeProvider.GetUtcNow());
|
||||
|
||||
await _dtoStore.UpsertAsync(dtoRecord, cancellationToken).ConfigureAwait(false);
|
||||
await _documentStore.UpdateStatusAsync(document.Id, DocumentStatuses.PendingMap, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
remainingDocuments.Remove(documentId);
|
||||
if (!pendingMappings.Contains(documentId))
|
||||
{
|
||||
pendingMappings.Add(documentId);
|
||||
Console.WriteLine($"Added mapping for {documentId}");
|
||||
_logger.LogDebug("JVN parsed document {DocumentId}", documentId);
|
||||
}
|
||||
}
|
||||
|
||||
var updatedCursor = cursor
|
||||
.WithPendingDocuments(remainingDocuments)
|
||||
.WithPendingMappings(pendingMappings);
|
||||
|
||||
await UpdateCursorAsync(updatedCursor, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
public async Task MapAsync(IServiceProvider services, CancellationToken cancellationToken)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
|
||||
var cursor = await GetCursorAsync(cancellationToken).ConfigureAwait(false);
|
||||
_logger.LogDebug("JVN map pending mappings: {PendingCount}", cursor.PendingMappings.Count);
|
||||
if (cursor.PendingMappings.Count == 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var pendingMappings = cursor.PendingMappings.ToList();
|
||||
|
||||
foreach (var documentId in cursor.PendingMappings)
|
||||
{
|
||||
cancellationToken.ThrowIfCancellationRequested();
|
||||
|
||||
var dto = await _dtoStore.FindByDocumentIdAsync(documentId, cancellationToken).ConfigureAwait(false);
|
||||
var document = await _documentStore.FindAsync(documentId, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (dto is null || document is null)
|
||||
{
|
||||
_logger.LogWarning("Skipping JVN mapping for {DocumentId}: DTO or document missing", documentId);
|
||||
pendingMappings.Remove(documentId);
|
||||
continue;
|
||||
}
|
||||
|
||||
var dtoJson = dto.Payload.ToJson(new MongoDB.Bson.IO.JsonWriterSettings
|
||||
{
|
||||
OutputMode = MongoDB.Bson.IO.JsonOutputMode.RelaxedExtendedJson,
|
||||
});
|
||||
|
||||
JvnDetailDto detail;
|
||||
try
|
||||
{
|
||||
detail = JsonSerializer.Deserialize<JvnDetailDto>(dtoJson, SerializerOptions)
|
||||
?? throw new InvalidOperationException("Deserialized DTO was null.");
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to deserialize JVN DTO for document {DocumentId}", document.Id);
|
||||
await _documentStore.UpdateStatusAsync(document.Id, DocumentStatuses.Failed, cancellationToken).ConfigureAwait(false);
|
||||
pendingMappings.Remove(documentId);
|
||||
continue;
|
||||
}
|
||||
|
||||
var (advisory, flag) = JvnAdvisoryMapper.Map(detail, document, dto, _timeProvider);
|
||||
|
||||
await _advisoryStore.UpsertAsync(advisory, cancellationToken).ConfigureAwait(false);
|
||||
await _jpFlagStore.UpsertAsync(flag, cancellationToken).ConfigureAwait(false);
|
||||
await _documentStore.UpdateStatusAsync(document.Id, DocumentStatuses.Mapped, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
pendingMappings.Remove(documentId);
|
||||
_logger.LogDebug("JVN mapped document {DocumentId}", documentId);
|
||||
}
|
||||
|
||||
var updatedCursor = cursor.WithPendingMappings(pendingMappings);
|
||||
await UpdateCursorAsync(updatedCursor, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
private async Task<JvnCursor> GetCursorAsync(CancellationToken cancellationToken)
|
||||
{
|
||||
var state = await _stateRepository.TryGetAsync(SourceName, cancellationToken).ConfigureAwait(false);
|
||||
return state is null ? JvnCursor.Empty : JvnCursor.FromBson(state.Cursor);
|
||||
}
|
||||
|
||||
private async Task UpdateCursorAsync(JvnCursor cursor, CancellationToken cancellationToken)
|
||||
{
|
||||
var cursorDocument = cursor.ToBsonDocument();
|
||||
await _stateRepository.UpdateCursorAsync(SourceName, cursorDocument, _timeProvider.GetUtcNow(), cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
19
src/StellaOps.Concelier.Connector.Jvn/JvnConnectorPlugin.cs
Normal file
19
src/StellaOps.Concelier.Connector.Jvn/JvnConnectorPlugin.cs
Normal file
@@ -0,0 +1,19 @@
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using StellaOps.Plugin;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn;
|
||||
|
||||
public sealed class JvnConnectorPlugin : IConnectorPlugin
|
||||
{
|
||||
public string Name => SourceName;
|
||||
|
||||
public static string SourceName => "jvn";
|
||||
|
||||
public bool IsAvailable(IServiceProvider services) => services is not null;
|
||||
|
||||
public IFeedConnector Create(IServiceProvider services)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
return ActivatorUtilities.CreateInstance<JvnConnector>(services);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
using System;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using StellaOps.DependencyInjection;
|
||||
using StellaOps.Concelier.Core.Jobs;
|
||||
using StellaOps.Concelier.Connector.Jvn.Configuration;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn;
|
||||
|
||||
public sealed class JvnDependencyInjectionRoutine : IDependencyInjectionRoutine
|
||||
{
|
||||
private const string ConfigurationSection = "concelier:sources:jvn";
|
||||
|
||||
public IServiceCollection Register(IServiceCollection services, IConfiguration configuration)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(configuration);
|
||||
|
||||
services.AddJvnConnector(options =>
|
||||
{
|
||||
configuration.GetSection(ConfigurationSection).Bind(options);
|
||||
options.Validate();
|
||||
});
|
||||
|
||||
services.AddTransient<JvnFetchJob>();
|
||||
services.AddTransient<JvnParseJob>();
|
||||
services.AddTransient<JvnMapJob>();
|
||||
|
||||
services.PostConfigure<JobSchedulerOptions>(options =>
|
||||
{
|
||||
EnsureJob(options, JvnJobKinds.Fetch, typeof(JvnFetchJob));
|
||||
EnsureJob(options, JvnJobKinds.Parse, typeof(JvnParseJob));
|
||||
EnsureJob(options, JvnJobKinds.Map, typeof(JvnMapJob));
|
||||
});
|
||||
|
||||
return services;
|
||||
}
|
||||
|
||||
private static void EnsureJob(JobSchedulerOptions options, string kind, Type jobType)
|
||||
{
|
||||
if (options.Definitions.ContainsKey(kind))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
options.Definitions[kind] = new JobDefinition(
|
||||
kind,
|
||||
jobType,
|
||||
options.DefaultTimeout,
|
||||
options.DefaultLeaseDuration,
|
||||
CronExpression: null,
|
||||
Enabled: true);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
using System;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Options;
|
||||
using StellaOps.Concelier.Connector.Common.Http;
|
||||
using StellaOps.Concelier.Connector.Jvn.Configuration;
|
||||
using StellaOps.Concelier.Connector.Jvn.Internal;
|
||||
|
||||
namespace StellaOps.Concelier.Connector.Jvn;
|
||||
|
||||
public static class JvnServiceCollectionExtensions
|
||||
{
|
||||
public static IServiceCollection AddJvnConnector(this IServiceCollection services, Action<JvnOptions> configure)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(services);
|
||||
ArgumentNullException.ThrowIfNull(configure);
|
||||
|
||||
services.AddOptions<JvnOptions>()
|
||||
.Configure(configure)
|
||||
.PostConfigure(static options => options.Validate());
|
||||
|
||||
services.AddSourceHttpClient(JvnOptions.HttpClientName, (sp, clientOptions) =>
|
||||
{
|
||||
var options = sp.GetRequiredService<IOptions<JvnOptions>>().Value;
|
||||
clientOptions.BaseAddress = options.BaseEndpoint;
|
||||
clientOptions.Timeout = TimeSpan.FromSeconds(30);
|
||||
clientOptions.UserAgent = "StellaOps.Concelier.Jvn/1.0";
|
||||
clientOptions.AllowedHosts.Clear();
|
||||
clientOptions.AllowedHosts.Add(options.BaseEndpoint.Host);
|
||||
clientOptions.DefaultRequestHeaders["Accept"] = "application/xml";
|
||||
});
|
||||
|
||||
services.AddTransient<MyJvnClient>();
|
||||
services.AddTransient<JvnConnector>();
|
||||
|
||||
return services;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:stixCommon="http://stix.mitre.org/common-1" targetNamespace="http://data-marking.mitre.org/Marking-1" elementFormDefault="qualified" attributeFormDefault="unqualified" version="1.1.1" xml:lang="English">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This schema was originally developed by The MITRE Corporation. The Data Marking XML Schema implementation is maintained by The MITRE Corporation and developed by the open STIX Community. For more information, including how to get involved in the effort and how to submit change requests, please visit the STIX website at http://stix.mitre.org. </xs:documentation>
|
||||
<xs:appinfo>
|
||||
<schema>Data Marking</schema>
|
||||
<version>1.1.1</version>
|
||||
<date>05/08/2014 9:00:00 AM</date>
|
||||
<short_description>Data_Marking - Schematic implementation for an independent, flexible, structured data marking expression.</short_description>
|
||||
<terms_of_use>Copyright (c) 2012-2014, The MITRE Corporation. All rights reserved. The contents of this file are subject to the terms of the STIX License located at http://stix.mitre.org/about/termsofuse.html. See the STIX License for the specific language governing permissions and limitations for use of this schema. When distributing copies of the Data Marking Schema, this license header must be included. </terms_of_use>
|
||||
</xs:appinfo>
|
||||
</xs:annotation>
|
||||
<xs:complexType name="MarkingType">
|
||||
<xs:annotation>
|
||||
<xs:documentation>MarkingType specifies a structure for marking information to be applied to portions of XML content.</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:sequence>
|
||||
<xs:element name="Marking" type="marking:MarkingSpecificationType" minOccurs="0" maxOccurs="unbounded">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This field contains specification of marking information to be applied to portions of XML content.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:element>
|
||||
</xs:sequence>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="MarkingStructureType">
|
||||
<xs:annotation>
|
||||
<xs:documentation>The MarkingStructureType contains the marking information to be applied to a portion of XML content.</xs:documentation>
|
||||
<xs:documentation>This type is defined as abstract and is intended to be extended to enable the expression of any structured or unstructured data marking mechanism. The data marking structure is simply a mechanism for applying existing marking systems to nodes. The data marking systems themselves define the semantics of what the markings mean, how multiple markings to the same node should be applied, and what to do if a node is unmarked.</xs:documentation>
|
||||
<xs:documentation>It is valid per this specification to mark a node with multiple markings from the same system or mark a node across multiple marking systems. If a node is marked multiple times using the same marking system, that system specifies the semantic meaning of multiple markings and (if necessary) how conflicts should be resolved. If a node is marked across multiple marking systems, each system is considered individually applicable. If there are conflicting markings across marking systems the behavior is undefined, therefore producers should make every effort to ensure documents are marked consistently and correctly among all marking systems.</xs:documentation>
|
||||
<xs:documentation>STIX provides two marking system extensions: Simple, and TLP. Those who wish to use another format may do so by defining a new extension to this type. The STIX-provided extensions are:</xs:documentation>
|
||||
<xs:documentation>1. Simple: The Simple marking structure allows for the specification of unstructured statements through the use of a string field. The type is named SimpleMarkingStructureType and is in the http://data-marking.mitre.org/extensions/MarkingStructure#Simple-1 namespace. The extension is defined in the file extensions/marking/simple_marking.xsd or at the URL http://stix.mitre.org/XMLSchema/extensions/marking/simple_marking/1.1.1/simple_marking.xsd.</xs:documentation>
|
||||
<xs:documentation>2. TLP: The TLP marking structure allows for the expression of Traffic Light Protocol statements through the use of a simple enumeration. The type is named TLPMarkingStructureType and is in the http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1 namespace. The extension is defined in the file extensions/marking/tlp_marking.xsd or at the URL http://stix.mitre.org/XMLSchema/extensions/marking/tlp/1.1.1/tlp_marking.xsd.</xs:documentation>
|
||||
<xs:documentation>3. Terms of Use: The Terms of Use marking structure allows for the specification of unstructured terms of use statements through the use of a string field. The type is named TermsOfUseMarkingStructureType and is in the http://data-marking.mitre.org/extensions/MarkingStructure#Terms_Of_Use-1 namespace. The extension is defined in the file extensions/marking/terms_of_use_marking.xsd or at the URL http://stix.mitre.org/XMLSchema/extensions/marking/terms_of_use/1.0.1/terms_of_use_marking.xsd.</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:attribute name="marking_model_name" type="xs:QName">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This field specifies the name of the marking model to be applied within this Marking_Structure.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="marking_model_ref" type="xs:anyURI">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This field contains a reference to an authoritative source on the marking model to be applied within this Marking_Structure.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="id" type="xs:QName">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies a unique ID for this Marking_Structure.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="idref" type="xs:QName">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies a reference to the ID of a Marking_Structure defined elsewhere.</xs:documentation>
|
||||
<xs:documentation>When idref is specified, the id attribute must not be specified, and any instance of this Marking_Structure should not hold content.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="MarkingSpecificationType">
|
||||
<xs:sequence>
|
||||
<xs:element name="Controlled_Structure" type="xs:string" minOccurs="0">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This field utilizes XPath 1.0 to specify the structures for which the Marking is to be applied.</xs:documentation>
|
||||
<xs:documentation>The XPath expression is NOT recursive and the marking structure does NOT apply to child nodes of the selected node. Instead, the expression must explicitly select all nodes that the marking is to be applied to including elements, attributes, and text nodes.</xs:documentation>
|
||||
<xs:documentation>The context root of the XPath statement is this Controlled_Structure element. Any namespace prefix declarations that are available to this Controlled_Structure element are available to the XPath.</xs:documentation>
|
||||
<xs:documentation>Note that all Controlled_Structure elements have a scope within the document for which their XPath is valid to reference.</xs:documentation>
|
||||
<xs:documentation>Usages of MarkingType may specify a "marking scope". The marking scope is always recursive and specifies the set of nodes that may be selected by the XPath expression (and therefore that may have the markings applied to them). If no marking scope is specified in the schema documentation or specification where the MarkingType is used, it should be assumed that the document itself and all nodes are within scope.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:element>
|
||||
<xs:element name="Marking_Structure" type="marking:MarkingStructureType" minOccurs="0" maxOccurs="unbounded">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This field contains the marking information to be applied to the portions of XML content specified in the ControlledStructure field. This field is defined as MarkingStructureType which is an abstract type the enables the flexibility to utilize any variety of marking structures.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:element>
|
||||
</xs:sequence>
|
||||
<xs:attribute name="id" type="xs:QName">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies a unique ID for this Marking.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="idref" type="xs:QName">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies a reference to the ID of a Marking defined elsewhere.</xs:documentation>
|
||||
<xs:documentation>When idref is specified, the id attribute must not be specified, and any instance of this Marking should not hold content.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="version" type="xs:string">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies the relevant Data_Marking schema version for this content.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
</xs:complexType>
|
||||
</xs:schema>
|
||||
133
src/StellaOps.Concelier.Connector.Jvn/Schemas/jvnrss_3.2.xsd
Normal file
133
src/StellaOps.Concelier.Connector.Jvn/Schemas/jvnrss_3.2.xsd
Normal file
@@ -0,0 +1,133 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<xs:schema
|
||||
targetNamespace="http://purl.org/rss/1.0/"
|
||||
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
|
||||
xmlns:rss="http://purl.org/rss/1.0/"
|
||||
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
||||
xmlns:dcterms="http://purl.org/dc/terms/"
|
||||
xmlns:sec="http://jvn.jp/rss/mod_sec/3.0/"
|
||||
xmlns:status="http://jvndb.jvn.jp/myjvn/Status"
|
||||
xmlns:marking="http://data-marking.mitre.org/Marking-1"
|
||||
xmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1"
|
||||
xmlns:xs="http://www.w3.org/2001/XMLSchema"
|
||||
elementFormDefault="qualified"
|
||||
attributeFormDefault="unqualified"
|
||||
version="3.2">
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema imports -->
|
||||
<!-- ================================================== -->
|
||||
<xs:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="xml.xsd"/>
|
||||
<xs:import namespace="http://www.w3.org/1999/02/22-rdf-syntax-ns#" schemaLocation="jvnrss_rdf_3.2.xsd"/>
|
||||
<xs:import namespace="http://purl.org/dc/elements/1.1/" schemaLocation="jvnrss_dc.xsd"/>
|
||||
<xs:import namespace="http://purl.org/dc/terms/" schemaLocation="jvnrss_dcterms.xsd"/>
|
||||
<xs:import namespace="http://jvn.jp/rss/mod_sec/3.0/" schemaLocation="mod_sec_3.0.xsd"/>
|
||||
<xs:import namespace="http://data-marking.mitre.org/Marking-1" schemaLocation="data_marking.xsd"/>
|
||||
<xs:import namespace="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1" schemaLocation="tlp_marking.xsd"/>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema description -->
|
||||
<!-- ================================================== -->
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">JVNRSS is based on RDF Site Summary (RSS) 1.0 and use the
|
||||
field dc:relation of Dublin Core / sec:references of mod_sec as index of grouping
|
||||
security information.</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">JVNRSS は、脆弱性対策情報の概要記述用 XML フォーマットで、サイトの概要をメタデータとして簡潔に記述する
|
||||
XML フォーマットである RSS (RDF Site Summary) 1.0 をベースとした仕様です。他サイトに掲載可能な形式で発信する仕組み、脆弱性対策情報のグループ化
|
||||
(dc:relation, sec:references)
|
||||
や抽出した情報の再構成などの点から、脆弱性対策情報の利活用を促進することを目的としています。</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">https://jvndb.jvn.jp/en/schema/jvnrss.html</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">https://jvndb.jvn.jp/schema/jvnrss.html</xs:documentation>
|
||||
<xs:appinfo>
|
||||
<schema>JVN RDF Site Summary (JVNRSS)</schema>
|
||||
<author>Masato Terada</author>
|
||||
<version>3.2</version>
|
||||
<date>2017-07-20T03:16:00+09:00</date>
|
||||
</xs:appinfo>
|
||||
</xs:annotation>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Element Declarations -->
|
||||
<!-- ================================================== -->
|
||||
<xs:element name="channel">
|
||||
<xs:complexType>
|
||||
<xs:sequence>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:title"/>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:link"/>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:description"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:language"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:publisher"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:rights"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:creator"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:subject"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:identifier"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="sec:identifier"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:date"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dcterms:issued"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dcterms:modified"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="sec:handling"/>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:items"/>
|
||||
</xs:sequence>
|
||||
<xs:attribute ref="rdf:about" use="required"/>
|
||||
</xs:complexType>
|
||||
</xs:element>
|
||||
<xs:element name="items">
|
||||
<xs:complexType>
|
||||
<xs:sequence>
|
||||
<xs:element ref="rdf:Seq" minOccurs="0" maxOccurs="unbounded"/>
|
||||
</xs:sequence>
|
||||
</xs:complexType>
|
||||
</xs:element>
|
||||
<xs:element name="item">
|
||||
<xs:complexType>
|
||||
<xs:sequence>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:title"/>
|
||||
<xs:element minOccurs="1" maxOccurs="1" ref="rss:link"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="rss:description"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:language"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:publisher"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:rights"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:creator"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:subject"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:identifier"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="dc:relation"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="sec:identifier"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:references"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:cpe"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:cvss"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dc:date"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dcterms:issued"/>
|
||||
<xs:element minOccurs="0" maxOccurs="1" ref="dcterms:modified"/>
|
||||
</xs:sequence>
|
||||
<xs:attribute ref="rdf:about" use="required"/>
|
||||
</xs:complexType>
|
||||
</xs:element>
|
||||
<xs:element name="title" type="xs:string"/>
|
||||
<xs:element name="link" type="xs:anyURI"/>
|
||||
<xs:element name="description" type="xs:string"/>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Change History -->
|
||||
<!-- ================================================== -->
|
||||
<!--
|
||||
v1.0 2003-07-15
|
||||
http://jvnrss.ise.chuo-u.ac.jp/jtg/jvnrss/
|
||||
v2.0 2009-03-09
|
||||
2nd version for JVNRSS
|
||||
http://jvnrss.ise.chuo-u.ac.jp/jtg/jvnrss/jvnrss_2.0.xsd
|
||||
v2.1 2009-04-28
|
||||
Publish JVNRSS specification and XML schema on jvndb.jvn.jp
|
||||
https://jvndb.jvn.jp/schema/jvnrss_2.0.xsd
|
||||
v2.2 2009-07-17
|
||||
[add] dc:language, dc:subject
|
||||
v3.1 2011-08-17
|
||||
3.1 version for MyJVN API V3
|
||||
[change] refer (from status_3.0.xsd) to status_3.1.xsd
|
||||
https://jvndb.jvn.jp/schema/jvnrss_3.1.xsd
|
||||
v3.2 2017-07-20
|
||||
3.2 version for MyJVN API hnd
|
||||
[change] refer (from mod_sec_2.0.xsd) to mod_sec_3.0.xsd
|
||||
https://jvndb.jvn.jp/schema/jvnrss_3.2.xsd
|
||||
-->
|
||||
|
||||
</xs:schema>
|
||||
168
src/StellaOps.Concelier.Connector.Jvn/Schemas/mod_sec_3.0.xsd
Normal file
168
src/StellaOps.Concelier.Connector.Jvn/Schemas/mod_sec_3.0.xsd
Normal file
@@ -0,0 +1,168 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<xs:schema
|
||||
targetNamespace="http://jvn.jp/rss/mod_sec/3.0/"
|
||||
xmlns:sec="http://jvn.jp/rss/mod_sec/3.0/"
|
||||
xmlns:marking="http://data-marking.mitre.org/Marking-1"
|
||||
xmlns:xs="http://www.w3.org/2001/XMLSchema"
|
||||
elementFormDefault="qualified"
|
||||
attributeFormDefault="unqualified"
|
||||
version="3.0">
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema imports -->
|
||||
<!-- ================================================== -->
|
||||
<xs:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="xml.xsd"/>
|
||||
<xs:import namespace="http://data-marking.mitre.org/Marking-1" schemaLocation="data_marking.xsd"/>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema description -->
|
||||
<!-- ================================================== -->
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">mod_sec describes RSS Extension of security information
|
||||
distribution, and definition of the tags for RSS 1.0, 2.0 and Atom.</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">mod_sec は、脆弱性対策情報などのセキュリティ情報を記述するための JVNRSS 拡張仕様で、RSS
|
||||
1.0、RSS 2.0、Atom での利用を想定した汎用的な仕様となっています。</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">https://jvndb.jvn.jp/en/schema/mod_sec.html</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">https://jvndb.jvn.jp/schema/mod_sec.html</xs:documentation>
|
||||
<xs:appinfo>
|
||||
<schema>Qualified Security Advisory Reference (mod_sec)</schema>
|
||||
<author>Masato Terada</author>
|
||||
<version>3.0</version>
|
||||
<date>2017-07-20T03:16:00+09:00</date>
|
||||
</xs:appinfo>
|
||||
</xs:annotation>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Element Declarations -->
|
||||
<!-- ================================================== -->
|
||||
<xs:element name="items">
|
||||
<xs:complexType>
|
||||
<xs:sequence>
|
||||
<xs:element maxOccurs="unbounded" ref="sec:item"/>
|
||||
</xs:sequence>
|
||||
</xs:complexType>
|
||||
</xs:element>
|
||||
<xs:element name="item">
|
||||
<xs:complexType>
|
||||
<xs:sequence>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:title"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:identifier"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:summary"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:link"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:references"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:cpe"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:cvss"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:published"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:updated"/>
|
||||
<xs:element minOccurs="0" maxOccurs="unbounded" ref="sec:author"/>
|
||||
</xs:sequence>
|
||||
</xs:complexType>
|
||||
</xs:element>
|
||||
|
||||
<xs:element name="title" type="xs:string"/>
|
||||
<xs:element name="identifier" type="xs:string"/>
|
||||
<xs:element name="summary" type="xs:string"/>
|
||||
<xs:element name="link" type="sec:link"/>
|
||||
<xs:element name="references" type="sec:references"/>
|
||||
<xs:element name="cvss" type="sec:cvss"/>
|
||||
<xs:element name="cpe" type="sec:cpe"/>
|
||||
<xs:element name="published" type="xs:dateTime"/>
|
||||
<xs:element name="updated" type="xs:dateTime"/>
|
||||
<xs:element name="author" type="sec:author"/>
|
||||
<xs:element name="handling" type="marking:MarkingType">
|
||||
<xs:annotation>
|
||||
<xs:documentation>Specifies the relevant handling guidance for this STIX_Package. The
|
||||
valid marking scope is the nearest STIXPackageType ancestor of this Handling element
|
||||
and all its descendants.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:element>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Complex Type Definitions -->
|
||||
<!-- ================================================== -->
|
||||
<xs:complexType name="link">
|
||||
<xs:attribute name="href" type="xs:anyURI" use="required"/>
|
||||
<xs:attribute name="rel" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="type" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="hreflang" type="xs:NMTOKEN" use="optional"/>
|
||||
<xs:attribute name="title" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="length" type="xs:positiveInteger" use="optional"/>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="references">
|
||||
<xs:simpleContent>
|
||||
<xs:extension base="xs:anyURI">
|
||||
<xs:attribute name="source" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="id" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="title" type="xs:string" use="optional"/>
|
||||
</xs:extension>
|
||||
</xs:simpleContent>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="cvss">
|
||||
<xs:attribute name="version" type="xs:decimal" use="required"/>
|
||||
<xs:attribute name="type" type="sec:cvssTypeEnum" use="required"/>
|
||||
<xs:attribute name="severity" type="sec:cvssSeverityEnum" use="required"/>
|
||||
<xs:attribute name="score" type="xs:decimal" use="required"/>
|
||||
<xs:attribute name="vector" type="xs:string" use="optional"/>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="cpe">
|
||||
<xs:simpleContent>
|
||||
<xs:extension base="xs:string">
|
||||
<xs:attribute name="version" type="xs:decimal" use="required"/>
|
||||
<xs:attribute name="vendor" type="xs:string" use="required"/>
|
||||
<xs:attribute name="product" type="xs:string" use="optional"/>
|
||||
<xs:attribute name="impact" type="sec:cpeImpactEnum" use="optional"/>
|
||||
</xs:extension>
|
||||
</xs:simpleContent>
|
||||
</xs:complexType>
|
||||
<xs:complexType name="author">
|
||||
<xs:simpleContent>
|
||||
<xs:extension base="xs:string">
|
||||
<xs:attribute name="href" type="xs:anyURI" use="optional"/>
|
||||
</xs:extension>
|
||||
</xs:simpleContent>
|
||||
</xs:complexType>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Simple Type Definitions -->
|
||||
<!-- ================================================== -->
|
||||
<xs:simpleType name="cvssTypeEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="Base"/>
|
||||
<xs:enumeration value="Temporal"/>
|
||||
<xs:enumeration value="Environmental"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="cvssSeverityEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="None"/>
|
||||
<xs:enumeration value="Low"/>
|
||||
<xs:enumeration value="Medium"/>
|
||||
<xs:enumeration value="High"/>
|
||||
<xs:enumeration value="Critical"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="cpeImpactEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="not vulnerable"/>
|
||||
<xs:enumeration value="vulnerable"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Change History -->
|
||||
<!-- ================================================== -->
|
||||
<!--
|
||||
v1.0 2005-10-31
|
||||
http://jvnrss.ise.chuo-u.ac.jp/jtg/mod_sec/
|
||||
v2.0 2009-01-05
|
||||
2nd version for mod_sec
|
||||
http://jvnrss.ise.chuo-u.ac.jp/jtg/mod_sec/mod_sec_2.0.xsd
|
||||
v2.1 2009-04-28
|
||||
Publish mod_sec specification and XML schema on jvndb.jvn.jp
|
||||
https://jvndb.jvn.jp/schema/mod_sec_2.0.xsd
|
||||
v3.0 2017-07-20
|
||||
Publish mod_sec specification and XML schema on jvndb.jvn.jp
|
||||
https://jvndb.jvn.jp/schema/mod_sec_3.0.xsd
|
||||
-->
|
||||
|
||||
</xs:schema>
|
||||
574
src/StellaOps.Concelier.Connector.Jvn/Schemas/status_3.3.xsd
Normal file
574
src/StellaOps.Concelier.Connector.Jvn/Schemas/status_3.3.xsd
Normal file
@@ -0,0 +1,574 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<xs:schema
|
||||
targetNamespace="http://jvndb.jvn.jp/myjvn/Status"
|
||||
xmlns:status="http://jvndb.jvn.jp/myjvn/Status"
|
||||
xmlns:xs="http://www.w3.org/2001/XMLSchema"
|
||||
elementFormDefault="qualified"
|
||||
attributeFormDefault="unqualified"
|
||||
version="3.2">
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema imports -->
|
||||
<!-- ================================================== -->
|
||||
<xs:import namespace="http://www.w3.org/XML/1998/namespace" schemaLocation="xml.xsd"/>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Schema description -->
|
||||
<!-- ================================================== -->
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">This is an XML Schema for the status information of MyJVN API.</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">MyJVN API のステータス情報を格納する XML スキーマ</xs:documentation>
|
||||
<xs:appinfo>
|
||||
<schema>Status Information of MyJVN API</schema>
|
||||
<author>Masato Terada</author>
|
||||
<version>3.3</version>
|
||||
<date>2017-07-20T03:16:00+09:00</date>
|
||||
</xs:appinfo>
|
||||
</xs:annotation>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Element Declarations -->
|
||||
<!-- ================================================== -->
|
||||
<xs:element name="Status" type="status:Status"/>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Complex Type Definitions -->
|
||||
<!-- ================================================== -->
|
||||
<xs:complexType name="Status">
|
||||
|
||||
<!-- ////////////////////////// common response parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="version" type="xs:decimal" use="required" fixed="3.3">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; MyJVN API Schema Version - MyJVN API Ver 3.0 [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; MyJVN API スキーマバージョン - MyJNV API Ver 3.0 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="retCd" type="xs:integer" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Return Code/Interger (0:success, 1:failure) [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; リターンコード/整数値 (0:成功, 1:エラー) [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="retMax" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter; Maximum number of Entry/Interger [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ; エントリ上限値/整数値 (APIごとに規定されている一度に取得できるエントリ件数の上限値, エラー時は空文字列) [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="errCd" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Error Code (Null:success) [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; エラーコード (空文字列:成功) [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="errMsg" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Error Message (Null:success) [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; エラーメッセージ (空文字列:成功) [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="totalRes" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Total number of Result entries [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; 応答エントリ総数: 整数値 (フィルタリング条件に当てはまるエントリの総件数) ；エラー時は空文字列 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="totalResRet" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Number of Result entries [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; 応答エントリ数: 整数値 (フィルタリング条件に当てはまるエントリのうち、レスポンスに格納されている件数) ；エラー時は空文字列 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="firstRes" type="xs:string" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter; Start entry number in Result entries [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ; 応答エントリ開始位置: 整数値 (フィルタリング条件に当てはまるエントリのうち、何番目からのデータを取得したのかを示す値) ；エラー時は空文字列 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// common request parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="method" type="status:methodEnum" use="required">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter; Method [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ; メソッド名 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="lang" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter; Language (ja/en) [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ; 表示言語 (ja/en) [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="startItem" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start entry number [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: エントリ開始位置 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="maxCountItem" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Read entry number [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: エントリ取得件数 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="xsl" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: XSL file enable/disable [common]</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: XSL ファイル 適用／未適用 [共通]</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="feed" type="status:feedEnum" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: feed name</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: フェードフォーマット(=APIバージョン)を示す名称</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// partial common request parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="cpeName" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Vendor CPE Name/Product CPE Name</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: ベンダ CPE 名/製品 CPE 名</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="vendorId" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Vendor unique numbers</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: ベンダの識別番号一覧</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="productId" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Product unique numbers</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 製品の識別番号一覧</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="keyword" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Keyword</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: キーワード</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="type" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Type of OVAL</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: OVAL 種別</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getOvalList, getVulnOverviewStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="lt" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Type of feed limit</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: フィード制限タイプ</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVendorList, getProductList,getVulnOverviewList, getVulnDetailInfo</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getProductList parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="category" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Product type (01/02/03)</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 製品タイプ (01/02/03)</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getProductList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="ver" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: MyJVN API Version</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: MyJVN API Version</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getProductList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="latestProductUpdate" type="xs:dateTime" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Paramter; ReLatest date of product registration/update</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ: 製品登録/更新の最新日</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getProductList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getVulnOverviewList parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="severity" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Severity</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: CVSS 深刻度</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="vector" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Vector of CVSS Base metric</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: CVSS 基本評価基準ベクタ</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="rangeDatePublic" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Range of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日の範囲指定</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="rangeDatePublished" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Range of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日の範囲指定</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="rangeDateFirstPublished" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Range of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日の範囲指定</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicStartY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start year of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日開始年</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVulnOverviewList</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicStartM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start month of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日開始月</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVulnOverviewList</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicStartD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start day of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日開始日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicEndY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End year of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日終了年</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVulnOverviewList</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicEndM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End month of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日終了月</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVulnOverviewList</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublicEndD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End day of Date Public</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発見日終了日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedStartY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start year of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日開始年</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedStartM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Star month of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日開始月</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedStartD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start day of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日開始日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedEndY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End year of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日終了年</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedEndM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End month of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日終了月</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="datePublishedEndD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End day of Date Last Updated</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日終了日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedStartY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start year of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日開始年</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedStartM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start month of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日開始月</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedStartD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Start day of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日開始日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedEndY" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End year of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日終了年</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedEndM" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End month of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日終了月</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="dateFirstPublishedEndD" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: End day of Date First Published</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日終了日</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getVulnDetailInfo parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="vulnId" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Vulnerability ID</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 脆弱性対策情報 ID</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getVulnDetailInfo</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getCvrfInfo parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="cvrfdoc" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Vulnerability ID</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 脆弱性対策情報 ID</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getCvrfInfo</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getOvalList parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="platform" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Type of OS</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: OS 種別</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getOvalList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="internalType" type="status:internalTypeEnum" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Type of OVAL definition</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: OVAL定義のタイプ</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getOvalList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="mode" type="status:modeEnum" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Type of Application condition</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: アプリケーションの動作モード</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getOvalList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getOvalData parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="ovalId" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: OVAL ID</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: OVAL ID</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getOvalData</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getXccdfData parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="benchmarkId" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Benchmark ID</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: ベンチマーク ID</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getXccdfCheckData</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
|
||||
<!-- ////////////////////////// getStatistics parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="theme" type="status:themeEnum" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Graph theme</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: グラフ テーマ</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="retMaxCnt" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Response Parameter: Maxium number of cntAll</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">レスポンスパラメタ: cntAll の最大値</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="cweId" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: CWE ID</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: CWE 識別子</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getStatistics</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
<xs:attribute name="pid" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Product unique numbers</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 製品の識別番号一覧</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<!-- ////////////////////////// getCPEdictionary parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="reference" type="xs:string" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: reference</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 参考情報</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getCPEdictionary</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<!-- ////////////////////////// getAlertList parameter ////////////////////////// -->
|
||||
|
||||
<xs:attribute name="datePublished" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Date Last Updated (Year 4digits)</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 更新日年</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getAlertList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attribute name="dateFirstPublished" type="xs:integer" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: Date First Published (Year 4digits)</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 発行日年</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getAlertList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attribute name="ft" type="status:ftEnum" use="optional">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Request Parameter: reference</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">リクエストパラメタ: 参考情報</xs:documentation>
|
||||
<xs:documentation xml:lang="en-US">method=getAlertList</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
</xs:complexType>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Simple Type Definitions -->
|
||||
<!-- ================================================== -->
|
||||
<xs:simpleType name="versionPattern">
|
||||
<xs:annotation>
|
||||
<xs:documentation xml:lang="en-US">Define the version Number of Status XSD</xs:documentation>
|
||||
<xs:documentation xml:lang="ja">Status XSD のバージョン番号</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:pattern value="[0-9].[0-9]"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="methodEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="getVendorList"/>
|
||||
<xs:enumeration value="getProductList"/>
|
||||
<xs:enumeration value="getVulnOverviewList"/>
|
||||
<xs:enumeration value="getVulnDetailInfo"/>
|
||||
<xs:enumeration value="getCvrfInfo"/>
|
||||
<xs:enumeration value="getOvalList"/>
|
||||
<xs:enumeration value="getOvalData"/>
|
||||
<xs:enumeration value="getXccdfList"/>
|
||||
<xs:enumeration value="getXccdfData"/>
|
||||
<xs:enumeration value="getStatistics"/>
|
||||
<xs:enumeration value="getCPEdictionary"/>
|
||||
<xs:enumeration value="getAlertList"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="internalTypeEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="inventory"/>
|
||||
<xs:enumeration value="vulnerability"/>
|
||||
<xs:enumeration value="compliance"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="modeEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="x32"/>
|
||||
<xs:enumeration value="x64"/>
|
||||
<xs:enumeration value="wow64"/>
|
||||
<xs:enumeration value="x32-x64"/>
|
||||
<xs:enumeration value="x32-wow64"/>
|
||||
<xs:enumeration value="x64-wow64"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="themeEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="sumAll"/>
|
||||
<xs:enumeration value="sumJvnDb"/>
|
||||
<xs:enumeration value="sumCvss"/>
|
||||
<xs:enumeration value="sumCwe"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="feedEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="hnd"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
<xs:simpleType name="ftEnum">
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="xml"/>
|
||||
<xs:enumeration value="json"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
|
||||
<!-- ================================================== -->
|
||||
<!-- ===== Change History -->
|
||||
<!-- ================================================== -->
|
||||
<!--
|
||||
v1.0
|
||||
Draft version for MyJVN
|
||||
v2.0 2008-11-11
|
||||
Initial version for MyJVN
|
||||
http://jvnrss.ise.chuo-u.ac.jp/jtg/apis/status_2.0.xsd
|
||||
v3.0 2009-09-30
|
||||
3rd version of MyJVN
|
||||
https://jvndb.jvn.jp/schema/status_3.0.xsd
|
||||
v3.1 2011-08-17
|
||||
3.1 version for MyJVN API V3
|
||||
https://jvndb.jvn.jp/schema/status_3.1.xsd
|
||||
v3.2 2014-08-11
|
||||
3.2 version for MyJVN API V3
|
||||
https://jvndb.jvn.jp/schema/status_3.2.xsd
|
||||
v3.3 2017-07-20
|
||||
3.3 version for MyJVN API V3
|
||||
https://jvndb.jvn.jp/schema/status_3.3.xsd
|
||||
[add] getAlertList
|
||||
-->
|
||||
|
||||
</xs:schema>
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:marking="http://data-marking.mitre.org/Marking-1" xmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1" targetNamespace="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1" elementFormDefault="qualified" attributeFormDefault="unqualified" version="1.1.1" xml:lang="English">
|
||||
<xs:annotation>
|
||||
<xs:documentation>This schema was originally developed by The MITRE Corporation. The Data Marking Schema implementation is maintained by The MITRE Corporation and developed by the open STIX Community. For more information, including how to get involved in the effort and how to submit change requests, please visit the STIX website at http://stix.mitre.org. </xs:documentation>
|
||||
<xs:appinfo>
|
||||
<schema>Data Marking Extension - TLP</schema>
|
||||
<version>1.1.1</version>
|
||||
<date>05/08/2014 9:00:00 AM</date>
|
||||
<short_description>Data Marking Extension - TLP Marking Instance - Schematic implementation for attaching a Traffic Light Protocol (TLP)designation to an idendified XML structure.</short_description>
|
||||
<terms_of_use>Copyright (c) 2012-2014, The MITRE Corporation. All rights reserved. The contents of this file are subject to the terms of the STIX License located at http://stix.mitre.org/about/termsofuse.html. See the STIX License for the specific language governing permissions and limitations for use of this schema. When distributing copies of the STIX Schema, this license header must be included. </terms_of_use>
|
||||
</xs:appinfo>
|
||||
</xs:annotation>
|
||||
<xs:import namespace="http://data-marking.mitre.org/Marking-1" schemaLocation="data_marking.xsd"/>
|
||||
<xs:complexType name="TLPMarkingStructureType">
|
||||
<xs:annotation>
|
||||
<xs:documentation>The TLPMarkingStructureType is an implementation of the data marking schema that allows for a TLP Designation to be attached to an identified XML structure. Information about TLP is available here: http://www.us-cert.gov/tlp.</xs:documentation>
|
||||
<xs:documentation>Nodes may be marked by multiple TLP Marking statements. When this occurs, the node should be considered marked at the most restrictive TLP Marking of all TLP Markings that were applied to it. For example, if a node is marked both GREEN and AMBER, the node should be considered AMBER.</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:complexContent>
|
||||
<xs:extension base="marking:MarkingStructureType">
|
||||
<xs:attribute name="color" type="tlpMarking:TLPColorEnum">
|
||||
<xs:annotation>
|
||||
<xs:documentation>The TLP color designation of the marked structure.</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
</xs:extension>
|
||||
</xs:complexContent>
|
||||
</xs:complexType>
|
||||
<xs:simpleType name="TLPColorEnum">
|
||||
<xs:annotation>
|
||||
<xs:documentation>The TLP color designation of the marked structure.</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value="RED"/>
|
||||
<xs:enumeration value="AMBER"/>
|
||||
<xs:enumeration value="GREEN"/>
|
||||
<xs:enumeration value="WHITE"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
</xs:schema>
|
||||
1566
src/StellaOps.Concelier.Connector.Jvn/Schemas/vuldef_3.2.xsd
Normal file
1566
src/StellaOps.Concelier.Connector.Jvn/Schemas/vuldef_3.2.xsd
Normal file
File diff suppressed because it is too large
Load Diff
287
src/StellaOps.Concelier.Connector.Jvn/Schemas/xml.xsd
Normal file
287
src/StellaOps.Concelier.Connector.Jvn/Schemas/xml.xsd
Normal file
@@ -0,0 +1,287 @@
|
||||
<?xml version='1.0'?>
|
||||
<?xml-stylesheet href="../2008/09/xsd.xsl" type="text/xsl"?>
|
||||
<xs:schema targetNamespace="http://www.w3.org/XML/1998/namespace"
|
||||
xmlns:xs="http://www.w3.org/2001/XMLSchema"
|
||||
xmlns ="http://www.w3.org/1999/xhtml"
|
||||
xml:lang="en">
|
||||
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
<h1>About the XML namespace</h1>
|
||||
|
||||
<div class="bodytext">
|
||||
<p>
|
||||
This schema document describes the XML namespace, in a form
|
||||
suitable for import by other schema documents.
|
||||
</p>
|
||||
<p>
|
||||
See <a href="http://www.w3.org/XML/1998/namespace.html">
|
||||
http://www.w3.org/XML/1998/namespace.html</a> and
|
||||
<a href="http://www.w3.org/TR/REC-xml">
|
||||
http://www.w3.org/TR/REC-xml</a> for information
|
||||
about this namespace.
|
||||
</p>
|
||||
<p>
|
||||
Note that local names in this namespace are intended to be
|
||||
defined only by the World Wide Web Consortium or its subgroups.
|
||||
The names currently defined in this namespace are listed below.
|
||||
They should not be used with conflicting semantics by any Working
|
||||
Group, specification, or document instance.
|
||||
</p>
|
||||
<p>
|
||||
See further below in this document for more information about <a
|
||||
href="#usage">how to refer to this schema document from your own
|
||||
XSD schema documents</a> and about <a href="#nsversioning">the
|
||||
namespace-versioning policy governing this schema document</a>.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
|
||||
<xs:attribute name="lang">
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
|
||||
<h3>lang (as an attribute name)</h3>
|
||||
<p>
|
||||
denotes an attribute whose value
|
||||
is a language code for the natural language of the content of
|
||||
any element; its value is inherited. This name is reserved
|
||||
by virtue of its definition in the XML specification.</p>
|
||||
|
||||
</div>
|
||||
<div>
|
||||
<h4>Notes</h4>
|
||||
<p>
|
||||
Attempting to install the relevant ISO 2- and 3-letter
|
||||
codes as the enumerated possible values is probably never
|
||||
going to be a realistic possibility.
|
||||
</p>
|
||||
<p>
|
||||
See BCP 47 at <a href="http://www.rfc-editor.org/rfc/bcp/bcp47.txt">
|
||||
http://www.rfc-editor.org/rfc/bcp/bcp47.txt</a>
|
||||
and the IANA language subtag registry at
|
||||
<a href="http://www.iana.org/assignments/language-subtag-registry">
|
||||
http://www.iana.org/assignments/language-subtag-registry</a>
|
||||
for further information.
|
||||
</p>
|
||||
<p>
|
||||
The union allows for the 'un-declaration' of xml:lang with
|
||||
the empty string.
|
||||
</p>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:simpleType>
|
||||
<xs:union memberTypes="xs:language">
|
||||
<xs:simpleType>
|
||||
<xs:restriction base="xs:string">
|
||||
<xs:enumeration value=""/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
</xs:union>
|
||||
</xs:simpleType>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attribute name="space">
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
|
||||
<h3>space (as an attribute name)</h3>
|
||||
<p>
|
||||
denotes an attribute whose
|
||||
value is a keyword indicating what whitespace processing
|
||||
discipline is intended for the content of the element; its
|
||||
value is inherited. This name is reserved by virtue of its
|
||||
definition in the XML specification.</p>
|
||||
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
<xs:simpleType>
|
||||
<xs:restriction base="xs:NCName">
|
||||
<xs:enumeration value="default"/>
|
||||
<xs:enumeration value="preserve"/>
|
||||
</xs:restriction>
|
||||
</xs:simpleType>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attribute name="base" type="xs:anyURI"> <xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
|
||||
<h3>base (as an attribute name)</h3>
|
||||
<p>
|
||||
denotes an attribute whose value
|
||||
provides a URI to be used as the base for interpreting any
|
||||
relative URIs in the scope of the element on which it
|
||||
appears; its value is inherited. This name is reserved
|
||||
by virtue of its definition in the XML Base specification.</p>
|
||||
|
||||
<p>
|
||||
See <a
|
||||
href="http://www.w3.org/TR/xmlbase/">http://www.w3.org/TR/xmlbase/</a>
|
||||
for information about this attribute.
|
||||
</p>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attribute name="id" type="xs:ID">
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
|
||||
<h3>id (as an attribute name)</h3>
|
||||
<p>
|
||||
denotes an attribute whose value
|
||||
should be interpreted as if declared to be of type ID.
|
||||
This name is reserved by virtue of its definition in the
|
||||
xml:id specification.</p>
|
||||
|
||||
<p>
|
||||
See <a
|
||||
href="http://www.w3.org/TR/xml-id/">http://www.w3.org/TR/xml-id/</a>
|
||||
for information about this attribute.
|
||||
</p>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
</xs:attribute>
|
||||
|
||||
<xs:attributeGroup name="specialAttrs">
|
||||
<xs:attribute ref="xml:base"/>
|
||||
<xs:attribute ref="xml:lang"/>
|
||||
<xs:attribute ref="xml:space"/>
|
||||
<xs:attribute ref="xml:id"/>
|
||||
</xs:attributeGroup>
|
||||
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div>
|
||||
|
||||
<h3>Father (in any context at all)</h3>
|
||||
|
||||
<div class="bodytext">
|
||||
<p>
|
||||
denotes Jon Bosak, the chair of
|
||||
the original XML Working Group. This name is reserved by
|
||||
the following decision of the W3C XML Plenary and
|
||||
XML Coordination groups:
|
||||
</p>
|
||||
<blockquote>
|
||||
<p>
|
||||
In appreciation for his vision, leadership and
|
||||
dedication the W3C XML Plenary on this 10th day of
|
||||
February, 2000, reserves for Jon Bosak in perpetuity
|
||||
the XML name "xml:Father".
|
||||
</p>
|
||||
</blockquote>
|
||||
</div>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div xml:id="usage" id="usage">
|
||||
<h2><a name="usage">About this schema document</a></h2>
|
||||
|
||||
<div class="bodytext">
|
||||
<p>
|
||||
This schema defines attributes and an attribute group suitable
|
||||
for use by schemas wishing to allow <code>xml:base</code>,
|
||||
<code>xml:lang</code>, <code>xml:space</code> or
|
||||
<code>xml:id</code> attributes on elements they define.
|
||||
</p>
|
||||
<p>
|
||||
To enable this, such a schema must import this schema for
|
||||
the XML namespace, e.g. as follows:
|
||||
</p>
|
||||
<pre>
|
||||
<schema . . .>
|
||||
. . .
|
||||
<import namespace="http://www.w3.org/XML/1998/namespace"
|
||||
schemaLocation="http://www.w3.org/2001/xml.xsd"/>
|
||||
</pre>
|
||||
<p>
|
||||
or
|
||||
</p>
|
||||
<pre>
|
||||
<import namespace="http://www.w3.org/XML/1998/namespace"
|
||||
schemaLocation="http://www.w3.org/2009/01/xml.xsd"/>
|
||||
</pre>
|
||||
<p>
|
||||
Subsequently, qualified reference to any of the attributes or the
|
||||
group defined below will have the desired effect, e.g.
|
||||
</p>
|
||||
<pre>
|
||||
<type . . .>
|
||||
. . .
|
||||
<attributeGroup ref="xml:specialAttrs"/>
|
||||
</pre>
|
||||
<p>
|
||||
will define a type which will schema-validate an instance element
|
||||
with any of those attributes.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
|
||||
<xs:annotation>
|
||||
<xs:documentation>
|
||||
<div id="nsversioning" xml:id="nsversioning">
|
||||
<h2><a name="nsversioning">Versioning policy for this schema document</a></h2>
|
||||
<div class="bodytext">
|
||||
<p>
|
||||
In keeping with the XML Schema WG's standard versioning
|
||||
policy, this schema document will persist at
|
||||
<a href="http://www.w3.org/2009/01/xml.xsd">
|
||||
http://www.w3.org/2009/01/xml.xsd</a>.
|
||||
</p>
|
||||
<p>
|
||||
At the date of issue it can also be found at
|
||||
<a href="http://www.w3.org/2001/xml.xsd">
|
||||
http://www.w3.org/2001/xml.xsd</a>.
|
||||
</p>
|
||||
<p>
|
||||
The schema document at that URI may however change in the future,
|
||||
in order to remain compatible with the latest version of XML
|
||||
Schema itself, or with the XML namespace itself. In other words,
|
||||
if the XML Schema or XML namespaces change, the version of this
|
||||
document at <a href="http://www.w3.org/2001/xml.xsd">
|
||||
http://www.w3.org/2001/xml.xsd
|
||||
</a>
|
||||
will change accordingly; the version at
|
||||
<a href="http://www.w3.org/2009/01/xml.xsd">
|
||||
http://www.w3.org/2009/01/xml.xsd
|
||||
</a>
|
||||
will not change.
|
||||
</p>
|
||||
<p>
|
||||
Previous dated (and unchanging) versions of this schema
|
||||
document are at:
|
||||
</p>
|
||||
<ul>
|
||||
<li><a href="http://www.w3.org/2009/01/xml.xsd">
|
||||
http://www.w3.org/2009/01/xml.xsd</a></li>
|
||||
<li><a href="http://www.w3.org/2007/08/xml.xsd">
|
||||
http://www.w3.org/2007/08/xml.xsd</a></li>
|
||||
<li><a href="http://www.w3.org/2004/10/xml.xsd">
|
||||
http://www.w3.org/2004/10/xml.xsd</a></li>
|
||||
<li><a href="http://www.w3.org/2001/03/xml.xsd">
|
||||
http://www.w3.org/2001/03/xml.xsd</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</xs:documentation>
|
||||
</xs:annotation>
|
||||
|
||||
</xs:schema>
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<EmbeddedResource Include="Schemas\*.xsd" />
|
||||
<ProjectReference Include="../StellaOps.Plugin/StellaOps.Plugin.csproj" />
|
||||
<ProjectReference Include="..\StellaOps.Concelier.Models\StellaOps.Concelier.Models.csproj" />
|
||||
<ProjectReference Include="..\StellaOps.Concelier.Normalization\StellaOps.Concelier.Normalization.csproj" />
|
||||
<ProjectReference Include="..\StellaOps.Concelier.Connector.Common\StellaOps.Concelier.Connector.Common.csproj" />
|
||||
<ProjectReference Include="..\StellaOps.Concelier.Storage.Mongo\StellaOps.Concelier.Storage.Mongo.csproj" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
13
src/StellaOps.Concelier.Connector.Jvn/TASKS.md
Normal file
13
src/StellaOps.Concelier.Connector.Jvn/TASKS.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# TASKS
|
||||
| Task | Owner(s) | Depends on | Notes |
|
||||
|---|---|---|---|
|
||||
|MyJVN client (JVNRSS+VULDEF) with windowing|BE-Conn-JVN|Source.Common|**DONE** – windowed overview/detail fetch with rate limit handling implemented.|
|
||||
|Schema/XSD validation and DTO sanitizer|BE-Conn-JVN, QA|Source.Common|**DONE** – parser validates XML against schema before DTO persistence.|
|
||||
|Canonical mapping (aliases, jp_flags, refs)|BE-Conn-JVN|Models|**DONE** – mapper populates aliases, jp_flags, references while skipping non-actionable affected entries.|
|
||||
|SourceState and idempotent dedupe|BE-Conn-JVN|Storage.Mongo|**DONE** – cursor tracks pending docs/mappings with resume support.|
|
||||
|Golden fixtures and determinism tests|QA|Source.Jvn|**DONE** – deterministic snapshot test in `JvnConnectorTests` now passes with offline fixtures.|
|
||||
|Async-safe overview query building|BE-Conn-JVN|Source.Common|DONE – `MyJvnClient` now builds query strings synchronously without blocking calls.|
|
||||
|Reference dedupe + deterministic ordering|BE-Conn-JVN|Models|DONE – mapper merges by URL, retains richer metadata, sorts deterministically.|
|
||||
|Console logging remediation|BE-Conn-JVN|Observability|**DONE** – connector now uses structured `ILogger` debug entries instead of console writes.|
|
||||
|Offline fixtures for connector tests|QA|Source.Jvn|**DONE** – tests rely solely on canned HTTP responses and local fixtures.|
|
||||
|Update VULDEF schema for vendor attribute|BE-Conn-JVN, QA|Source.Jvn|**DONE** – embedded XSD updated (vendor/product attrs, impact item), parser tightened, fixtures & snapshots refreshed.|
|
||||
Reference in New Issue
Block a user