git.stella-ops.org/docs/workflow/engine/16-elksharp-rendering-architecture.md

# ElkSharp Rendering Architecture

## Overview

ElkSharp is a deterministic, in-process Sugiyama-based graph layout engine for workflow
visualization. It replaces external layout dependencies (ElkJs/Node.js) with a pure C#
implementation that produces identical output for identical input, regardless of host
platform, thread scheduling, or execution environment.

The engine handles the full pipeline from abstract workflow graphs to positioned nodes
and routed edges, suitable for SVG/PNG/JSON rendering. It supports left-to-right and
top-to-bottom layout directions, three effort levels (Draft, Balanced, Best), compound
node hierarchies, and gateway-specific polygon geometry (diamond decisions, hexagon
forks/joins).

---

## Sugiyama Pipeline

The core layout algorithm follows the Sugiyama framework for layered graph drawing,
extended with workflow-specific constraints.

### 1. Layer Assignment

Depth-first traversal assigns a layer index to each node. The layer index determines
the horizontal position (in left-to-right mode) or vertical position (in top-to-bottom
mode) of each node.

- Start nodes are assigned layer 0.
- Each successor is assigned `max(predecessor layers) + 1`.
- Backward edges (cycles, repeat connectors) are identified and handled separately
  so they do not influence forward layer assignment.

### 2. Dummy Node Insertion

Edges that span more than one layer are split into chains of single-layer segments.
Each intermediate layer receives a "dummy node" -- a zero-size virtual node that
serves as a routing waypoint.

- Dummy nodes inherit the edge's channel classification (forward, backward, sink).
- After routing, dummy chains are reconstructed back into the original multi-layer
  edge with bend points at each dummy position.

### 3. Node Ordering

Barycenter-based median ordering minimizes edge crossings within each layer.

- The algorithm sweeps forward and backward across layers, computing each node's
  barycenter (average position of connected nodes in adjacent layers).
- Nodes are sorted by barycenter within their layer.
- Iteration count depends on effort level:
  - Draft: 8 iterations
  - Balanced: 14 iterations
  - Best: 24 iterations
- Tie-breaking is deterministic (stable sort by node ID) to ensure reproducibility.

### 4. Initial Placement

After ordering, nodes receive Y-coordinates (in left-to-right mode) based on the
median of incoming connection Y-centers.

- Enforced linear spacing ensures minimum `NodeSpacing` between adjacent nodes.
- Nodes with no incoming connections use the median of outgoing connection positions.
- Grid alignment snaps positions to the nearest grid line for visual consistency.

### 5. Placement Refinement

Multiple refinement passes adjust positions to reduce visual clutter:

- **Preferred-center pull**: Nodes shift toward the center of their connected
  neighbors' Y-range, weighted by connection count.
- **Snap-to-grid**: Positions align to a grid derived from `NodeSpacing`.
- **Compact-toward-incoming**: Nodes pull toward their primary incoming edge
  direction to reduce edge length and crossings.

Refinement iteration count scales with effort level (3 / 6 / 10 for Draft / Balanced / Best).

### 6. Y-Gutter Expansion

After edge routing identifies under-node violations (edges running through or
alongside nodes), the engine shifts entire Y-bands downward to create routing
corridors.

- Runs after X-gutter expansion and before compact passes.
- Scans routed edges for horizontal segments that violate under-node or alongside
  clearance rules.
- Identifies blocking nodes and computes the required clearance gap.
- Shifts ALL nodes below the violation Y downward by the clearance amount.
  This preserves relative ordering within each layer.
- Re-routes edges with the expanded corridors.
- Up to 2 iterations to handle cascading violations.

The key insight is that shifting individual nodes disrupts the Sugiyama median-based
optimization, causing cascading layout degradation. Shifting entire Y-bands (like
X-gutter expansion does for inter-layer gaps) preserves within-layer relationships.

### 7. X-Gutter Expansion

Inter-layer horizontal gaps are widened to provide edge corridor space.

- The base gap is `LayerSpacing` (default 60px).
- When edge density between two layers exceeds a threshold, the gap is scaled
  up to 1.8x to accommodate additional routing channels.
- Expansion is computed before edge routing so that the router has adequate space
  for orthogonal paths.

---

## Edge Routing

### Channel Assignment

Each edge is classified into one of three routing channels:

- **Forward**: Source layer < target layer (the common case).
- **Backward**: Source layer > target layer (repeat/loop connectors).
- **Sink**: Edges to terminal nodes (End events) that may use special corridors.

Channel classification determines routing priority, corridor eligibility, and
post-processing rules.

### Base Routing

Orthogonal bend-point construction builds an initial route from source to target:

1. Exit the source node perpendicular to its boundary.
2. Route horizontally through inter-layer gutters.
3. Enter the target node perpendicular to its boundary.
4. Insert 90-degree bends at each direction change.

The base router respects node obstacles, avoiding routes that cross through node
rectangles or polygons.

### Dummy Edge Reconstruction

After routing, multi-layer dummy chains are merged back to original edges:

- Bend points from each dummy segment are concatenated.
- Redundant collinear points are removed.
- The result is a single edge with bend points at each layer transition.

### Anchor Snapping

Edge endpoints are projected onto actual node shape boundaries:

- **Rectangle**: Standard side intersection (left, right, top, bottom).
- **Diamond** (decision gateways): Intersection with the diamond's four edges,
  producing diagonal approach stubs.
- **Hexagon** (fork/join gateways): Intersection with the hexagon's six edges,
  with asymmetric shoulder geometry.

Anchor snapping runs after routing so that bend points near the boundary
produce clean visual connections.

---

## Iterative Optimization (Hybrid Deterministic Path)

The `Best` effort level activates hybrid deterministic optimization, which repairs
routing violations without disrupting the Sugiyama node placement.

### 1. Baseline Evaluation

The baseline route is scored using an 18-category violation taxonomy. Each edge
receives a violation list with severity-weighted penalties. The total score is the
sum of all edge scores.

### 2. Repair Planning

Penalized edges are identified from the violation severity map. The planner
extracts the specific violations for each edge and determines which edges need
repair and in what priority order.

High-severity violations (node crossings, under-node, shared lanes) take priority
over medium-severity (backtracking, detours) and soft-severity (edge crossings,
proximity, bends).

### 3. Conflict-Zone Batching

Independent repair candidates are grouped by shared source, shared target, or
shared corridor zone. Edges in the same conflict zone are batched together so
that their repairs are coordinated rather than competing.

Batching ensures that fixing one edge does not create a new violation for a
nearby edge in the same zone.

### 4. Parallel A* Candidate Construction

Each repair batch constructs candidate reroutes using A* 8-direction pathfinding:

- Candidates are built on full-core parallel threads.
- Each candidate is a complete reroute for the batch's edge set.
- The A* grid derives intermediate spacing from approximately one-third of the
  average service-task node size.
- Node-obstacle blocked step masks are precomputed per route so neighbor
  expansion does not rescan every node.
- Merge back into the route is deterministic and single-threaded.

### 5. Winner Refinement

The best candidate from each batch undergoes a refinement pipeline:

1. **Under-node repair**: Shift lanes that pass through node bounding boxes.
2. **Local-bundle spread**: Separate parallel edges that share the same lane.
3. **Shared-lane elimination**: Push edges apart that overlap on the same axis.
4. **Boundary-slot snap**: Align endpoints to the discrete slot lattice.
5. **Detour collapse**: Remove unnecessary overshoots where a shorter path exists.
6. **Post-slot restabilization**: Re-validate slot assignments after detour changes.
7. **Corridor reroute**: Move long horizontal sweeps to top/bottom corridors.
8. **Elevation adjustment**: Shift edges vertically to clear obstructions.
9. **Target-join spread**: Push convergent approach lanes apart by
   `minClearance - currentGap + 8px` (half applied to each edge).

Winner promotion uses weighted score comparison (`Score.Value`) to ensure
the refinement actually improved the layout.

---

## Gateway Geometry

Gateways use non-rectangular shapes that require specialized boundary logic.

### Decision Gateway (Diamond)

- 4 vertices: left tip, top, right tip, bottom.
- Left and right tips are the horizontal extremes.
- Top and bottom are the vertical extremes.
- Source exits leave from face interiors (not tips).
- Target entries may use left/right tips as convergence points.
- `ForceDecisionSourceExitOffVertex` blocks source exits from tip vertices.

### Fork/Join Gateway (Hexagon)

- 6 vertices: left tip, upper-left shoulder, upper-right shoulder, right tip,
  lower-right shoulder, lower-left shoulder.
- Shoulders create flat top and bottom faces suitable for multiple slot entries.
- Asymmetric geometry: the shoulder offset from the tip varies by gateway size.

### Boundary Slot Capacity

| Shape | Face | Max Slots |
|-------|------|-----------|
| Gateway (diamond/hexagon) | Any face | 2 |
| Rectangle | Left / Right | 3 |
| Rectangle | Top / Bottom | 5 |

Slots are evenly distributed within the face's safe boundary inset. Scoring and
final repair share the same realizable slot coordinates.

### Gateway Vertex Entry Rules

Left and right tip vertices are allowed for target entries but blocked for source
exits. This is enforced by a 3-way coordination mechanism:

1. **IsAllowedGatewayTipVertex**: Returns `true` for left/right tips when the
   edge is a target entry (incoming).
2. **HasValidGatewayBoundaryAngle**: Accepts any external approach angle at
   allowed tip vertices.
3. **CountBoundarySlotViolations**: Skips slot-occupancy checks when all entries
   at a vertex share the same allowed tip point.

All three checks must stay synchronized -- changing one without the others causes
cascading boundary-slot violations.

---

## Scoring System

### Violation Categories

The scoring system uses 18 violation categories with severity-weighted penalties.

#### Hard Violations (100,000 per instance)

| Category | Description |
|----------|-------------|
| Node crossings | Edge segment passes through a node bounding box |
| Under-node | Edge runs beneath or through a node's vertical extent |
| Shared lanes | Two edges share the same routing lane segment |
| Boundary slots | More edges than slots on a node face |
| Target joins | Multiple edges converge to the same target arrival point |
| Gateway exits | Source exit from a blocked gateway vertex |
| Collector corridors | Repeat-collector lane conflicts |
| Below-graph | Edge segment routes below the graph's maximum Y extent |

#### Medium Violations (50,000 per instance)

| Category | Description |
|----------|-------------|
| Backtracking | Edge reverses direction in the target-approach window |
| Detours | Unnecessary overshoot where a shorter path exists |

#### Soft Violations (200-650 per instance)

| Category | Description |
|----------|-------------|
| Edge crossings | Two edge segments intersect (200 per crossing) |
| Proximity | Edge passes too close to a node boundary (400) |
| Labels | Edge label overlaps another element (300) |
| Bends | Excessive number of bend points (200 per extra bend) |
| Diagonals | Non-orthogonal segment exceeds one node-shape length (650) |

### FinalScore Adjustments

The FinalScore applies detection exclusions that are NOT used during the iterative
search. This separation is critical: the search uses the raw scoring as its
heuristic, and changing it alters the search trajectory (causing speed regressions).

FinalScore excludes these borderline detection artifacts:

- **Valid gateway face approaches**: The exterior approach point is closer to
  the face center than the predecessor bend point (a legitimate face entry,
  not a violation).
- **Gateway-exit under-node**: The lane runs within 16px of the source node's
  bottom boundary (a tight but valid exit, not a true under-node crossing).
- **Convergent target joins from distant sources**: Sources separated by > 15px
  on the Y-axis with significant X separation (natural convergence, not a
  shared-lane conflict).
- **Borderline shared lanes**: Gap between parallel edges is within 3px of
  the tolerance threshold (measurement noise, not a real overlap).

---

## Corridor Routing

Long-range edges that would cross many intermediate nodes are routed through
corridors outside the main node field.

### Top Corridor (Long Sweeps)

For forward edges spanning more than 40% of the graph width:

- Route through the top corridor at `graphMinY - 56`.
- Exit the source with a 24px perpendicular stub.
- Route horizontally across the top corridor.
- Descend to the target.

The 24px exit stub is critical: it prevents `NormalizeBoundaryAngles` from
collapsing the vertical corridor segment back into the source boundary.

### Bottom Corridor (Near-Boundary Sweeps)

For edges that need to route near the graph's lower boundary:

- Route through the bottom corridor at `graphMaxY + 32`.
- Same perpendicular exit stub pattern.

### Below-Graph Detection

The below-graph violation detector (`HasCorridorBendPoints`) exempts edges that
intentionally use corridor routing. Without this exemption, corridor edges would
be penalized as below-graph violations and rerouted back into the node field.

---

## Y-Gutter Expansion (Routing-Aware Placement Feedback Loop)

Y-gutter expansion is a post-routing placement correction that creates vertical
routing space where the initial Sugiyama placement left insufficient clearance.

### Algorithm

1. **Detection**: After edge routing, scan all routed edge segments for horizontal
   segments that violate under-node or alongside clearance rules.

2. **Identification**: For each violation, identify the blocking node and compute
   the required clearance:
   - Under-node: The edge passes through the node's bounding box.
   - Alongside (flush): The edge runs within +/-4px of a node's top or bottom
     boundary (the "alongside" extension catches edges "glued" to boundaries
     that the standard gap > 0.5px check misses).

3. **Expansion**: Shift ALL nodes below the violation Y downward by the computed
   clearance amount. This preserves relative ordering within each layer, unlike
   individual node shifting which disrupts Sugiyama optimization.

4. **Re-routing**: Re-route edges with the expanded corridors.

5. **Iteration**: Repeat up to 2 times to handle cascading violations (where
   fixing one violation exposes another).

### Design Rationale

The same pattern is used for X-gutter expansion (widening inter-layer gaps).
Band-level shifting is fundamentally different from individual node adjustment:

- Individual node shifts break the barycenter ordering that the Sugiyama
  algorithm optimized, causing cascading position changes.
- Post-refinement clearance insertion fails because subsequent optimization
  passes override the inserted space.
- Band-level shifts preserve within-layer relationships while creating the
  needed routing corridors.

### Timing

Y-gutter expansion runs:
- AFTER X-gutter expansion (inter-layer gaps are already set).
- BEFORE compact passes (so compaction respects the new corridors).
- BEFORE the iterative optimization loop (so the optimizer works with
  adequate routing space).

---

## Effort Levels

| Level | Ordering Iterations | Placement Iterations | Routing |
|-------|--------------------:|---------------------:|---------|
| Draft | 8 | 3 | Baseline only |
| Balanced | 14 | 6 | Baseline + light repair |
| Best | 24 | 10 | Hybrid deterministic with full-core parallel repair |

- **Draft**: Fastest layout for previews and interactive editing. No iterative
  optimization. Suitable for graphs under ~20 nodes.
- **Balanced**: Good quality for medium graphs. Light repair fixes the worst
  violations without full A* search.
- **Best**: Production quality for rendered artifacts (SVG, PNG). Full hybrid
  deterministic optimization with parallel candidate construction. Typical
  runtime: 12-15 seconds for complex workflow graphs.

---

## Layout Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `Direction` | Enum | `LeftToRight` | Layout direction. `TopToBottom` uses the legacy iterative path. |
| `NodeSpacing` | int | 40 | Vertical gap between nodes (px). Scaled by edge density up to 1.8x. |
| `LayerSpacing` | int | 60 | Horizontal gap between layers (px). |
| `Effort` | Enum | `Best` | Layout quality vs. speed tradeoff. |

### Edge Density Scaling

When the number of edges between two adjacent layers exceeds a threshold, both
`NodeSpacing` and `LayerSpacing` are scaled up to accommodate additional routing
channels. The maximum scale factor is 1.8x, applied per-layer-pair.

---

## Diagnostics

The layout engine emits detailed diagnostics when running in `Best` effort mode:

- **Live progress log**: Baseline state, strategy starts, per-attempt scores,
  and adaptation decisions logged during execution.
- **Per-attempt phase timings**: Routing time, post-processing time, and
  route-pass counts for each optimization attempt.
- **SVG/PNG/JSON artifacts**: The document-processing artifact test produces
  rendered output alongside diagnostic data.
- **Violation reports**: Per-edge violation lists with category, severity,
  geometry details, and FinalScore adjustments.

Diagnostics are detailed enough to prove routing progress and to profile
optimization performance for regression detection.