Skip to content

DINOForge — QA Matrix

Last updated: 2026-03-28 Status legend: ✅ PASS · ❌ FAIL · ⚠️ PARTIAL · 🔄 IN_PROGRESS · 🆕 new v0.11.0: .NET 11 migration, Desktop Companion, Asset Swap timing fixes, CLI JSON output


Tier Pyramid

           ┌──────────────────────────────┐
           │   P2 · Game Launch / E2E     │  live game process, bridge
           │   (game-launch.yml — manual) │
           ├──────────────────────────────┤
           │   P1 · UI Automation         │  FlaUI (Companion) · UiSelectorEngine (overlay)
           │   (ui-automation.yml)        │
           ├──────────────────────────────┤
           │   P1 · Integration           │  FakeBridge, mock ECS, real FS
           │   (ci.yml — every PR)        │
           ├──────────────────────────────┤
           │   P0 · Unit / Arch / Schema  │  pure .NET, no game dep
           │   (ci.yml — every PR)        │
           └──────────────────────────────┘

CI Workflows

WorkflowTriggerTier(s) Covered
ci.ymlpush/PR → mainP0 unit, P0 arch, P1 integration
validate-packs.ymlpush packs/ or schemas/Pack schema + CLI validation
fuzz.ymlnightly 02:00 UTCP2 property / fuzz
mutation-test.ymlweekly Mon 06:00 UTCSDK mutation (Stryker.NET)
benchmarks.ymlon demandP1 performance
ui-automation.ymlmanual + weeklyP1 companion UI automation (FlaUI)
game-launch.ymlmanual + weekly (self-hosted)P2 game launch + overlay automation

Local QA Infrastructure

IDDescriptionStatusNotes
QA-001pre-commit hooks (Lefthook)✅ PASSformat-check, check-yaml, check-json, check-merge-conflicts
QA-002pre-push gate (Lefthook)✅ PASSbuild + unit + integration
QA-003CI integration tests✅ PASSci.yml runs DINOForge.Tests.Integration on every PR
QA-004Local test runner✅ PASSscripts/test-local.ps1

Master Matrix

P0 — SDK / Pack System

IDScenarioTypeFilePass CriteriaStatus
SDK-001Valid manifest loads all registriesunit/bddBddSpecs.csLoadedPacks.Count == loaded
SDK-002Missing manifest returns error, no throwunitContentLoaderTests.csLoadPack_MissingManifest_Fails
SDK-003Circular dependency detectedunitDependencyResolverTests.csDependencyException with full chain
SDK-004Missing dependency detectedunitDependencyResolverTests.csError names missing pack
SDK-005Load order respects dependency graphunitContentLoaderTests.csWithLoadOrder_LoadsInCorrectOrder
SDK-006Registry: register + retrieve by IDunitRegistryTests.csExact object returned
SDK-007Registry: duplicate raises conflictunitRegistryTests.csConflict event fired
SDK-008All pack YAML validates against schemaschemaSchemaValidationTests.cs + validate-packs.ymlZero violations
SDK-009Conflicting packs both refusedunitCompatibilityCheckerTests.csConflict reason in result
SDK-010Corrupt YAML fails gracefullyunitContentLoaderEdgeCaseTests.csError returned, no unhandled exception

P0 — Architecture Enforcement

IDScenarioTypeFilePass CriteriaStatus
ARCH-001SDK has no dependency on RuntimearchArchitectureTests.csNetArchTest passes
ARCH-002SDK has no dependency on DomainsarchArchitectureTests.csNetArchTest passes
ARCH-003Public interfaces reside in SDK namespacearchArchitectureTests.csNetArchTest passes

P0 — Domain: Warfare

IDScenarioTypeFilePass CriteriaStatus
DOM-001All canonical archetypes resolveunitWarfareTests.cs≥ 20 archetype IDs present
DOM-002Clone Trooper ≠ ARC Trooper archetypeunitWarfareTests.csDifferent BaseStats records
DOM-003IndustrialSwarm has correct modifiersunitWarfareTests.csModifier values match spec
DOM-004Wave definitions reference valid squad IDsunitSkillWaveSquadTests.csAll squad IDs resolvable
DOM-005Doctrine modifiers are non-zerounitWarfareTests.csAt least one modifier ≠ 1.0

P0 — Bridge Protocol (Offline)

IDScenarioTypeFilePass CriteriaStatus
BRG-0016-step offline round-trip (FakeGameBridge)integrationBridgeRoundTripTests.csAll steps pass: load→query→override→read→reload
BRG-002Ping returns healthyintegrationPingTests.csresult.Healthy == true
BRG-003ComponentMap resolves 30+ DINO typesintegrationComponentMapTests.csAll mappings non-null
BRG-004StatModifier applies HP overrideintegrationStatTests.csEntity HP == override value
BRG-005Resource delivery system integrationintegrationResourceTests.csResource count matches delivery

P0 — Asset Pipeline

IDScenarioTypeFilePass CriteriaStatus
AST-001AddressablesCatalog loads valid catalogintegrationCatalogTests.csBundle paths resolve to files
AST-002Bundle path placeholder replaced with StreamingAssetsunitAddressablesCatalogTests.csPath starts with StreamingAssets
AST-003Non-existent catalog path throws FileNotFoundExceptionunitAddressablesCatalogTests.csCorrect exception type
AST-004AssetSwapRegistry de-duplicates same addressunitAssetSwapRegistryTests.csOnly one patch written
AST-005Patched bundle differs from sourceintegrationAssetSwapRegistryTests.csByte-level diff detected
AST-006ReadCatalog() failure in AssetSwapSystem is caughtunitAssetSwapRegistryTests.csPhase 2 still executes

P1 — Integration (CI gate)

IDScenarioTypeFilePass CriteriaStatus
INT-001Full pack loading against mock ECSintegrationPackLoadingTests.csPack types registered in mock world
INT-002JSON-RPC bridge round-trip (in-process)integrationBridgeRoundTripTests.csProtocol framing correct
INT-003Hot reload fires within 5s on file changeintegrationHotReloadTests.csEvent ≤ 5000 ms
INT-004Hot reload on invalid YAML keeps old stateintegrationHotReloadTests.csOld pack still loaded, error logged
INT-005PackCompiler CLI validate exits 0 for valid packscli-integrationvalidate-packs.ymlExit code 0
INT-006PackCompiler CLI validate exits non-zero for invalid packcli-integrationnew PackCompilerCliTests.csExit code ≠ 0, message contains path🆕
INT-007Bridge latency < 50ms (FakeBridge, 1000 req)perfnew BridgeLatencyTests.csP99 < 50 ms🆕

P1 — Performance (benchmarks.yml)

IDScenarioTypeFilePass CriteriaStatus
PERF-001Asset import < 5s/modelperfPerformanceBenchmarkTests.csP99 < 5 000 ms
PERF-002Full 9-model pipeline < 5 minperfPerformanceBenchmarkTests.csTotal < 300 s
PERF-003Bridge round-trip P99 < 50 msperfnew BridgeLatencyTests.csP99 < 50 ms🆕
PERF-004AssetSwapSystem phase 1 completes before frame 5perfnew AssetSwapLatencyTests.csPatch exists by frame 5 mock🆕

P1 — Runtime: BepInEx Plugin & Bridge (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
RT-001BepInEx plugin loads without exceptionunitRuntimeBootstrapTests.csPlugin Awake() completes, no unhandled
RT-002HideAndDontSave root survives DINO two-boot cycleintegrationRuntimePersistenceTests.csRoot GameObject exists after 2 game restarts⚠️ PARTIAL
RT-003F9 keypress toggles debug overlayintegrationKeyInputSystemTests.csWin32 watcher hook fires; message → KeyInputSystem.OnInput → overlay toggle✅ PASS
RT-004F10 keypress toggles mod menu overlayintegrationKeyInputSystemTests.csWin32 watcher hook fires; message → KeyInputSystem.OnInput → menu toggle✅ PASS
RT-005RuntimeDriver.Update survives ≥ 600 framesintegrationRuntimeDriverTests.csOnDestroy not called within 600 frames; root persists❌ FAIL

P1 — Asset Swap System (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
AST-007Phase 1: bundle patched to BepInEx/dinoforge_swap/ before frame 5integrationAssetSwapPhase1Tests.csPatched bundle file exists by frame 5
AST-008Phase 2: RenderMesh swapped on live entity after phase 1integrationAssetSwapPhase2Tests.csQueryUnits()[0].MeshId matches swapped mesh
AST-009Prefab mesh extraction (MeshFilter/SkinnedMeshRenderer fallback)unitMeshExtractionTests.csTrySwapRenderMeshFromBundle handles prefab + mesh fallback
AST-010visual_asset alignment (warfare-starwars bundle names match unit def)integrationVisualAssetAlignmentTests.csAll 28 units resolve bundles successfully

P1 — CLI Tools (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
CLI-001All 9 commands support --format jsoncli-integrationCliFormatTests.csstatus, query, resources, override, dump, reload, screenshot, component-map, verify all emit JSON
CLI-002JSON output parses and has required fieldsunitCommandOutputTests.csCommandOutput.WriteJson() produces valid JSON
CLI-003Errors suppress ANSI markup when --format json activeunitCommandOutputTests.csNo [Red] or [/] in JSON error messages
CLI-004ui tree command returns XAML node treecli-integrationCliUiTreeTests.csHierarchical JSON with node names, IDs, enabled states
CLI-005ui click targets button by AutomationId and fires clickcli-integrationCliUiClickTests.csButton event handler invoked🔄 IN_PROGRESS
CLI-006ui wait polls for text and times out after 30scli-integrationCliUiWaitTests.csText found within timeout OR exit code 124 on timeout🔄 IN_PROGRESS
CLI-007ui expect sets exit code 1 if assertion fails (JSON mode)cli-integrationCliUiExpectTests.cs--format json --expect "text" "not found" → exit 1🔄 IN_PROGRESS

P1 — MCP Server Tools (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
MCP-001game_launch tool starts game exe, awaits bridge pingmcpGameLaunchMcpTests.csGame process alive, bridge responds
MCP-002game_status returns entity count, loaded packsmcpGameStatusMcpTests.csJSON with entityCount, packs[]
MCP-003game_query_entities filters by component typemcpGameQueryMcpTests.csReturns only entities with requested component
MCP-004game_get_stat reads stat on entitymcpGameGetStatMcpTests.csReturns numeric stat value
MCP-005game_apply_override persists after reloadmcpGameApplyOverrideMcpTests.csStat value persists after ReloadPacks()
MCP-006game_reload_packs hot-reloads within 5smcpGameReloadPacksMcpTests.csPack version increments within 5s
MCP-007game_dump_state triggers entity dump exportmcpGameDumpStateMcpTests.csJSON file written to BepInEx logs
MCP-008game_screenshot captures game windowmcpGameScreenshotMcpTests.csPNG file created, dimensions match window
MCP-009game_verify_mod checks DINOForge loadedmcpGameVerifyModMcpTests.csReturns {"verified": true} if bridge responds
MCP-010game_wait_for_world polls until ECS readymcpGameWaitForWorldMcpTests.csReturns when CalculateEntityCount() &gt; 0
MCP-011game_ui_automation drives in-game UImcpGameUiAutomationMcpTests.csClick/toggle/screenshot work in-game🔄 IN_PROGRESS

P1 — Hot Reload (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
HOT-001File watcher detects pack YAML changeintegrationHotReloadWatcherTests.csFileChanged event fires within 1s
HOT-002Bundle change detection (checksum mismatch)integrationHotReloadBundleTests.csChecksum mismatch triggers reload
HOT-003Invalid YAML keeps old pack, logs errorintegrationHotReloadErrorTests.csOld pack still active, error file written
HOT-004Reload fires OnAfterPackReload eventintegrationHotReloadEventTests.csEvent handler invoked with pack ID + version

P1 — Native Menu Injection (v0.11.0)

IDScenarioTypeFilePass CriteriaStatus
NATIVE-001"Mods" button injected between Options and CreditsintegrationNativeMenuInjectionTests.csButton exists in main menu, clickable, AutomationId="ModsButton"✅ PASS
NATIVE-002"Mods" button click opens overlay (F10 equivalent)integrationNativeMenuInjectionTests.csButton click sets ModMenuOverlay.IsVisible=true; equals F10 behavior❌ FAIL
NATIVE-003Menu injection survives scene reload + RuntimeDriver persistenceintegrationNativeMenuInjectionTests.cs"Mods" button still present after 2+ scene changes; RuntimeDriver.OnDestroy not fired❌ FAIL

P1 — UI: Desktop Companion (ViewModel unit)

IDScenarioTypeFilePass CriteriaStatus
COMP-001Pack list loads from data serviceunitCompanionTests/ViewModelTests.csObservable collection has items
COMP-002Toggle updates IsEnabled stateunitCompanionTests/ViewModelTests.csState flips on toggle
COMP-003Error pack shows error badge stateunitCompanionTests/ViewModelTests.csHasErrors=true
COMP-004Status text correct when no errorsunitCompanionTests/ViewModelTests.cs"All N pack(s) loaded OK"
COMP-005Disabled pack service persists across restartunitCompanionTests/DisabledPacksServiceTests.csPack IDs survive round-trip
COMP-006Dashboard shows pack count + status summaryunitCompanionTests/DashboardViewModelTests.csTitle text includes pack count
COMP-007Settings page loads game path from registryunitCompanionTests/SettingsViewModelTests.csInitial path matches saved value
COMP-008Debug panel queries entity count from BridgeunitCompanionTests/DebugViewModelTests.csCalls GameClient.GetStatusAsync() on load

P1 — UI Automation: Desktop Companion (FlaUI — ui-automation.yml, Windows)

Tooling: FlaUI.Core + FlaUI.UIA3 (Windows Automation API) Runner: windows-latest GitHub Actions Category trait: [Trait("Category", "UiAutomation")]Project: src/Tests/UiAutomation/DINOForge.Tests.UiAutomation.csproj

IDScenarioTypeFilePass CriteriaStatus
COMP-UI-001Main window launches and is visibleui-auto-companionnew CompanionLaunchTests.csFlaUI finds window by AutomationId="MainWindow"🆕
COMP-UI-002Pack list ListView shows ≥ 1 itemui-auto-companionnew CompanionPackListTests.csListView.Items.Count ≥ 1🆕
COMP-UI-003Toggle pack changes service stateui-auto-companionnew CompanionPackToggleTests.csFlaUI ToggleButton.Toggle() → service IsEnabled flips🆕
COMP-UI-004Settings page saves game pathui-auto-companionnew CompanionSettingsTests.csFlaUI edit TextBox(AutoId="GamePathInput") → settings file updated🆕
COMP-UI-005Status bar shows "Not connected" when bridge absentui-auto-companionnew CompanionStatusBarTests.csStatusBar.Text matches pattern🆕
COMP-UI-006Keyboard shortcut Ctrl+R triggers reloadui-auto-companionnew CompanionShortcutTests.csReload event fired after key send🆕

P1 — UI Automation: In-Game Overlay (UiSelectorEngine — game-launch.yml)

Tooling: BepInEx in-game test plugin + UiSelectorEngine + UiActionTraceExecution: plugin runs inside game process, results exported via UiActionTrace.SaveToFile()Category trait: [Trait("Category", "GameLaunch")] on the bridge-side assertions Project: src/Tests/GameLaunch/ (bridge-side) + src/Runtime/Tests/ (in-game plugin side)

IDScenarioTypeFilePass CriteriaStatus
OVL-001F10 keypress opens overlayui-auto-overlaynew OverlayToggleTests.csModMenuOverlay.IsVisible == true🆕
OVL-002Second F10 closes overlayui-auto-overlaynew OverlayToggleTests.csModMenuOverlay.IsVisible == false🆕
OVL-003Pack list shows all loaded packsui-auto-overlaynew OverlayPackListTests.csUiSelectorEngine.Query("pack-list").Nodes.Count == loadedPacks.Count🆕
OVL-004Clicking pack toggle fires OnPackToggledui-auto-overlaynew OverlayPackToggleTests.csCallback invoked with correct pack ID + false🆕
OVL-005Debug panel shows numeric entity countui-auto-overlaynew DebugPanelTests.csUiSelectorEngine.Query("debug-entity-count").Text matches \d+🆕
OVL-006HUD indicator shows mod-active badgeui-auto-overlaynew HudIndicatorTests.csUiSelectorEngine.Query("hud-mod-active").IsVisible == true🆕
OVL-007UiActionTrace exports valid JSONui-auto-overlaynew UiActionTraceTests.cs (unit, no Unity)GetHistory().Count ≥ 1, exported JSON parses🆕
OVL-008Settings panel saves preference and survives reloadui-auto-overlaynew OverlaySettingsTests.csPreference file updated, value persists after ReloadPacks()🆕

P2 — Game Launch / End-to-End (game-launch.yml — self-hosted, game installed)

Tooling: GameClient (JSON-RPC bridge) launched from GameLaunchFixtureRunner: self-hosted Windows runner with DINO installed at DINO_GAME_PATHCategory trait: [Trait("Category", "GameLaunch")]Project: src/Tests/GameLaunch/DINOForge.Tests.GameLaunch.csprojTimeout: per-test 120s, collection fixture timeout 300s

IDScenarioTypeFilePass CriteriaStatus
GL-001BepInEx bootstraps DINOForge plugingame-launchnew GameLaunchSmokeTests.csBridge PingAsync() responds Healthy=true within 30s🆕
GL-002warfare-starwars loads 28 units in live cataloggame-launchnew GameLaunchPackTests.csGetCatalog() totalUnits == 28🆕
GL-003Phase 1: bundle patched to disk before entity loadgame-launchnew GameLaunchAssetSwapTests.csPatched bundle file exists by frame 5 (queried via bridge)🆕
GL-004Phase 2: RenderMesh.mesh swapped on clone troopergame-launchnew GameLaunchAssetSwapTests.csQueryUnits("rep_clone_trooper")[0].MeshId == swappedMeshId🆕
GL-005HP override persists after ReloadPacks()game-launchnew GameLaunchStatTests.csReadStat(entityId, "hp") == 999 post-reload🆕
GL-006F10 overlay opens in live game (MCP overlay_status)game-launchnew GameLaunchUiTests.csBridge tool overlay_status returns {"visible":true}🆕
GL-007Hot reload: pack YAML change reloads within 5sgame-launchnew GameLaunchHotReloadTests.csBridge GetStatus() reports new pack version within 5s🆕
GL-008Economy pack changes resource rate in live gamegame-launchnew GameLaunchEconomyTests.csReadStat(resourceEntity,"rate") matches pack definition🆕

P2 — Property / Fuzz (fuzz.yml — nightly)

IDScenarioTypeFilePass CriteriaStatus
FUZZ-001Arbitrary YAML does not crash loaderpropertyFuzzTargets/PackManifest.csNo unhandled exception
FUZZ-002Malformed JSON-RPC does not crash bridgepropertyFuzzTargets/Json.csNo unhandled exception
FUZZ-003Version strings parse without throwspropertyFuzzTargets/Semver.csNo unhandled exception
FUZZ-004Corrupt asset catalog fails gracefullypropertynew FuzzTargets/AssetCatalog.csFileParseException, not unhandled🆕
FUZZ-005Schema validator rejects all invalid docspropertynew FuzzTargets/SchemaValidation.csNo valid doc incorrectly rejected🆕

New Tests: Implementation Summary

src/Tests/GameLaunch/ — Game Launch Project

DINOForge.Tests.GameLaunch.csproj    ← net11.0, xunit, FluentAssertions, GameClient ref
GameLaunchFixture.cs                  ← IAsyncLifetime: launch process, await bridge ping
GameLaunchSmokeTests.cs               ← GL-001
GameLaunchPackTests.cs                ← GL-002
GameLaunchAssetSwapTests.cs           ← GL-003, GL-004
GameLaunchStatTests.cs                ← GL-005
GameLaunchUiTests.cs                  ← GL-006
GameLaunchHotReloadTests.cs           ← GL-007
GameLaunchEconomyTests.cs             ← GL-008

src/Tests/UiAutomation/ — Desktop Companion FlaUI Project

DINOForge.Tests.UiAutomation.csproj  ← net11.0-windows10.0.26100.0, FlaUI.Core, FlaUI.UIA3
CompanionFixture.cs                   ← IAsyncLifetime: launch companion, find MainWindow
CompanionLaunchTests.cs               ← COMP-UI-001
CompanionPackListTests.cs             ← COMP-UI-002
CompanionPackToggleTests.cs           ← COMP-UI-003
CompanionSettingsTests.cs             ← COMP-UI-004
CompanionStatusBarTests.cs            ← COMP-UI-005
CompanionShortcutTests.cs             ← COMP-UI-006

In-Game Overlay (runs inside game via BepInEx test plugin)

Overlay tests (OVL-001…008) execute inside the game process. The BepInEx plugin (DINOForge.Tests.InGame) uses UiSelectorEngine + UiActionTrace to drive and record overlay interactions, then writes results to BepInEx/logs/dinoforge_ui_test_results.json. The game-launch.yml workflow reads that file and asserts pass/fail.

New CI Workflows

ui-automation.yml — Windows runner, FlaUI companion tests:

yaml
on: [workflow_dispatch, schedule: "0 4 * * 1"]  # weekly Monday
runs-on: windows-latest
filter: dotnet test --filter "Category=UiAutomation"

game-launch.yml — self-hosted runner, live game:

yaml
on: [workflow_dispatch, schedule: "0 5 * * 1"]  # weekly Monday
runs-on: [self-hosted, windows, dino-installed]
env: DINO_GAME_PATH, DINO_BEPINEX_PATH
filter: dotnet test --filter "Category=GameLaunch"

Coverage Targets (v0.11.0)

LayerCurrentTargetStatusGap
SDK (unit)~65%80%INT-006, fuzz targets
Bridge Protocol~70%85%BRG latency tests
Asset Swap~75%85%AST-007/008 added
Runtime / BepInEx0%60%RT-001…005 (F9/F10 blocking)
CLI Tools~50%80%🔄CLI-005/006/007 in progress
MCP Server~60%85%🔄MCP-011 in progress
Hot Reload~70%85%HOT-001…004 added
Native Menu0%100%NATIVE-001…003 (RuntimeDriver blocker)
Overlay UI logic~30%70%⚠️OVL-001…008, depends on RT-003/004
Desktop Companion UI~60%80%COMP-001…008 all added
E2E / Game Launch~40%80%🔄GL-001…008 in progress

v0.11.0 Known Issues (Blocking Tests)

IssueImpactTest(s)Fix Status
F9/F10 key input brokenWin32 watcher → KeyInputSystem path fails; keys not reaching SystemBaseRT-003, RT-004, OVL-001/002, NATIVE-002🔄 IN_PROGRESS (RuntimeDriver.OnDestroy fires at frame ~6s)
RuntimeDriver.OnDestroy earlyRoot GameObject destroyed unexpectedly at frame ~6s, HideAndDontSave not persistingRT-005, NATIVE-001, NATIVE-003🔄 IN_PROGRESS (under investigation)
UGUI overlay alpha visibleHudStrip AlphaBase not suppressing overlay until user F10 pressOVL-006⚠️ PARTIAL (fix merged, needs verification)

Released under the MIT License.