Repository navigation
Description
Classification
- Type: bounded research / product-discovery + feasibility spike
- Priority: P3 / research backlog, not a production feature request
- Working name: Requirements-to-Code Conformance Checker ("BDD plus proof-oriented checks")
- Area: natural-language requirements → reviewed machine-checkable contracts → C# code/event mapping → static/runtime verification → source-level counterexamples → PR/CI traceability
- Related Own.NET: Research: FeaVer/Ultimate-style property-guided model extraction and source-level witnesses (bounded spike) #447 (FeaVer/Ultimate model extraction and witnesses; underlying technical spike), Research: evaluate BAIT Büchi inclusion and antichain fixpoints for temporal ownership protocols (bounded spike) #446 (BAIT/Büchi inclusion; separate temporal research), P1: Add Obligation Protocols for project-specific barrier-sensitive rules #272 (obligations), P2: Extend MOS/effect summaries to protocol and barrier effects #274 (protocol/effect summaries), feat(progress): Own.Progress — detect consume-or-exit loop stalls (PRG001) #275 (loop progress), Canonical first-party call facts: preserve interprocedural calls instead of collapsing them into frontend release/use/escape guesses #382 (canonical call facts), Research: IKOS-inspired abstract-state soundness oracle and operator audit (bounded spike) #433 (soundness oracle), Research: Tai-e-inspired semantic fact assertions and selective precision (bounded spike) #443 (semantic-fact assertions)
- Relevant prior art: NASA FRET (constrained language → formal properties), FeaVer (code-to-model/property verification), Microsoft Coyote (systematic C# concurrency testing), Reqnroll (BDD examples/step definitions). These tools solve parts of the pipeline; none alone proves arbitrary natural-language text matches arbitrary C# code.
Hypothesis
Teams already document desired behavior in issue descriptions, acceptance criteria, business rules, ADRs, and sometimes regulatory documents, but a passing BDD scenario or unit test covers only the selected examples.
Could Own.NET help check generalizable, explicitly approved properties against an implementation, and flag previously unanticipated executable paths or version/state combinations?
Desired product-level outcome:
Given requirement R (with source/version), an approved formal interpretation P, and code revision C, report VIOLATED with a reproducible source-linked witness, PROVEN only within an explicitly declared complete model, or UNKNOWN / NOT COVERED when the system cannot justify either.
The natural-language → property step is not trusted. LLM output is a candidate for human review, never an independent proof that the requirement was interpreted correctly.
Motivating example (illustrative; not a request to embed customs semantics in Own.NET)
Human description: "A declaration may be submitted only after validation succeeds."
Ambiguities to surface before analysis:
- Must validation apply to the same version of the declaration that is submitted?
- Does editing fields invalidate prior validation?
- What counts as a blocking error vs warning?
- Are there explicit, source-backed exceptions?
Candidate reviewed property:
Submit(d, version=v) ⇒ a successful Validate(d, version=v) happened since the last relevant mutation and no blocking errors remain
A branch that bypasses validation, or a mutation between validation and submit, may violate this property. The verifier must map events to real C# calls/effects and object identity/version; simply checking that Validate() is somewhere earlier in the method is inadequate.
What makes this different from BDD
- BDD/Reqnroll: curated executable examples through step definitions. Very useful, but does not exhaust unenumerated paths.
- Property-based testing: many generated inputs under a test oracle; strong testing, not a proof of all behaviors.
- Own.NET-backed static verification: explores represented branches/states/transfers within the declared model, returning a valid source witness or stating limits.
- Runtime/systematic testing (optional): can validate a candidate counterexample or explore scheduling-dependent paths; explicitly label tested vs proved.
Do not market one green BDD suite, a bounded search, or an incomplete call graph as a universal conformance proof.
Gate 0: inventory and product constraints
- Review Research: FeaVer/Ultimate-style property-guided model extraction and source-level witnesses (bounded spike) #447 and the existing Roslyn→OwnIR→CFG/MOS/obligations/progress pipeline. Identify reusable event facts, call resolution, source locations, analysis completion status, and diagnostic
codeFlows. - Check existing required/available BDD and property-based testing paths in the repo. Do not introduce Reqnroll, FRET, LTL, Promela, SMT, or a new DSL simply because prior-art tools exist.
- Define a traceability record: requirement ID + original text/span/URL + source document version/effective date + approved interpretation/reviewer + property schema version + mapped code symbols/commit + verifier result/coverage limits. Do not assert current legal correctness from an outdated or unverified regulation.
- Determine whether this is better developed as an external opt-in layer/product consuming Own.NET evidence, rather than widening ownership diagnostics by default.
Gate 1: choose a narrow property language and baseline
Start with a small, auditable set of business/protocol patterns:
- Never / forbidden-before: operation B forbidden while an obligation is open or before event A.
- Only-after / version-matched: B requires a successful A for the same identified entity/version with no intervening invalidation.
- Must-close-before-exit: obligation opened must be discharged before return/throw (when modeled).
- Conditional permission: operation permitted only if a specified predicate has been established; unknown predicate ≠ false/true.
Avoid arbitrary prose, unrestricted first-order logic, temporal liveness, distributed-system correctness and whole-program equivalence in the first vertical.
For each candidate:
- Pin the original requirement, reviewer-approved expected semantics, and one or more relevant code entry points.
- Map business verbs to actual operations/effects with manual correction and source citations.
- Specify unknown policy for opaque calls, reflection, dynamic code, external services, state persistence, async/concurrency, missed mutation/alias edges, and missing data.
- Compare a cheap baseline first: handwritten assertions/BDD and existing Own.NET rules; ensure the idea offers distinct marginal value beyond P1: Add Obligation Protocols for project-specific barrier-sensitive rules #272/P2: Extend MOS/effect summaries to protocol and barrier effects #274/Research: FeaVer/Ultimate-style property-guided model extraction and source-level witnesses (bounded spike) #447.
Gate 2: bounded pilot, not product rollout
Use 10 real, permission-appropriate requirements from a single C# domain with 2–3 implementable property shapes. Freeze the acceptance oracle before looking at tool output. Prepare independently reviewed fixtures per requirement:
- correct implementation,
- seeded/real violating implementation (including a branch bypass),
- near-neighbor negative to test false positives,
- one
UNKNOWNcase with deliberately unresolved call/effect or incomplete model.
At least one case should involve validation → mutation/version change → use, to defeat naive "check A was called sometime" heuristics, provided the required identity/version facts exist; otherwise mark it as a verified representational blocker.
Optional system shape:
human requirement + provenance
↓
LLM candidate property + ambiguities
↓
human-reviewed property / scope / assumptions
↓
mapped source operations + OwnIR facts
↓
existing checker / small test-only monitor
↓
VIOLATED + path | PROVEN (scoped) | UNKNOWN + reason
↓
requirement↔code↔commit evidence for PR/CI
Evaluate two distinct questions:
- Specification fidelity: how often did LLM propose the correct meaning, miss a condition, invent a constraint, or require clarification? Do not silently repair meaning after seeing model output.
- Implementation analysis: with an approved fixed property, how often were real violations found, false alarms raised, and unknowns honestly reported? Keep errors in spec translation separate from errors in code analysis.
Baselines: existing BDD/unit tests and current Own.NET rules, using the same reviewed corpus. Report number of new confirmed violations not already caught by those baselines, not just raw findings.
Success / STOP criteria
- Short state-of-the-art / donor map (FRET vs FeaVer vs Reqnroll/Coyote vs existing Own.NET).
- Concrete traceability record and a human-approval step for every property.
- Frozen corpus: 10 reviewed requirements, corresponding implementation/control fixtures, and explicit assumptions.
- Distinct spec-translation outcomes and code-verification outcomes (TP/FP/FN/UNKNOWN, by case and reason); no invented accuracy percentage.
- At least one adjudicated new violation missed by the existing baseline, or compelling quantified reduction in review/diagnostic effort; otherwise
STOP-NO-INCREMENTAL-VALUE. - A sample report linking requirement text/version → approved property → relevant C# symbols → source-level witness → code commit, with clear coverage/completeness limits.
- Decide
GO-EXTERNAL-PILOT,DEFER-MISSING-FACTS, orSTOP-NO-UTILITY; any production changes become separately scoped issues, with owner approval and Python/Rust parity requirements where applicable.
Hard guardrails
- No autonomous legal interpretation: regulation references are inputs with revision, effective date, jurisdiction and human approval, not proof of legal compliance.
- No universal "code implements issue" claim from an LLM, incomplete CFG, missing calls, bounded execution or unsound abstraction.
- Never convert
UNKNOWN / INCOMPLETEtoPASSor use an unverified candidate witness as an executable bug claim. - No production code/OwnIR schema or language overhaul, no default CI blocking, and no user-authored Gherkin/Promela requirement.
- Do not duplicate Research: FeaVer/Ultimate-style property-guided model extraction and source-level witnesses (bounded spike) #447 (mechanics of model extraction), Research: evaluate BAIT Büchi inclusion and antichain fixpoints for temporal ownership protocols (bounded spike) #446 (Büchi inclusion), or P1: Add Obligation Protocols for project-specific barrier-sensitive rules #272/P2: Extend MOS/effect summaries to protocol and barrier effects #274/feat(progress): Own.Progress — detect consume-or-exit loop stalls (PRG001) #275 (specific existing analyzers).
Lexpedite / Blawx Rules as Code donor (2026-11-14)
Lexpedite / Blawx Rules as Code donor: review Blawx-powered agentic coding and bmcpdemo for human-approved rule model → inverse/forward scenarios → Hypothesis properties → implementation tests, and validation roadmap for approval/revision/gated revalidation. Full notes. Fit only as an optional external contract oracle/test-generation consumer of current OwnIR facts and source-linked witnesses; do not expand default ownership engine. A passing finite PBT suite is TESTED, never PROVEN; semantic legal/requirements interpretation needs human review. Test positive/negative/missing/exception/date boundaries, deliberately corrupted implementations, evidence gaps and explicit UNKNOWN/NOT_COVERED. Compare against plain contracts + existing testing; STOP if no measurable coverage or witness gain.
Logical English 2 / InterProlog donor audit (2026-11-14). Research page · LE2 source · docs · InterProlog. LE2 offers controlled-language typed rule templates, scenario/expected/unknown tests, explanations, Prolog and s(CASP) execution, inverse queries, local SWI-Prolog + REST/MCP. Core is Apache-2.0; translators/InsurLE extensions licensed separately; no native Russian support stated. InterProlog is JVM↔Prolog infrastructure, not C# bridge. Bounded research only; compare current pure rules/decision-table baseline before any runtime/DSL dependency. Distinguish evidence/test coverage from semantic/legal correctness.
Use LE2 only as optional approved rule → executable scenarios → implementation witness oracle. Compare default Prolog and s(CASP), negative mutations, UNKNOWN/NOT_COVERED; do not infer whole-code proof or introduce Prolog into ownership analyzer. InterProlog's Java bridge has no direct advantage for C#/Go.
Potassco ASP: clasp / clingo / Anthem / asplain — bounded donor (2026-11-14)
Полная страница исследования · clingo · clasp · Anthem · asplain. clasp — CDNL solver для grounded ASP; clingo совмещает gringo+clasp и должен быть первым кандидатом на эксперименты. Anthem через Vampire исследует ограниченные equivalence/specification свойства формальных ASP-программ; asplain — контрастивные объяснения при изменении фактов/правил, не доказательство юридической правильности. Brave = хотя бы одна допустимая модель, cautious = все допустимые модели при существовании моделей. UNSAT подтверждает невыполнимость формальной постановки, а не правовую невозможность. ASP not — default negation; UNKNOWN из отсутствующего факта не превращать в FALSE. Исследование / KILL-FIRST, не разрешение на интеграцию или production изменения.
Requirements-to-code: на human-reviewed formalized contracts проверить гипотезу поиска несовместимых свойств (policy constraints/required vs forbidden states) средствами clingo по маленькому явному abstraction model. Это contract-level model SAT, не анализ произвольного C#/OwnIR и не доказательство реальной реализации. Если в модели несколько допустимых путей, brave/cautious различают «существует» и «во всех», но не исправляют incomplete interprocedural facts. Anthem + Vampire могут проверить ограниченную equivalence формальных ASP правил; сравнить с обычным typed assertion/SMT baseline. Negative control: исключение пропущено, UNKNOWN переосмыслен в false, недоказанная ветвь выдана за доказанную. Без интеграции в ownership engine.
2026-11-15 — Future Conditions / verification evidence ladder
New prior art: Yahui Song's future-conditions study, AutoRocq, CTL-guided repair, incorrectness of OO code; full analysis.
Extend approved requirement taxonomy to preconditions, postconditions, future safety and conditional liveness with explicit fairness. Verification/evidence levels: TRACE-VIOLATION, TRACE-CONFORMS-BOUNDED, MODEL-CHECKED-SCOPED, CODE-PROVED-SCOPED, UNKNOWN. A passing finite trace does not establish liveness or full program correctness; a model proof does not prove implementation-model conformance. Proposed kill-first evaluation: 1 small authored model (3–6 states), 2 safety + 1 liveness properties, 2 happy / 3 mutated / 2 incomplete traces, independent oracle and comparison against current existing assertions/PBT. If no incremental bug/evidence value, KILL. No production DSL, new solver, CI gate, or LLM-authored property approved.
2026-11-15 — NTP4VC donor: VC generation and proof checking are distinct layers
Source: NTP4VC (ICLR 2026), analysis. Why3/Frama-C generate source-derived verification conditions; separate rule-based translation maps them to Lean/Rocq/Isabelle. The contribution is a benchmark and extraction/translation pipeline, not a C# or OwnIR verification backend. Do not claim OwnIR currently emits sound VCs or that Frama-C accepts arbitrary C#.
Potential requirement-to-proof evidence structure: reviewed R and version + source hashes → explicit code/CFG model scope → trusted VC generator and mapped source locations → proof language/axioms/libraries/translation assumptions → deterministic ATP baseline → optional LLM-generated proof candidate → trusted checker verdict + replay. Any unmodeled alias/dynamic dispatch/reflection/missing identity stays UNKNOWN; proof acceptance establishes only the formalized obligation under assumptions.
A specific future contract such as after refresh-token rotation, old token cannot be accepted first needs a precise transition abstraction and source mapping. A proof of one VC does not establish liveness, distributed consistency, or full program correctness. Start with #272 finite obligations and #453 auth conformance baseline; only research new VC machinery if baseline leaves a verified, consequential gap.
Kill-first: compare existing OwnIR diagnostics and a tiny state model against 1 meaningful example before attempting Why3 extraction from C#; no expensive 2400-rule translation port or custom proof generator by default. NTP4VC has a deliberately difficulty-selected corpus, mixed licensing, Frama-C 18 and Rocq 8.20.1 reproduction constraints; paper pass rates are not a predictor of own analyzer quality.
2026-11-15 — Structured proof trees and countermodel evidence
Two source-backed additions: Qiyuan Xu AoA/Minilang as a proof-workflow reference (AST JSON edits, proof-state tree, proof-store replay, deterministic Sledgehammer-first); Simon Cruanes Zipperposition/SMBC as distinct automated theorem prover vs bounded model finder. The targeted requirement → approved property → actual source events/identities → test/checker/oracle evidence chain remains unchanged. CHECKER-ACCEPTED, VALIDATED-COUNTERMODEL, TEST-FAIL, UNKNOWN/INCONCLUSIVE must never collapse into generic PROVED; source-code semantics, unmodeled branches, and legal/business specification fidelity still require independent justification. No new proof language, CAS engine, or solver dependency before a relevant concrete property.
Activity
PhysShell commentedon Nov 14, 2026
Addendum (2026-11-14): Act + Gambit + Echidna as method donors for the existing #448 pilot
Relevant reviewed upstreams:
Concrete refinement to Gate 2, not an additional framework: for 2–3 already-reviewed requirements, freeze the rule and independent acceptance oracle first; seed narrowly chosen implementation mutants:
Ask: which mutants are killed by existing tests/BDD, which only by an additional executable property, which by supported OwnIR analysis, and which remain
UNKNOWNdue to missing call/effect/version facts? Include correct near-neighbor cases to avoid vacuous false positives.A real extra signal is one reproducible specification-blind-spot / defect caught beyond the baseline, not a high mutation score from lots of irrelevant mutants. Report mutant type, source witness, model coverage and human adjudication. Keep the #448 GO/STOP budget unchanged; don't auto-upgrade
TESTEDtoPROVEN.Separate application-level vertical: AI-Assistent-skeleton #22 examines business-rule applicability in generated answers (dates, exceptions, conflicting revisions, claim-level uncertainty). It should reuse a narrow test-oracle/mutation technique without turning Own.NET's checker into a RAG system. That issue is not a request to analyze internal company code or documents here.
PhysShell commentedon Nov 14, 2026
Research addendum (2026-11-14): ACL2 / Kestrel APT / Syntheto / Axe as external verification donors
Scope: supplemental methods for #448 requirements→code conformance; not an OwnIR rewrite, ACL2 dependency, or green-light for universal source equivalence. Relevant researcher: Alessandro Coglio.
Upstream, checked in code
simplifydocs show the generatedf1-is-f2theorem.isodatadocs describe proving behavior preservation across isomorphic data representations, subject to applicability/guard restrictions.finite-differenceimplements a proposed incrementalization path but explicitly saysIN-PROGRESS.isomorphism.synthshowssubtype,ensures,transform. Its example status README explicitly identifies a non-working frontend isomorphism example; frontend repo has no published commits after Dec 2021. Not an implementation-ready UI.Recommended #448 use: a deliberately tiny equivalence/refinement case
PROVEDis limited to the mathematical model.UNKNOWN, notPROVEN.This is a narrow research lane for provable pure-function transformations, not a promise of general old-vs-new implementation equivalence. For customs payment rules, prove equivalence only in a specifically delimited unchanged-law subdomain; intentional changes in legislation must be tested against new approved normative contracts, not called regressions.
Connected notes: Assurance-as-Code · Semantic Requirements Workbench · Demand Radar #4.