feat(aimds): aidefence-core Rust crate over the shared AIDefence pattern packs by ruvnet · Pull Request #102 · ruvnet/midstream · GitHub
Skip to content

feat(aimds): aidefence-core Rust crate over the shared AIDefence pattern packs - #102

Merged
ruvnet merged 1 commit into
mainfrom
feat/aidefence-core-rust
Sep 5, 2026
Merged

feat(aimds): aidefence-core Rust crate over the shared AIDefence pattern packs#102
ruvnet merged 1 commit into
mainfrom
feat/aidefence-core-rust

Conversation

@ruvnet

@ruvnet ruvnet commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Summary

Adds AIMDS/crates/aidefence-core, a Rust detection core in the AIMDS Cargo workspace, so Rust services (the cognitum-one Slack bot, cognitum-cogs) run the same AIDefence detection as the npm package.

Stacked on feat/aidefence-injection-packs (the pattern-pack PR #105, #105, head 2bb5f27): the base of this PR is that branch, so the diff here is the crate only. Merge that PR first, then retarget this PR to main (gh pr edit 102 --base main) before the pack branch is deleted.

  • One pattern source, nothing hand-copied. The crate embeds AIMDS/patterns/*.json at build time (include_str!) and can load the same directory at runtime (Registry::from_dir). The core pack is the 25-pattern set of @claude-flow/aidefence 3.0.2 (ruflo v3/@claude-flow/aidefence/src/domain/services/threat-detection-service.ts, the code behind aidefence_scan); the other five packs cover the misses measured 2026-09-04. Note: midstream AIMDS/src and the published aidefence@2.3.0 carry no detection regexes of their own.
  • Engine parity. Mirrors AIMDS/src/detection/engine.ts: normalisation (NFKC, invisibles, the same confusables table, newlines kept), separator / leet / compact variants with \s+-free compact regexes, bounded base64 / hex / url / rot13 / reverse decoding with one rescan, per-id dedupe, variant confidence penalties.
  • Explicit divergence ledger (Registry::divergences, asserted in tests): CORE-005's trailing (?!going|about|ready) becomes a not_followed_by post-check (one documented gap: JS would still match you are now going with two spaces). Counted repeats with a bound of 1024+ would be relaxed to unbounded and listed too, but the pack PR already replaced its {n,2048} bounds with +/*?, so today: 1 rewritten, 0 skipped, 0 errors across 53 patterns. Mid-pattern lookaround or backreferences are load errors; "engine": "js" patterns that fail become Skipped, never silent drops.
  • Parity test on the shared corpus. tests/parity.rs runs AIMDS/tests/fixtures/injection-corpus.json (85 cases) and requires, per case, the same pattern ids, matching variant, decoding and confidence as the TS engine (tests/fixtures/ts-expected.json, generated with tsx from the TS engine at 2bb5f27; regeneration one-liner in the README). The TS unit assertions are mirrored one for one (known FPs F13/F27 = exactly CORE-018, leet case, decoded-from, no recursion, candidate cap, core-only 3.0.2 verdicts).
  • API: detect, is_safe, sanitize (idempotent, PII-masking, never truncates), normalize, decode_and_rescan, Detector + Config (pack overrides honouring enabledByDefault, variants/decode toggles, 100 KB truncation, unsafe threshold). PII (6 patterns) lives in the crate's patterns/pii.json because the shared packs carry none; the TS email class [A-Z|a-z] typo is fixed there.
  • wasm feature (wasm-bindgen exports) is cargo checked on wasm32-unknown-unknown; no wasm-pack build or JS smoke test.
  • AIMDS/Cargo.toml: crate added to members; regex crates set to opt-level = 3 in the dev profile (the packs' heavy regexes made the 100 KB latency gate ~290 ms unoptimised).

Not ported (stated in README): ThreatLearningService (ReasoningBank-style learning, HNSW similarity, mitigation tracking), behavioural analysis, policy verification, inputHash, quickScan/getStats, and 3.0.2's confidence arithmetic (both engines use the pack model instead).

Recommendation. AIMDS/crates/aimds-detection still carries its own older 10-literal + 5-regex matcher and PII sanitizer; it is untouched here and should be unified onto these packs in a follow-up so the workspace has one detection engine.

Test plan

cargo test -p aidefence-core            # 45 tests: 20 unit, 6 examples, 7 parity, 3 perf, 6 proptest, 2 loader, 1 doctest
cargo clippy -p aidefence-core --all-targets -- -D warnings
cargo fmt -p aidefence-core -- --check
cargo check -p aidefence-core --target wasm32-unknown-unknown --features wasm
cargo check -p aimds-detection          # workspace neighbour still builds
  • Perf, 100 KB adversarial input (4 variants, 3 decoded candidates), 200 ms bound: dev profile 25 ms detect / 32 ms sanitize; release 10 ms / 3 ms.
  • Not covered by CI: rust-ci.yml runs the root workspace only and AIMDS is a separate workspace with a gitignored Cargo.lock, so these gates ran locally (rustc 1.97; rust-version = "1.81" declared, not verified). The crate embeds files outside its package root, so it is a git dependency, not publishable to crates.io as-is.

🤖 Generated with RuFlo

https://claude.ai/code/session_013PKv3picsfzoQJDKLLW7Rt

@ruvnet ruvnet changed the title feat(aimds): aidefence-core Rust crate sharing AIDefence pattern packs feat(aimds): aidefence-core Rust crate over the shared AIDefence pattern packs Sep 4, 2026
@ruvnet
ruvnet force-pushed the feat/aidefence-core-rust branch from d69beb3 to eb1160c Compare September 4, 2026 21:14
@ruvnet
ruvnet changed the base branch from main to feat/aidefence-injection-packs September 4, 2026 21:14
@ruvnet
ruvnet force-pushed the feat/aidefence-core-rust branch 2 times, most recently from 546468d to 98877c9 Compare September 4, 2026 21:18
@ruvnet
ruvnet changed the base branch from feat/aidefence-injection-packs to main September 5, 2026 13:32
…pattern packs

Adds AIMDS/crates/aidefence-core to the AIMDS workspace: a deterministic
detection core so Rust services (cognitum-one Slack bot, cognitum-cogs)
run the same detection as the aidefence npm package.

- Loads the shared AIMDS/patterns/*.json packs (core = the 25 patterns of
  @claude-flow/aidefence 3.0.2, plus tool_invocation, exfil_url,
  encoded_instruction, slack_markup_forgery, instruction_override_i18n):
  include_str! at build time and Registry::from_dir at runtime. No regex
  is hand-copied into Rust.
- Mirrors AIMDS/src/detection/engine.ts: normalisation, separator/leet/
  compact variants, bounded base64/hex/url/rot13/reverse decoding with one
  rescan, per-id dedupe, variant confidence penalties.
- JS-to-regex-crate translation (ASCII \b \d \w, inline flags) with an
  explicit divergence ledger: CORE-005's trailing (?!...) becomes a
  not_followed_by post-check; {n,2048} repeats in EX-001/002/004 and
  SL-004 are relaxed to {n,}. Skipped or failing patterns are reported,
  never silently dropped.
- tests/parity.rs: id-for-id, variant, decoding and confidence parity
  with the TS engine on the shared 85-case corpus (snapshot in
  tests/fixtures/ts-expected.json generated from the TS engine).
- API: detect / is_safe / sanitize (idempotent, PII-masking, never
  truncates) / normalize / decode_and_rescan, Detector + Config (pack
  overrides, 100 KB truncation, unsafe threshold), PII via patterns/pii.json.
- 45 tests: unit, examples, parity, proptest, runtime loader, 100 KB
  latency gate (25 ms dev with regex crates at opt-level 3, 10 ms release).
  wasm feature checked on wasm32-unknown-unknown.

Learning/HNSW/behavioural layers are intentionally not ported;
aimds-detection is untouched (unification recommended as follow-up).

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_013PKv3picsfzoQJDKLLW7Rt
@ruvnet
ruvnet force-pushed the feat/aidefence-core-rust branch from 98877c9 to a5e8c6c Compare September 5, 2026 13:34
@ruvnet
ruvnet merged commit abe78d6 into main Sep 5, 2026
25 of 39 checks passed
@ruvnet
ruvnet deleted the feat/aidefence-core-rust branch September 5, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant