What AgentVetter sends to SIE, what it escalates, and why
scan_runs.item_id NOT NULL audited in db/schema.sql), not per-run. Four signals; three pre-computed, one judged by SIE.SIE_ENDPOINT, SIE_API_KEY; chat → ALIBABA_OPENAI_BASE_URL, DASHSCOPE_API_KEY; visuals → DASHSCOPE_HOST, ALIBABA_DASH_SCOPE_API_URL. MODEL_STUDIO_MODEL defaults to qwen3.8-max; SIE_MODEL to gen-4b. Router rows carry scanner_source=tiered_router and are excluded from severity rollups; a prior router row is replaced only after a successful SIE decision (never batch-delete first), so a SIE outage cannot wipe dashboard strips./v1/score schema is documented (reranking upgrade un-deferred) · /v1/generate + chat with JSON-schema output (typed envelope straight from the model) · X-Trace-ID + timing headers to log per decision.SIE_MODEL now defaults to gen-4b (a Cloud alias; see prototypes/sie-studio/models.json), missing keys warn-and-skip rather than fail the scan, and the never-batch-delete rule is in the ADR./v1/score against the reference set · Emit the §5 envelope via structured output instead of post-parsing
The Native Inference page draws a five-stage pipeline "on SIE Cluster, one endpoint". It lines up closely with the router you built — and shows what is still missing.
| # | Superlinked reference stage | Model named | AgentVetter today | Gap / idea (mine unless noted) |
|---|---|---|---|---|
| 1 | Safety and PII gate — blocks unsafe prompts, masks personal data | Qwen3Guard 4B | Third-party scanners (Cisco, Snyk, Tessl, DepShield, Ossprey); no semantic guard or PII step | T1 (Qwen3Guard) + superlinked/pii-redact (v0.10.1) to sanitise findings before logging |
| 2 | Semantic cache — answers a repeated question at once | Qwen3-Embedding-4B | semantic_novelty (cosine < 0.6 to a reference set) with all-MiniLM-L6-v2 | Same embedding call, opposite use: AgentVetter wants novelty, a cache wants similarity. Reusing prior arbitration notes for near-duplicate findings is T2 |
| 3 | Intent and complexity — "picks the skill, and how strong a model it needs" | GLiNER2.5 Decide | SIE chat judgement (low_confidence) via gen-4b | T9 — typed Decide call; 13 ms vs ~1 s; −8 points accuracy vs frontier, so gate rather than replace |
| 4 | LLM — plans and answers, calling SIE tasks as tools | Qwen3.8 27B | Model Studio qwen3.8-max (Arbitration/Triage) on escalation only | The 27B is the open-model option for the second hop if Model Studio must be avoided (T6/C3 theme) |
| 5 | Verify — checks the answer before it goes out | Granite Guardian 2B | Router envelope logged; no post-check | A guard pass on the arbitration note before it reaches the dashboard; or the threat-mapper's "re-check the exact quote" step |
| + | Cost routing (gallery request-routing/cost) | bge-m3 kNN | Four fixed escalation signals | T11 — similarity-weighted success over a graded bank; threshold chosen on the bank; −38.6 % cost at 95.8 % quality in their study |
| + | Data governance (SIE Relay) | — | Not addressed | T10 — remote="forbid", explicit base URL, log served_by |
/security solution page, which today showcases the ATT&CK mapper instead.The hub is Superlinked-first, but the router has two tiers and the merged Alibaba + Superlinked report is the only place the Model Studio side was inventoried. Kept here so this document is the single source of truth.
| Category | What's there |
|---|---|
| Models / serving | Qwen family (qwen-max, qwen-plus …), OpenAI-compatible endpoint (ALIBABA_OPENAI_BASE_URL + DASHSCOPE_API_KEY) |
| RAG & retrieval | Managed Knowledge Base, text-embedding-v4, gte-rerank, RAG optimisation playbook; batch embedding API at 50% discount |
| Agents | Assistant API, Agent apps, multi-agent composition |
| Fine-tuning & eval | Fine-Tuning API with baseline-vs-tuned harness |
| AI Guardrails | Prompt-injection / jailbreak / sensitive-data detection, malicious file & URL risk, code-vuln / credential-leak scan, content moderation (text/image/video), PII detection |
| Visual (hackathon track) | qwen-image-2.0-pro, qwen-image-plus, qwen-image-edit; Wan wan2.7-image-pro, wan2.6-image (DASHSCOPE_HOST, ALIBABA_DASH_SCOPE_API_URL) |
| Auth / regions | API key + AK/SK signature; region-gated — Singapore recommended for International accounts; free quota, savings plans |
| # | Idea | E·C·R |
|---|---|---|
| MS-T1 | Guardrails prompt-injection → new Finding source; register "alibaba-guardrails" in scan_run_scanners | 2·2·2 |
| MS-T2 | File/URL risk detection on MCP-manifest URLs → Finding.evidence | 2·2·2 |
| MS-T3 | Code-vuln / credential-leak scan as a 4th adapter (steps partially recovered) | 2·2·2 |
| MS-R5 | Batch embedding API (50% off) for sweep re-validation | 1·1·1 |
| MS-R8 | PII detection as pre-ingestion sanitisation for real-customer sweep data | 2·1·1 |
| MS-C1–C6 | Self-auditing RAG agent · sweep-engine transfer to scanner selection · fine-tuned Qwen classifier vs Guardrails · multi-agent bridge over both repos' tools · watermarked synthetic corpora · multimodal scanning of MCP outputs (needs Finding.modality) | 2–3 |
| M1 | SIE Guard as default path; Model Studio Guardrails as optional "cloud second opinion" for ambiguous / high-severity findings — the most consequential platform call in the report | — |
| M2 | "Explore cheap, confirm managed": SIE Encode for bulk sweeps, re-validate top-N on text-embedding-v4 batch | — |
| M3 · M4 | SIE Extract pre-filters spans before Guardrails code-scan · two independent classifier baselines (C3 vs SIE Guard) | — |
MS-C7 = shared batch-evaluation infrastructure across both repos; MS-C8 = a "Coding Plan" for contributor tooling (titles only — their card bodies were never recovered). MS-T4–T8 / MS-R1–R3 remain title-level; the 31-row unified scope matrix in the merged report is the fuller source.
env-vars.md now state it precisely — routing is optional for scans; once routing runs, both key sets are required and validated up front; Model Studio is called only on escalation.The SIE examples, models and defaults the router actually leans on
Your Aug session ranked nine docs examples for the tiered-router idea (Stripe Link Fraud 8/10, Regulatory RAG 6/10, Document OCR 5/10 …). The gallery has since gained examples that are closer fits. New ranking against the same question: "what should I read before touching the router gate?"
| # | Example | Primitives | Why it matters to AgentVetter / the router | Fit |
|---|---|---|---|---|
| 1 | request-routing new | encode · chat | Qwen3-Embedding-4B + frozen classifier (threshold 0.29) routes directly; below threshold Qwen3.8-27B answers under a JSON schema. On 3,000 paired cases: 76.8 % of requests avoid generation, 89.6 % accuracy vs 90.6 % (GPT-6 Luna) / 89.0 % (Sonnet 5), modeled cost ≤ $0.101 per 1k vs ≥ $3.697 for Sonnet. Honest caveat in the README: non-inferiority to both rivals does not pass — it supports a cost claim, not parity. This is the tiered router's T1 shape with evidence. | 9/10 |
| 2 | typed-decisions new | extract · chat | Three typed questions per CVE (weakness class, attack vector, remote-unauthenticated) against NVD gold, pre-registered on 224 dev records, scored on 160. Smart lane (GLiClass-large, one call) answers all three in 28 ms median — 96× faster than the Qwen3-4B LLM lane and with probabilities the LLM cannot give; GLiNER2.5-Decide scores 146/160 on weakness. Recorded on an L4. Directly maps to Arbitration/Triage lanes. | 9/10 |
| 2b | request-routing/cost new | encode | bge-m3 nearest-neighbour cost router on SIE Serverless: 20 nearest of a 729-request graded bank, similarity-weighted cheap-model success, threshold set for 98 % quality. On 728 held-out requests GPT-6 Luna→Sol cost $3.19 per 1,000, 38.6 % less than always-Sol at 95.8 % of its quality (77.2 % sent to Luna); Haiku 5.5→Sonnet 5.5 saved 11.7 % at 99.0 %. Router cost $0.013 per 1,000. Directly reusable as the escalate? decision. | 8/10 |
| 3 | agent-action-monitor | encode · score · extract | The DUSK project's gate (github.com/TFT444/DUSK): judges the proposed action against a per-agent behavioural baseline, with enforce vs watch modes. Docker Compose: gate service, self-hosted SIE, n8n, mock downstream; keyless by default. Runnable reference for T6/C3. | 8/10 |
| 4 | stripe-link-fraud | extract · encode · score | Multi-signal → gate → escalate inside one request round-trip. Still the cleanest gate code; now outranked on domain by 1–3. | 8/10 |
| 5 | guardrails new | chat · extract | Seven screens on 4,768 human-labelled prompts, paired bootstrap, pre-registered bars: Qwen3Guard 4B 82.7 % F1 beats GPT-6 Sol (76.1 %) and Haiku 4.5 (76.2 %); GLiGuard 77.7 % at $4.68/M prompts. The methodology to borrow for T1 model selection (mind the ToxicChat non-commercial licence). | 7/10 |
| 6 | threat-report-attck-mapper | generate · extract · encode · score | ATT&CK Enterprise 19.2 mapping with the exact source quote beside every suggestion; every mapping stays needs_analyst_review. Ensemble: Qwen3.6-27B (strict-JSON extraction + verification), GLiNER2, Qwen3-Embedding-8B + jina-colbert-v2 (dense + late interaction), Qwen3-Reranker-4B over 75 fused candidates. Reference for T8. | 7/10 |
| 7 | classify · redact | extract | GLiClass action routing with overlapping labels; PII masking vs Presidio with per-model price. Cheap building blocks for T4 and for sanitising findings before logging. | 6/10 |
| 8 | support-assistant-policy · incident-status-updates | chat | Policy-adherence under pressure (275 attempts) and leak-checking generated text. Patterns for the SIE "reasoning" field AgentVetter logs. | 5/10 |
| 9b | coding-agent-opencode new | chat | Points OpenCode at a laptop server, a self-hosted cluster or SIE Cloud (GLM-5.3-Flash served via SIE Relay) and runs an agent-loop conformance check: streaming tool-call deltas, parallel calls, 64K+ prompts, finish reasons, usage, reasoning field. Relevant to T6. | 5/10 |
| 9 | regulatory-rag · retrieval-ablation | encode · score | LoRA encoder hot-loading; eight retrieval strategies by NDCG@10 on 1,854 SEC queries. rag-params-finder's R2/R4 cross-check baselines. | 5/10 (R-side) |
main). Reproducibility without API spend is a real differentiator.The SIE task layer has ten tasks. Seven of them have a touchpoint in AgentVetter or rag-params-finder; three do not. Task definitions live in the Superlinked SIE hub — this is only the mapping.
| SIE task | AgentVetter / rag-params-finder touchpoint |
|---|---|
| Search | rag-params-finder sweep axes; semantic_novelty signal |
| Document to markdown | T7 bundled-docs scanning |
| Structured output | T4 scanner-output normalisation; the routing envelope |
| Decide new | Candidate for the router's judgement step (T9 on the opportunity map) |
| Classify new | Finding-category triage (T4/T5) |
| Guard content | T1 semantic content-safety scanner |
| Run the agent loop | T6 PreToolUse hook; router T1 judgement |
| Translate · See images · Transcribe audio | — no touchpoint today |
SIE_EMBEDDING_MODEL default (384-d)gen-4b (default) · gen-27bprovider='sie' already maps to/v1/embeddings plus Cohere-compatible /v1/rerank and /v2/rerank are exactly the primitives the semantic_novelty signal relies on — the migration is a base-URL swap, not a rewrite.Three things that bite when wiring a security scanner to a shared inference server.
remote="forbid" on the router's SIE client, log served_by next to the existing timing headers, and keep Model Studio as the only intentionally external hop. The relay default is on, so today a cold start could route router traffic to a partner.SIE_ENDPOINT + SIE_API_KEY explicitly, but with the 0.9.0 env behaviour and the 0.10.1 hosted fallback, pin the base URL explicitly and re-test before upgrading the SDK; (3) Qwen3Guard-Gen-4B and GLiNER2.5-Decide — both on the integration map — changed defaults in 0.10.1.Where the two projects already touch, and what is still open
The T1–T8 / R1–R6 / A·B·C lists from the deep-dive artifact, with effort · cost · risk (1–3) and what changed. Items where SIE now ships a runnable reference are marked.
| # | Repo | Opportunity | Primitive | E·C·R | Oct 2026 status |
|---|---|---|---|---|---|
| T1 | AgentVetter | Semantic content-safety scanner adapter in sandbox/scanners.py | Guard | 2·2·2 | ref exists examples/guardrails: Qwen3Guard 4B 82.7 % F1 leads, GLiGuard 77.7 % is the cheap option; granite-guardian untested there |
| T2 | AgentVetter | Embedding-based finding de-duplication before the heatmap | Encode | 2·1·1 | Unchanged; Qwen3-Embedding-4B now the obvious model |
| T3 | AgentVetter | Known-bad semantic fingerprint index | Encode | 3·2·3 | Partly shipped as semantic_novelty; lookalike-search example shows the hard-negative problem |
| T4 | AgentVetter | Structured extraction from raw scanner output | Extract | 2·1·1 | ref exists structured-output + named-entity-extraction examples |
| T5 | AgentVetter | Score-based finding triage against an operator risk query | Score | 2·1·1 | /v1/score schema now documented — unblocked |
| T6 | AgentVetter | Local agent-loop for the Agent Guard PreToolUse hook | Generate | 3·3·3 | ref exists agent-action-monitor (Docker Compose) |
| T7 | AgentVetter | Document-to-markdown for bundled docs | Extract/OCR | 2·2·1 | docling example + MCP server make this a tool call |
| T8 | AgentVetter | Evidence-grounded mapping onto CWE / OWASP-LLM / ATLAS | 4 primitives | 2·2·2 | Now productised on superlinked.com/security: 697 ATT&CK definitions searched by dense + ColBERT, 75 candidates reranked, 27B verifier re-checks the quote; worked example separates cookie reuse (T1550.004) from theft (T1539). typed-decisions adds the CVE lane pattern |
| T9 new | AgentVetter | Replace the SIE low_confidence chat judgement with a typed Decide call (e.g. GLiNER2.5-Decide: severity class, "scanners contradict?" yes/no) — returns probabilities, ~28 ms in typed-decisions vs ~2.7 s for a small LLM | Extract (Decide) | 2·1·2 | My suggestion from this pass, not in any prior session. Needs a CVE-style dev/test split and pre-registration like typed-decisions; keep the LLM lane as fallback. Corroborated 13 Oct: Superlinked's own native-inference reference pipeline uses GLiNER2.5 Decide for "Intent and complexity — picks the skill, and how strong a model it needs". On /serverless it scores 0.875 accuracy at 13.1 ms and $0.0035 per 1,000 messages vs GPT-6.1 Sol 0.973 at ~1.5 s and $0.53 |
| T10 new | AgentVetter | Pin remote="forbid" and an explicit base URL on the SIE client: scanned skill/MCP content is sensitive, and SIE Relay may answer generation/extraction through a third-party partner during cold starts; v0.10.1 also adds a hosted-service SDK fallback | config | 1·1·1 | My suggestion; needs SDK ≥ 0.10.0. Check X-SIE-Served-By in logs. Note embeddings/rerank and grammar-constrained output are never relayed |
| T11 new | AgentVetter | Neighbour-weighted escalation score (request-routing/cost pattern): bank of past findings with graded "SIE answer held up?" outcomes → escalate when similarity-weighted success < threshold | Encode | 3·2·2 | My suggestion; generalises Darwin Router's self-updating examples with a pre-registered threshold |
| R1 | rag-params-finder | Expand reranker sweep axis (provider='sie') | Score | 1·1·1 | verified _rerank_sie() calls score_documents_sie() for real; whether CI smoke tests still mock the HTTP call was not checked |
| R2 | rag-params-finder | Self-hosted embedding sweep axis | Encode | 3·2·2 | shipped gallery guide: BGE-M3 / Stella-v5 / SPLADE-v3 via example-sie.yaml |
| R3 | rag-params-finder | Golden-dataset auto-tagging | Extract | 2·1·1 | knowledge-graph example (GLiNER2 with offsets) |
| R4 | rag-params-finder | Cross-check vs SIE retrieval-ablation benchmark | — | 1·1·1 | Docs-only pass; NDCG@10 on 1,854 SEC queries |
| R5 | rag-params-finder | Guard-content as an sc_ diagnostic | Guard | 2·1·2 | Needs the "non-LLM evals only" principle check; GLiGuard is a classifier, which may satisfy it |
| R6 | rag-params-finder | OpenAI-compatible endpoint as drop-in | Encode | 1·1·1 | Effectively what the gallery guide documents |
| A1 | AgentVetter | Extract → Encode → Score as one triage pipeline (T4+T2+T5) | multi | 2·1·2 | contract-review-agent is the canonical "ten calls, one cluster" chain; risk compounds across stages |
| A2 | AgentVetter | Guard verdict + fingerprint match merged into one Finding | Guard+Encode | 2·1·2 | Schema/UI merge of T1+T3 outputs, not new inference |
| A3 | rag-params-finder | Fully self-hosted SIE-only sweep profile (no external APIs) | Encode+Score | 3·2·2 | Largely delivered by example-sie.yaml + self-hosted Docker on :8720 |
| A4 | rag-params-finder | Generate-based query rewriting as a sweep axis | Generate | 3·2·2 | First use of Generate in the project; Qwen3.5-4B / Qwen3-4B-Instruct are cheap candidates |
| B1 | both | Sweep AgentVetter's own model/threshold choices with rag-params-finder's Optuna methodology | multi | 3·2·2 | Methodology transplant, needs its own harness; request-routing's score.py is a template |
| B2 | both | Point AgentVetter's scanners at rag-params-finder's plugin ecosystem | — | 2·1·2 | Config-level; real supply-chain value |
| B3 | both | Transplant golden-dataset eval philosophy into AgentVetter | — | 3·1·2 | typed-decisions' pre-registration file is the model to copy |
| B4 | both | Shared internal SIE client/config layer | — | 2·1·1 | Consolidates glue in sie_embedder.py and cli/src/router.js |
| C1 | beyond | Standalone, project-agnostic SIE developer guide | — | 1·1·1 | this hub + the gallery onboarding pages |
| C2 | beyond | Generalised semantic fingerprint-match micro-library | Encode | 3·1·2 | Three potential homes; lookalike-search example documents the hard-negative trap |
| C3 | beyond | Standalone PreToolUse safety-gate package (Guard + Generate, no cloud on hot path) | Guard+Generate | 3·2·3 | agent-action-monitor is the runnable starting point; safety-critical, earliest-stage idea |
"Tripwire is a metal detector for AI tools." Primitives: generate (chat completions via /v1/chat/completions for post-scan triage). Mock demo and Live scans work without routing; agentvetter route / auto-route need both SIE_* and Model Studio keys.
Pages: getting-started · sie-integration · what-sie-does · troubleshooting. Ports: dashboard 8765; SIE hosted (api.superlinked.com / EU). The guide says upstream setup docs "may still call Model Studio optional"; env-vars.md and ADR-0016 now say the same — routing is optional for scans, but both key sets are required once you route (resolveRouteConfig() validates up front). The gallery folder was renamed to examples/agentvetter between 5 and 14 Oct (old path now 404); the guide now uses the AgentVetter name and agentvetter route, but still lists only generate as the primitive.
Sweep embeddings × chunking × retrieval on your data. Primitives: encode (BGE-M3 / Stella-v5 / SPLADE-v3), optional score (SIE rerank). SIE is opt-in — default stack runs without it. MIT. Repo README now also offers Postgres/pgvector (incl. Supabase-hosted) alongside Atlas, and links an "SIE ecosystem demos" page.
Ports: API 8001, dashboard 5374, self-hosted SIE 8720 (not started by ./start-services.sh). Happy path: Mongo stack → dashboard → remote SIE gateway → one example-sie.yaml sweep.
Qwen3.5-4B on SIE reads every result and calls qwen-max when scanners contradict one another, coverage is incomplete or the evidence does not support a clear severity. The dashboard records the full chain including the arbitration note; in one test it returned amber instead of red for a tool-poisoning finding because two of five scanners were unreachable. The post groups AgentVetter (as "AgentVetter") with Darwin Router (Superlinked-track winner, 82.5% routing accuracy, ~60% cost cut) as the two projects that "made the split easiest to see". All nine teams: Darwin Router (Superlinked-track winner), AliBC (Alibaba visual-track winner), Echo (social-media prize), Aida (Mythic; Raspberry Pi Zero 2 W), Sieve (ShieldTech), AgentVetter (TheSandbox; "AgentVetter" in the post) — yours, RentalFinder (Octuple), Venue Visualiser, Yappy.generate as the primitive used — the /v1/embeddings-based semantic_novelty signal (Encode) isn't credited. Worth a one-line PR to examples/agentvetter/what-sie-does.md.provider='sie' → score_documents_sie() path (verified in server/core/rerank/reranker.py on 5 Oct).encode in the AgentVetter guide · Link the hackathon blog post from both guides · Neither is visible on the docs Examples page
Everything this doc cites, plus where the platform detail lives
Split out of the Superlinked SIE learning hub so the platform reference and the AgentVetter integration can be read — and updated — independently · 16 October 2026