One open-source inference cluster for every small model an agent calls — plus a hosted Cloud and an MCP edge
Apache 2.0 inference server + production cluster. Encode, Score, Extract, Generate through one API. OpenAI-compatible endpoints. Laptop → Kubernetes with the same image.
77 docs pages · Helm + Terraform (GKE/EKS/AKS/ACK) · 3,360 ★ · 317 forks · 18 open issues
Same API, no cluster. Sign up at console.superlinked.com, get an API key, $20 in credits. SOC 2 Type 2, US/EU endpoints, inference-grant programme for selected projects.
api.superlinked.com (us-east-2) · eu.api.superlinked.com · keys sk-sie-… · credit-priced per model · marketed as SIE Serverless (launch pending) · docs: Cold starts & SIE Relay
Self-hostable MCP server (packages/sie_mcp) that offloads document conversion, image description, doc-QA, summarisation, NER and PII masking from Claude / any MCP client to your cluster — saves agent tokens.
Added v0.6.10 · expanded v0.6.13 · blog walkthrough Jun 2026
/tasks shows four cards; the /models footer says "13 tasks, 85+ models". Partner docs (Chroma, Qdrant, LanceDB) still say 85+./serverless and /native-inference product pages — both marked noindex (pre-launch). Still no public price list: pricing appears only as per-task comparisons and a "$50 example credit balance". superlinked.com/cloud and /cloud/quickstart (referenced by external project configs) remain unopened.superlinked PyPI page says three capabilities
Your three prior sessions (12–20 Aug) surveyed SIE at ~2.1–2.4K ★ and v0.7.0-ish. Here is the delta against the live surfaces on 5 Oct 2026.
examples/README.md; most are "recorded examples" with pinned HF evidence. External project guides are now a listed category. Docs page still shows only 8 + 12.examples/request-routing (embedding classifier + conditional generation, 3,000-case verification) and examples/typed-decisions (CVE triage with fast/smart/LLM lanes) are now direct references for the tiered routerexamples/guardrails scores seven screens on 4,768 human-labelled prompts: Qwen3Guard 4B 82.7 % harmful-F1 vs GPT-6 Sol 76.1 %, Claude Haiku 4.5 76.2 %, GLiGuard 77.7 % ($4.68 per million prompts), OpenAI Moderation 66.3 %. README Guard row now lists qwen3guard-gen-4b first, then granite-guardian-2b, opir-multitask-large, opir-edge. granite-guardian is not in the benchmark table.api.superlinked.com (us-east-2) + eu.api.superlinked.com; SOC 2 badge in every footer; inference-grant form offers "Managed inference" or "Agent plugin"; footer tagline "Run it yourself, or let us run it for you"request-routing/cost and coding-agent-opencode.pip install "sie-server[local]" && sie-server serve is the first line of the README quickstart (Python 3.12); MLX generation backend since v0.6.15/v1/rerank + /v2/rerank (v0.6.21); /v1/audio/transcriptions; /v1/responses; /v1/generate streams via SSESIE_MODEL hard-errors when unset; T2 = qwen-maxSIE_MODEL defaults to gen-4b; MODEL_STUDIO_MODEL defaults to qwen3.8-max; keys are sk-sie-… from console → Keys; US api.superlinked.com, EU eu.api.superlinked.comexamples/README.md · A "what's new" digest on the docs home (the launch banner is 8 months old) · Third-party caches carry stale star/fork counts (1.9K, 2.7K, 262 forks) — the site nav (3.4K) and GitHub (3,360) now agree
Docs release-notes page (current: "Latest version v0.10.1, 2026-10-13") plus GitHub release pages. v0.8.x–v0.10.1 were missing from hub v1.0–v1.2; v0.10.x from v1.3.
| Release | Date | What matters to a builder |
|---|---|---|
| v0.7.2 / 0.7.3 | 27 Aug / 3 Sep | Qwen3.8-27B-FP8 (256K-ctx profiles) · Alibaba ACK/OSS Helm · MADLAD translation · Rust/Candle worker · opt-in TensorRT-LLM bundle |
| v0.8.0 | 23 Sep | One model across several GPUs (tensor parallelism) · GLM-5.3-Flash on 8 accelerators · Qwen3-VL-8B (video) · translategemma-4b-it · HA Helm values · MCP OAuth-origin hardening · breaking extra_launch_args may not carry placement/listener flags |
| v0.8.1–0.8.3 | 23–26 Sep | 0.8.1/0.8.2 release-pipeline fixes · 0.8.3: GLiFormer, GLiClass prompts/label groups, Opir, GLiNER relex, GLiNER bi-encoder v2 + PII, Laya and GLiNER2.5-Decide typed-decision models, CUDA-graph replay, 2 MiB item-size cap |
| v0.9.0 | 30 Sep | Thinking profiles for Qwen3.8-27B-FP8 · cached-token usage · operator-defined upstreams · TopK-Embed-V1 · Hy-MT2-1.8B · breaking (self-hosted Helm) NATS auth on by default, no default gateway Ingress, split admin tokens, bounded pool API; SDKs auto-use SIE_API_KEY for the SIE_BASE_URL origin |
| v0.10.0 | 9 Oct | New models: SAM 3 (boxes, needs HF token + Meta licence), GLiNER2.5 small/base, GLiNER2 PII multi, OpenAI privacy-filter, Parakeet-TDT 0.6B v3 (25 EU languages), Qdrant SPLADE-ecommerce, Qwen3.5-122B-A10B-FP8 (2×80 GB, text-only), Qwen3Guard-Gen-0.6B and -4B, ZeRank 2 · self-hosted remote profiles (SIE or OpenAI-compatible upstreams, cold-model fallback; threshold routing and saturation spill opt-in) · long-document fixes for GLiNER/GLiNER2 · Whisper concurrency · SIGTERM drain |
0.10.0 upgrade notes LoRAs with PEFT bias="all"/"lora_only" no longer load · grammar-constrained generation now defaults temperature and penalties to 0 · gliner-biomed default threshold 0.8 · GLiNER2 returns per-item INPUT_TOO_LONG instead of partial success · config writes reject undefined upstreams · Qwen3.8 JSON-schema output compact, single image up to 3,211,264 px · OWLv2 fast processor | ||
| v0.10.1 | 13 Oct | LightOnOCR-3-4B (grounding / no custom prompt) · superlinked/pii-redact (combined GLiNER + NuNER spans in one extract call) · opt-in include_reasoning: true returns reasoning_content (buffered and streaming) · Gemma 4 31B high-res thinking profiles · GLM-5.3-Flash tool calling · remote profiles can declare max_seq_length · OCR pages complete independently · Laya graphs and Qwen3-reranker fast profiles (opt-in; scores can shift) · TypeScript extract no longer needs labels |
0.10.1 upgrade notes local chat now adds json_schema to the prompt by default (schema_in_prompt: false to opt out) · self-hosted SDK users must set SIE_BASE_URL or an explicit base URL with their key to avoid the new hosted-service fallback · TypeScript threshold now actually applies · GLiNER2.5-Decide uses CUDA graphs by default (scores may change; cuda_graphs: off) · GLM-OCR fast image processor default (−44 % GPU time per photo, text may change) · Qwen3Guard-Gen-4B streams every 16 decode steps · profile validation rejects output caps above the served context window |
--nccl-port).From superlinked.com/serverless, /native-inference and the 13 Oct community-channel update. pre-launch both product pages carry noindex; the community post is not public documentation.
| Option | What it is (their words, condensed) |
|---|---|
| Serverless | "Pay per call, no GPUs to run": one API, 21 task recipes, 129 models, 9 integrations. SOC 2 Type 2, US or EU residency, SDK/API/CLI/MCP, "zero lock-in — self-host the same stack". Live code runner is a waitlist; example: $50 credit ≈ 1.3B tokens embedded, 120k pages parsed, 27M tokens generated. Migration snippets from OpenAI, Anthropic, Bedrock and Textract. |
| Native Inference | A platform licence for SIE Cluster with Superlinked engineers in a shared Slack, helping answer customers' security reviews; SIE stays Apache-2.0. Loop: Eval → Choose → Post-train → Ship. Deploy in your cloud or your customer's, incl. air-gapped; Terraform for AWS, GCP, Azure, Alibaba Cloud; per-tenant worker pools. Case study: GoodData (founder interview "coming soon"). |
| SIE Cluster / Local | Cluster: Terraform + Helm, scale-to-zero, Apache-2.0. Local: pip install "sie-server[local]" && sie-server serve (Apple Silicon / NVIDIA) or ghcr.io/superlinked/sie-server:latest-cuda13-sglang-cu130. |
superlinked.com/home-next (not opened)./security, /sre.Qwen/Qwen3.5-4B while the recipes use Qwen3.8-27B-FP8.| You are... | Start with | Then add |
|---|---|---|
| Hackathon / solo builder with no GPU | SIE Cloud ($20 credits) or sie-server[local] on a Mac | Swap base URL later — same SDK calls on a cluster |
| Team replacing OpenAI/Cohere embeddings | Encode via /v1/embeddings drop-in | Score reranker via /v1/rerank (Cohere shape) |
| RAG team tuning retrieval | Retrieval-ablation benchmark example | Parameter sweeps over SIE encode/score (external guides) |
| Security / AI-safety tooling | Guard content + typed-decisions + agent-action-monitor | Tiered scan routing · threat-report-attck-mapper (worked example) |
| Agent platform team | Generate (Qwen3.8-27B) + structured output | contract-review-agent pattern: ten model calls, one cluster |
| Document pipeline owner | Extract → OCR (LightOnOCR / docling / MinerU) | SIE MCP server so your LLM calls it as a tool |
| Platform / infra engineer | Helm + Terraform (GKE/EKS/AKS/ACK) | KEDA scale-to-zero · GitOps config · multi-GPU |
/tasks page is the real persona entry point, and it lives outside the docs./tasks from the docs sidebar · State Python 3.12 / CUDA 12.9 / Docker requirements in one prerequisites box
Four verbs, one SDK, 100+ swappable models · plus the task layer built on top of them
Dense, sparse (SPLADE/bge-m3), multi-vector (ColBERT/ColQwen), multimodal (SigLIP/CLIP), Matryoshka + int8/binary quantisation, MUVERA.
POST /v1/encode/:model · /v1/embeddings
Cross-encoders (bge-reranker-v2-m3, Qwen3-Reranker 0.6B/4B, R3), multi-vector late-interaction scoring, image+text rerankers (Qwen3-VL-Reranker).
POST /v1/score/:model · /v1/rerank · /v2/rerank
Zero-shot NER (GLiNER 2.x/2.5, NuNER), relations (GLiREL), classification (GLiClass, NLI), vision (Florence-2, OWLv2, Grounding DINO), OCR (LightOnOCR, GLM-OCR, MinerU, PaddleOCR-VL, docling), audio (Whisper).
POST /v1/extract/:model · /v1/audio/transcriptions
Open LLMs via SGLang/MLX: Qwen3.8-27B-FP8, Qwen3.6-27B, Qwen3.5-4B, Gemma 4, Qwen3-0.6B. Tool calls, JSON-schema/regex/grammar output, streaming, LoRA per request.
POST /v1/generate/:model · /v1/chat/completions · /v1/responses
from sie_sdk import SIEClient
from sie_sdk.types import Item
c = SIEClient("http://localhost:8080") # or api.superlinked.com + API key
v = c.encode("sentence-transformers/all-MiniLM-L6-v2", Item(text="Hello"))
s = c.score("cross-encoder/ms-marco-MiniLM-L-6-v2",
Item(text="What is ML?"),
[Item(text="ML learns from data."), Item(text="Sunny today.")])
e = c.extract("urchade/gliner_multi-v2.1",
Item(text="Tim Cook is the CEO of Apple."),
labels=["person", "organization"])
g = c.generate("Qwen/Qwen3-0.6B", "Capital of France, one word.",
max_new_tokens=16, temperature=0.0)Accept: application/json for JSON. /v1/generate wants the SIE-safe id (Qwen__Qwen3-4B-Instruct-2507); the SDKs convert slashes for you./v1/generate and the "chat/completions is gateway-only (dev convenience on sie-server)" caveat are documented but easy to miss. GLiNER 2.5 thresholds changed in v0.6.10 — only in release notes.The README frames SIE by agent task; every model name links to its config, whose sie_id is what you pass to the SDK. Rewritten 5 Oct from the live README — the earlier 5-row version was stale.
| Task | What it does | Models named | Primitive |
|---|---|---|---|
| Search | Embed, match, rerank | bge-m3, splade-v3, colbertv2, qwen3-reranker (4B) | encode score |
| Document to markdown | PDFs, Office files, scans → markdown | lightonocr, glm-ocr, mineru, paddleocr-vl, docling | extract |
| Structured output | Schema-valid JSON, extracted or generated | gliner2, gliner-relex, gliformer, nuner-zero, qwen3.8-27b, qwen3.6-27b | extract generate |
| Decide new | Choice, yes/no and score answers with probabilities to typed questions about a text or JSON state | laya, laya-multilingual, laya-typed-decisions, gliner2.5-decide, gliner2.5-multi-decide, gliner2.5-decide-1b | extract |
| Classify new | Zero-shot labels; several label groups in one call; instruct models take few-shot examples | gliclass-large-v3, gliclass-instruct-large, gliclass-multilang-mini | extract |
| Guard content | Safe / Unsafe / Controversial with categories; or Yes/No at a config threshold; or safe/unsafe scores with a per-request threshold | qwen3guard-gen-4b, granite-guardian-2b, opir-multitask-large, opir-edge (GLiGuard is in the catalog and the guardrails example) | generate / extract |
| Run the agent loop | Plan steps and call tools with an open LLM, streaming | qwen3.8-27b, qwen3.6-27b | generate |
| Translate | 400+ languages | madlad400-3b-mt (translategemma-4b-it and Hy-MT2-1.8B added in v0.8.0 / v0.9.0) | generate |
| See images | Caption, detect, visual QA | florence-2, owlv2, grounding-dino | extract |
| Transcribe audio | Speech to text | whisper-large-v3-turbo | extract |
sie_id is what you pass) is exactly what a builder needs./tasks 4 cards (Search /encode, Document to markdown /extract, Guard content /generate, Run the agent loop /generate), /models footer "13 tasks", docs sidebar 4 primitives. Guard content is tagged /generate on /tasks although GLiGuard and Opir are served through /extract./tasks cards /guard-content and /run-the-agent-loop are now /guardrails and /chat in the nav) ·
3 packages/sie_server/models (configs) ·
4 examples/guardrails
Superlinked's own pre-launch comparisons on /serverless (14 Oct). Quality metrics differ per task; prices are per the unit shown. This table keeps the losses in — they are the useful part.
| Task | SIE model | SIE price · quality · latency | Best / notable alternative | Read |
|---|---|---|---|---|
| Search | Qwen3-Embedding-4B | $0.06/1M tok · 0.546 · 16 ms | Voyage 4 Large 0.547 ($0.12); OpenAI 3-large 0.508 ($0.13, 238 ms) | parity, half price |
| Rerank | Qwen3-Reranker-4B | $15/100k queries · 0.912 · 55 ms | GPT-6 Luna high 0.984 ($86); Voyage rerank-3 0.926 ($17); Cohere 4 Pro 0.842 ($250) | cheapest, not best |
| Sparse | OpenSearch doc-v3-gte | $0.029/1M · 119 (BM25 = 100) | Pinecone sparse $0.08 | win |
| Multi-vector | GTE-ModernColBERT | $2.11 / 11.8k articles · 0.616 | voyage-context-4 0.641 ($10.15); Cohere Embed v5 Pro 0.442 ($9.47) | near, 1/5 price |
| Image search | SigLIP so400m | $24 / 1M photos · 0.832 | Cohere v5 Pro 0.825 ($335); captions with Sol/Sonnet $3,025+ | win |
| Visual doc search | Tomoro ColQwen3 4B | $0.20 / 1k pages · 0.641 | Cohere v5 Pro 0.648 ($0.74); OCR + OpenAI emb. 0.560 ($1.57) | parity |
| OCR → Markdown | LightOnOCR-3 | $2.60 / 1k pages · 0.851 | Sonnet 5.5 0.841 ($25.42); Mistral OCR 4.1 0.833 ($4.00) | win |
| Doc field extraction | Gemma 4 31B | $2.11 / 1k docs · 0.943 | GPT-6.1 Sol 0.961 ($14.63); Luna high 0.923 ($1.12) | near, 1/7 price |
| Structured output | Qwen3.8 27B | $727 / 1M docs · 0.835 exact values | Sonnet 0.772 ($7,680); Luna 0.766 ($227) | best quality; Luna cheaper |
| Chat | Qwen3.8 27B | $135 / 100k convs · 0.989 rules kept · 0.74 s | Sol 1.000 ($504); Haiku 5.5 0.993 ($63); Luna 0.993 ($45) | fast; Haiku/Luna cheaper |
| Named entities | NuNER Zero-span | $0.05/1M · F1 0.630 · 24 ms | Sonnet 5.5 0.796; Sol 0.792; AWS Comprehend 0.390 | quality gap |
| Knowledge graph | GLiNER relex Large | $48 / 1M articles · 110 (Sonnet = 100) | Sol 103 ($6,147); Sonnet $8,853 | win |
| Guardrails | Qwen3Guard 4B | $24.38 / 1M checks · F1 0.827 | Haiku 5.5 0.760 ($14.45); Luna 0.783 ($37.96); Sol 0.787–0.790 ($290–586); OpenAI Moderation 0.663 (free) | best F1; Haiku cheaper |
| Redact (PII) | GLiNER PII + NuNER Zero | $73 / 1M docs · 0.909 · 61 ms | Sol 0.957 ($2,487); Haiku 0.944 ($188); AWS Comprehend 0.842 ($351); OpenAI Privacy Filter 0.697 ($23) | mid quality, low price |
| Typed decisions | GLiNER2.5 Decide | $0.0035 / 1k msgs · 0.875 · 13 ms | Sol 0.973 ($0.53); Sonnet 0.963 ($1.05); Luna high 0.958 ($0.037); latencies ≈ 1–1.5 s | 100× faster, −8 pts |
| Detect | SAM 3 | $0.25 / 1k photos · F1 0.62 · 83 ms | Sol high 0.60 ($59.98, 80.7 s); Sonnet 0.39 | win |
| Image classify | SigLIP so400m | $0.0233 / 1k images · 0.723 | Sonnet 0.923; Haiku 0.906; Rekognition 0.268 | quality gap |
| Image questions | Qwen3.8 27B | $0.42 / 1k · made-up answers 0.475 (lower is better) | Haiku 0.793; Sol 0.806; Sonnet 0.846 | fewest fabrications |
| Speech to text | Parakeet TDT 0.6B v3 | $45 / 1,000 audio h · WER 0.081 | GPT-Transcribe 0.092 ($270); 4o-mini-transcribe 0.099 ($99) | win |
| OCR (photos) · Screenshots | GLM-OCR · Gemma 4 31B | $0.25 / 1k photos · 0.710 | $1.88 / 1k shots · 0.941 | Sol low 0.782 ($6.64) · Sol medium 0.962 ($6.07); Haiku answered only 78/100 screenshots | cheaper, slightly lower |
/serverless but 1.000 on /security (and 683 ms vs 0.74 s). Guardrails ground truth matches the gallery example (F1 0.827 vs the example's 82.7 %), but GPT-6.1 Sol appears as 0.787–0.790 here and "Sol 76.1 %" in the example (different reasoning settings).Laptop in 2 minutes · Cloud in 30 seconds · Kubernetes with one Helm command
# 1. macOS (Apple Silicon) or Linux, native — Python 3.12
pip install "sie-server[local]" && sie-server serve
# 2. Linux, NVIDIA GPU (encode/score/extract)
docker run --gpus all -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cuda12-default
# 2b. Generation image (Qwen/Gemma via SGLang)
docker run --gpus all -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cuda12-sglang
# 3. Linux, CPU — enough for dev and the tiered-router smoke test
docker run -p 8080:8080 \
-v sie-hf-cache:/app/.cache/huggingface \
ghcr.io/superlinked/sie-server:latest-cpu-default
curl http://localhost:8080/readyz # expect: okSIEClient("https://api.superlinked.com", api_key=...) (EU: eu.api.superlinked.com; keys are sk-sie-…). Since v0.9.0 both SDKs also read SIE_API_KEY from the environment when the URL shares the origin of SIE_BASE_URL; v0.10.1 adds a hosted-service fallback — self-hosted users must now set SIE_BASE_URL or an explicit base URL.fetch.py + score.py reproduce published numbers with no key.default, transformers5 (LightOnOCR, GLM-OCR), sglang (generation), sglang-vision-extract, sglang-cu130 (was cuda13-gemma, renamed v0.7.2), opt-in TensorRT-LLM (v0.7.3).Install the SDK: pip install sie-sdk · npm install @superlinked/sie-sdk. Notebook: examples/quickstart.ipynb (runs on a free Colab GPU).
sie-server[local], Python 3.12), each copy-pasteable; first-call weight download behaviour is explained. Verified against the live README.api.superlinked.com (us-east-2) and eu.api.superlinked.com; keys sk-sie-… from console → Keys. Unverified: superlinked.com/cloud and /cloud/quickstart.SIE_API_KEY header name explicitly
The first page in the docs' new SIE Cloud section. Dedicated GPU capacity can scale to zero; while it restarts, SIE Relay may answer through a vetted inference partner instead of making you wait.
| Question | Answer (docs, 14 Oct) |
|---|---|
| When is a request relayed? | Only before dedicated capacity accepts it (never runs twice), when the model has scaled to zero or is starting. Same validation, same response format, same price; latency and output "can differ slightly". |
| Never relayed | Embedding and rerank requests (Relay serves only generation and extraction) · requests with X-SIE-Remote: forbid · org opted out · named dedicated profiles · grammar-constrained structured output · batch/jobs APIs · no partner in your key's region. |
| Relay-only models | Some models/profiles are served only via Relay and labelled so in the console (e.g. zai-org/GLM-5.3-Flash:relay in the OpenCode example). |
| Who served it? | Headers X-SIE-Served-By (local/remote), X-SIE-Upstream (sie-relay), X-SIE-Fallback-Reason (provisioning), X-SIE-Fallback-Error. SDK ≥ 0.10.0 exposes them as result["request"] (served_by, upstream, …) / result.request. |
| Data handling | "Partners state that they keep no copy of the requests". Availability is per region; an unpinned key is checked against the deployment that receives the request. |
from sie_sdk import SIEClient
# Every request from this client stays on dedicated capacity
client = SIEClient(
"https://api.superlinked.com", # be explicit (v0.10.1 hosted fallback)
api_key=os.environ["SIE_API_KEY"],
remote="forbid", # sends X-SIE-Remote: forbid
)
r = client.generate("Qwen/Qwen3.5-4B", "…", max_new_tokens=64)
print(r["request"].get("served_by")) # "local" expectedremote="forbid" (client-wide — there is no per-call option, so use a second client) · org-wide in console Settings → SIE Relay (owners/admins), after which relay-only models answer 403 THIRD_PARTY_SERVING_DISABLED.503 PROVISIONING + Retry-After (5–300 s on native endpoints, ≤ 60 s on OpenAI-compatible ones). SDKs wait up to 900 s by default (provision_timeout_s / provisionTimeout). v0.10.1: Python makes one final attempt after the last wait, so it can overshoot the budget by one attempt timeout.ProvisioningError carries no request metadata in current SDKs.remote option · Offer an opt-in default for orgs with data-residency rules · Cross-link from the Native Inference security-review pagefrom openai import OpenAI
client = OpenAI(api_key="sk-...")
r = client.embeddings.create(
model="text-embedding-3-small",
input=["Hello world"])from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1",
api_key="unused-locally")
r = client.embeddings.create(
model="BAAI/bge-m3",
input=["Hello world"])Same response objects. Works with LangChain OpenAIEmbeddings too. Cohere clients get /v1/rerank and /v2/rerank (v0.6.21).
LangChain, LlamaIndex, Haystack, DSPy, CrewAI — Python + TypeScript guides. sie-langchain gives SIEEmbeddings and SIEReranker (a ContextualCompressionRetriever drop-in).
Chroma (sie-chroma, dense + sparse functions, on Chroma's own docs), Qdrant (native sparse), Weaviate (v4 client, enrichment), LanceDB (incremental table enrichment).
"SIE vs hosted embedding APIs: ~97% of the quality at ~1/12th the cost" and "SIE vs OpenAI: use OpenAI for frontier models, SIE for private open-model inference" — the two posts that justify a tiered design.
/v1/chat/completions) is documented in the API reference but the OpenAI → SIE migration guide is embeddings-centric.dimensions on /v1/embeddings is now rejected if unsupported (v0.7.2) rather than silently downcast
# pick one overlay: values-gke.yaml / values-aws.yaml / values-aks.yaml
# pin a chart version for reproducible installs (README example: --version 0.7.3)
helm upgrade --install sie-cluster \
oci://ghcr.io/superlinked/charts/sie-cluster \
--namespace sie --create-namespace \
--set hfToken.create=true --set hfToken.value=$HF_TOKEN \
-f https://raw.githubusercontent.com/superlinked/sie/main/deploy/helm/sie-cluster/values-gke.yaml
# Terraform modules: terraform-google-sie · terraform-aws-sie · terraform-azure-sie · terraform-alicloud-sie (Alibaba ACK)
# v0.7.2: Alibaba Cloud ACK Helm settings + OSS object storage + RRSA identitypool · machine_profile · bundle; you pay for GPU only while inference runs. Provisioning returns 503 + Retry-After (v0.6.5, was 202) — the SDKs wait up to 900 s by default.helm upgrade.| Page | What you'll learn |
|---|---|
| Docker · GCP · AWS · Azure | per-cloud overlays, workload identity, cert-manager TLS |
| Offline / Air-gapped | pinned, verified artifacts; local cache serving (v0.7.3) |
| Scale-from-Zero & Autoscaling | KEDA, min warm workers, static pools |
| Hardware & Capacity · Multi-GPU | L4 24GB holds "several standard models"; multiple worker children per sidecar (v0.6.17) |
| Monitoring · Usage & token counting | OTLP → Prometheus 9464, Grafana, optional Tempo tracing |
| Upgrade Runbook · Config GitOps | profile hot reload, config service, breaking-change checklist |
Engine docs cover Architecture, Model Adapters, Gateway, Adding Models, Config API, Model Profiles, LoRA Adapters, Bundles. Evals docs cover quality, performance and custom evals — "every supported model has quality and latency targets checked in CI".
0.7.3 while the latest release is 0.9.0 · Alibaba Cloud ACK now has a Terraform module (terraform-alicloud-sie) and OSS/RRSA Helm support but no docs deployment page — the docs nav lists only Docker, GCP, AWS, Azure
100+ configs in packages/sie_server/models/ · every one has CI quality + latency targets · compare at superlinked.com/models/compare
Model names as they appear in release notes and the gallery. The HF id is what you pass to the SDK; the config's sie_id is the short alias.
SIE_EMBEDDING_MODEL default (384-d)provider='sie' reranker setting maps to by defaultgen-4b · gen-27b/v1/extract with labels safe / unsafe57 posts on the index (54 since the Feb 2026 launch) · comparisons, benchmarks, talks
Endpoints, errors, headers, env vars, links
v1.4, 14 Oct 2026. ✓ checked against a live source in this or the previous pass · partial real gaps · open needs you.
| Area | Covered in | Status | What is still thin |
|---|---|---|---|
| Product packaging: Serverless, Native Inference, Cluster, Local | §01 | ✓ | Pages are pre-launch (noindex); no price list; /home-next unopened |
| Releases to v0.10.1 incl. breaking/upgrade notes | §01 | ✓ | — |
| Community update 13 Oct (launch plan, asks, events) | §01 | partial | Only the pasted message; replies and thread context unseen; not public documentation |
| Task recipes with evals (23 task pages) | §02 | partial | All eval cards read from /serverless; the 23 task pages themselves, 7 industry pages, /sre and /home-next not opened |
| Four primitives, API schemas, error codes | §02, §08 | ✓ | API reference not re-fetched since 5 Oct; new 0.10.x request fields come from release notes |
| Quickstart, migrate, deploy; SIE Relay docs | §03 | ✓ | Docs sub-pages (Docker, GCP, AWS, Azure, tuning…) not individually read; no Cloud quickstart page |
| Model catalog + new 0.10.x models | §04 | ✓ | Catalog page not re-fetched on 14 Oct; counts disagree (129 / 100+ / 85+) |
| Gallery (53 rows) re-ranked; new cost-router and OpenCode examples | §05 | ✓ | 9 of 53 READMEs read in full or in part; the rest known from the table row |
| Reference pipeline vs router; T9–T11 | §06 | ✓ | T9–T11 are suggestions, not decisions |
| Hackathon recap (nine teams) | §05, §06 | ✓ | Not re-fetched 14 Oct |
| Integration map T1–T11 / R1–R6 / A·B·C | §06 | ✓ | From conversation excerpts, not the original deep-dive file |
| Visual track (assets, clip) | §06 | open | No record anywhere |
| Blog / comparisons | §07 | partial | Blog index from 5 Oct (no new posts checked); only the recap read in full |
| Doubleword content | — | none | Template used for structure only |
| Source | Result |
|---|---|
| Docs release-notes page | Current to v0.10.1; includes new SIE Cloud nav section |
| /serverless, /native-inference, /security | Read in full; eval tables, deploy options, reference pipeline captured |
| docs/cloud/sie-relay | Read in full |
| GitHub: gallery README, README, raw guides | 53 rows; request-routing/cost and coding-agent-opencode read |
| PyPI / npm | sie-sdk, sie-server and @superlinked/sie-sdk = 0.10.1 (13 Oct) |
| GitHub release tags | v0.10.0 and v0.10.1 present |
Carried from 5 Oct: docs nav (77 pages), blog index, /tasks, /models, docs/examples, recap.
| Item | Why |
|---|---|
| /home-next, /sre, 7 industry pages, 21 of 23 task pages | Announced in the community update; fetches are very large — their evals appear on /serverless instead |
| GitHub stars / forks / issues | HTML structure changed; could not re-parse (site nav still 3.4K) |
superlinked.com/cloud, /cloud/quickstart | Referenced by external project configs; never opened |
| New blog posts after 5 Oct | Blog index not re-fetched |
| HAMi / llm-d comparisons | Mentioned as upcoming; no page exists yet |
| Cloud price list | None public |
"Latest release v0.9.0" → v0.10.1 · "docs have no Cloud section" → one-page SIE Cloud section · "docs release-notes page lags GitHub" (wrong: stale fetch) · v0.7.0 date 9 Aug → 8 Aug (docs) · Gemma 4 31B now verified · model count 129 / 100+ / 85+ made explicit · 51 → 53 gallery rows.
| Endpoint | Where | Purpose |
|---|---|---|
POST /v1/encode/:model | server + gateway | dense / sparse / multivector |
POST /v1/score/:model | server + gateway | rerank (sorted, with rank) |
POST /v1/extract/:model | server + gateway | entities, relations, classes, objects, data |
POST /v1/generate/:model | server + gateway | text (preview); stream:true → SSE |
GET /v1/models[/:model] | server + gateway | catalog, state, revision, profiles |
POST /v1/embeddings | server + gateway | OpenAI-compatible (≤256 inputs) |
POST /v1/audio/transcriptions | server + gateway | OpenAI-compatible Whisper |
POST /v1/chat/completions | gateway (+server dev) | OpenAI-compatible chat, images inline |
POST /v1/completions · /v1/responses | gateway only | OpenAI-compatible |
POST /v1/rerank · /v2/rerank | gateway (+server dev) | Cohere-shape rerank |
/healthz · /livez · /readyz | all / server / all | probes (livez is GPU-aware) |
WS /ws/status · /ws/cluster-status | server / gateway | 200 ms status stream |
| Error code | HTTP | Retry? |
|---|---|---|
| MODEL_NOT_FOUND | 404 | no (nearby matches listed) |
| INVALID_INPUT · INPUT_TOO_LONG | 400 | no |
| MODEL_LOADING · LORA_LOADING | 503 | yes, Retry-After |
| MODEL_LOAD_FAILED | 502 | no — terminal |
| QUEUE_FULL · RESOURCE_EXHAUSTED | 503 | yes |
| INFERENCE_ERROR · INTERNAL_ERROR | 500 | case by case |
Headers to log per routing decision: X-Total-Time, X-Queue-Time, X-Inference-Time, X-Trace-ID, X-SIE-Worker. Metrics via OTLP → Prometheus :9464 (sie_gateway_requests_total, sie_worker_batch_size …).
Telemetry opt-out: SIE_TELEMETRY_DISABLED=1 or DO_NOT_TRACK=1. Pinned models: SIE_PINNED_MODELS / SIE_PRELOAD_MODELS (--model removed in v0.2.0).
/v1/rerank is described as "common wire shape" without naming Cohere, which the release notes do.