The building blocks (static version)
Open WebUI — Chat interface. The web application people actually type into. It handles user accounts, stores every conversation, and forwards messages to the model server. Think of it as your private ChatGPT page.
When it's exposed: Left open, strangers can read conversation history and sign up to use your models.
Fix card →
AnythingLLM — Chat + RAG app. A chat app that also does the document plumbing: ingests your files, embeds them, retrieves the relevant chunks at question time, and assembles the prompt for the model.
When it's exposed: Its default single-user mode has no login at all — anyone can use your models and read your documents.
Fix card →
Gradio — Demo UI. The quickest way to put a web interface on a model — the default choice for demos and prototypes. Often shipped "temporarily" and then forgotten, still public, six months later.
When it's exposed: Your ML demo is public — free compute for strangers, uploads and all.
Fix card →
Ollama — Model server. The engine that runs the language model on your machine and answers over an API. Ships with no authentication — it assumes it will only ever talk to localhost, an assumption that breaks the moment it touches a network.
When it's exposed: Anyone who finds it can list, run, copy or delete your models and burn your GPU for their own workloads.
Fix card →
vLLM — Model server (high throughput). A model server built for serving many requests at once — the production-grade choice when Ollama's single-stream approach becomes the bottleneck.
When it's exposed: Unauthenticated inference — free compute for whoever finds it first.
Fix card →
ComfyUI — Image workflow UI. A visual editor where people wire image-generation pipelines out of nodes (load model, prompt, sample, save). Hugely popular with the Stable Diffusion crowd, and its open API can queue arbitrary jobs on your GPU.
When it's exposed: Open workflow API — arbitrary image jobs on your GPU, plus known RCE history.
Fix card →
Qdrant — Vector database. Where document embeddings live — numeric fingerprints of meaning. It answers "which chunks are most similar to this question" in milliseconds. That retrieval step is what turns a chatbot into chat-with-your-documents.
When it's exposed: No auth by default — every embedding (read: your documents, in retrievable form) readable and deletable by anyone.
Fix card →
Chroma — Vector database. The lightweight vector database — the one tutorials reach for first, so it's the one most often found running with its defaults in production.
When it's exposed: Embeddings readable and deletable by anyone — auth is opt-in, not default.
Fix card →
Weaviate — Vector database. A vector database with more database around it — schemas, filters, hybrid search. Same retrieval job as Qdrant and Chroma, bigger feature surface to misconfigure.
When it's exposed: Schema and every vector open until you enable authentication yourself.
Fix card →
n8n — Workflow orchestrator. The engine that connects apps and runs the automation: "when this happens, ask the model, then do that." It stores the credentials for every system it touches — which makes it the highest-value block in any stack.
When it's exposed: An unclaimed setup page means the first visitor becomes admin and owns every stored credential.
Fix card →
Flowise — Agent builder (visual). Drag-and-drop builder for LLM chains and agents — wire prompts, tools and models together without writing the glue code.
When it's exposed: No auth by default — chatflows, credentials and model keys behind it.
Fix card →
Langflow — Agent builder (visual). Another visual agent/flow builder, same job as Flowise. These tools hold the API keys for everything they connect to.
When it's exposed: Flows and API keys behind it, with unauthenticated RCE in older versions.
Fix card →
Dify — LLM app platform. A full platform for building and operating LLM apps — prompts, RAG, agents, and the console to manage them.
When it's exposed: An unfinished setup means the first visitor becomes admin and owns your API keys.
Fix card →
Ray — Compute cluster. A distributed-compute framework: teams use it to scale training and data jobs across a cluster. Its dashboard lets anyone submit jobs — and a 'job' is arbitrary code, which is why an open Ray dashboard counts as remote code execution.
When it's exposed: Open dashboard — unauthenticated job submission is remote code execution on your cluster.
Fix card →
MCP servers — Tool access. Small services that give the model capabilities — read a file, run a query, call an internal API. MCP is the protocol that standardizes how models discover and call them. Most are built for localhost and exposed by accident.
When it's exposed: Strangers calling your tools, not just reading data — filesystem, databases, whatever you wired up.
Fix card →
Langfuse — Tracing / flight recorder. Records every prompt, response, and tool call so you can reconstruct what the system did and why. Debugging tool today, audit evidence tomorrow — it's increasingly the first thing auditors ask for.
When it's exposed: Reachable tracing dashboard — your prompts, outputs, and user data on display.
Fix card →
Jupyter — Notebook server. Interactive Python in the browser — where a lot of AI work actually happens before anything is "deployed". A notebook server is a shell on your machine with a friendlier face.
When it's exposed: An open notebook server is a terminal on your machine. Bots hijack these constantly.
Fix card →
OpenAI API — Model API (cloud). The model runs on OpenAI's infrastructure, not yours — you send prompts over HTTPS and pay per token. Nothing to install, patch, or GPU for; the trade-off is that your data leaves your perimeter on every request.
Cloud service: Nothing for an outside scanner to find — the exposure questions are contractual instead: data-processing agreement, retention settings, region, and whether prompts may be used for training.
Anthropic API — Model API (cloud). Claude as a managed API — same shape as OpenAI: their GPUs, their uptime, your prompts crossing the wire. The self-hosted vs cloud choice is usually about data residency and cost at scale, not capability.
Cloud service: Nothing for an outside scanner to find — the review is contractual: DPA, retention, region, and training-data usage terms.
Pinecone — Vector database (cloud). A managed vector database — the retrieval layer for RAG without running Qdrant yourself. Your document embeddings live in their clusters; you query over HTTPS with an API key.
Cloud service: Nothing to probe from outside — but your documents, in embedding form, now sit with a third party. Region selection and the DPA are the controls that matter.
LangSmith — Tracing (cloud). Managed tracing for LLM apps — the hosted counterpart to self-hosted Langfuse. Every prompt and response you trace is stored on their platform, which is worth remembering when the traces contain user data.
Cloud service: No dashboard to find from outside — but trace contents (prompts, user data) live with the vendor. Retention and access controls are configuration, not network, questions.
Cloud server (EC2 etc.) — Rented compute. The building block of cloud infrastructure: a virtual machine in someone else's datacenter that you configure and patch yourself. 'We run it in the cloud' usually means a box like this — and everything you install on it is your responsibility, including the locks.
Cloud service: The provider secures the datacenter; everything you install on the machine — including open AI services — is on you. Cloud does not mean someone else handles exposure.
Amazon Bedrock — Model platform (cloud). AWS's managed AI service: pick a model (from Anthropic, Meta, Amazon and others) and call it over HTTPS, billed per request. It sits between 'rent a model API from the model maker' and 'run it yourself' — your prompts stay inside your AWS account's region, which is why enterprises reach for it.
Cloud service: Prompts stay inside your AWS account and chosen region — but that region choice, model access scopes, and logging settings are configuration decisions someone must make and review.
Azure OpenAI — Model API (cloud). The same OpenAI models, delivered inside your organization's own Azure tenant and region. Enterprises choose it because the contract, billing and data handling sit inside their existing Microsoft agreement instead of a new vendor relationship.
Cloud service: Prompts stay within your Azure tenant and region per Microsoft's terms — but access scopes, content filters and logging are configuration someone must own and review.
Google Vertex AI — Model platform (cloud). GCP's managed AI platform: pick a model, call it over HTTPS, billed per use. Like Bedrock for AWS shops, it keeps prompts inside your Google Cloud project and chosen region, with access controlled through IAM.
Cloud service: Region choice, IAM roles and service-account scopes decide who and what can reach it — configuration decisions that belong in a review, not just a console.
Alibaba Model Studio — Model platform (cloud). Alibaba Cloud's managed AI service, common in APAC deployments. The Qwen models it serves are the same weights many teams self-host with Ollama — so the cloud-vs-self-host choice here is often direct, with identical model quality.
Cloud service: Region and jurisdiction decide which laws govern your prompts. For EU companies, where the data is processed is a legal question before it is a configuration one.
Server / GPU — Hardware. The machine everything runs on — a cloud VM or a box under a desk. Not an AI component, but it's where the bill and the risk live: every other block runs as a process here.
Your documents — Data. Contracts, wikis, customer records — whatever you feed the system. This is the block compliance cares most about: everything else in the diagram exists to serve or protect it.
Auth proxy / SSO — Access boundary. A reverse proxy or identity layer (oauth2-proxy, an SSO gate, basic auth at the load balancer) that sits between the internet and your blocks. The single most common real-world fix for 'this service has no login': you don't rebuild the service, you put a boundary in front of it.
Private network / VPN — Network boundary. WireGuard, Tailscale, or a classic VPN: the stack moves off the public internet entirely and only reachable machines can see it. The strongest mitigation there is, because there is nothing left to attack from outside — at the cost of convenience for every legitimate user.
Rate limit / quota — Consumption boundary. Request quotas at the proxy or app layer. It doesn't hide the service, but it bounds the damage an open one can do: the metered-API abuse and GPU-burning scenarios stop being unbounded.
Audit log / retention — Evidence. Centralized logs with a retention rule: access logs, admin actions, model calls. It prevents nothing by itself, but it's the difference between 'we were breached sometime last year, maybe' and 'on March 3rd at 14:02 this IP did exactly this.' Auditors ask for it before almost anything else.