Running a Local LLM on a No-KYC VPS: What CPU-Only Inference Really Delivers

Yes, you can run a local language model on a no-GPU VPS — but only small, quantized models, and they will feel slow compared to a hosted API. A 1–3B parameter model in 4-bit quantization runs usably on 4 GB of RAM; a 7–8B model needs around 8 GB and produces text noticeably slower than you read; anything larger than that on a shared-CPU VPS is a batch job, not a chatbot. The honest reason people do it anyway is privacy: the prompts never leave a server you control, and on a no-KYC host there's no account identity attached to that server in the first place.

This guide covers what actually fits on each plan size, how to predict speed before you pay, which workloads suit CPU-only inference, and where you're better off with a GPU box.

The short answer by plan size

Language model memory use is dominated by the weights. A 4-bit quantized model (the common Q4_K_M GGUF format) occupies roughly 0.55–0.65 GB per billion parameters, plus a few hundred megabytes for context (the KV cache) and the runtime itself.

That maps to IronBalkans' plans like this:

• Iron 1 (1 vCPU, 1 GB RAM, 15 GB NVMe, €4.99/mo) — not an inference host. It can serve as an API gateway, a queue worker, or a reverse proxy in front of inference running elsewhere. A 0.5–1B model technically loads, but quality at that size is poor for anything beyond classification.
• Iron 2 (2 vCPU, 4 GB RAM, 35 GB NVMe, €9.99/mo) — the realistic entry point. Comfortable for 1–3B models (Llama 3.2 3B, Qwen2.5 3B, Phi-3 Mini) at 4-bit, plus embedding models for search and retrieval. Disk is adequate for two or three model files.
• Iron 3 (4 vCPU, 8 GB RAM, 70 GB NVMe, €19.99/mo) — fits a 7–8B model at 4-bit with a modest context window, with RAM left for a web app and a database. This is the smallest plan where output quality starts resembling what people expect from "an assistant."
• Iron 4 (8 vCPU, 16 GB RAM, 130 GB NVMe, €39.99/mo) — comfortable 8B with long context, or a 13–14B model at 4-bit if you accept slow generation. Eight dedicated vCPUs also mean prompt processing (the compute-heavy part) finishes in seconds rather than tens of seconds.

If you're unsure how your other workloads stack on top of this, the broader no-KYC VPS plan sizing guide covers RAM, disk and bandwidth for non-AI services on the same server.

Why memory bandwidth, not vCPU count, sets your speed

The single most useful thing to understand before buying: during text generation, a CPU reads the entire model from RAM for every single token it emits. Generation speed therefore has a hard ceiling:

max tokens/sec ≈ memory bandwidth (GB/s) ÷ model size in RAM (GB)

A 2 GB model on a system that can sustain, say, 10 GB/s of effective memory bandwidth tops out around 5 tokens/second — and real-world throughput lands below that ceiling because of cache misses, quantization overhead and the fact that DDR4 bandwidth on a virtualized node is shared with neighbouring workloads. Adding vCPUs helps until you saturate bandwidth, then stops helping. That's why an 8 vCPU VPS doesn't generate four times faster than a 2 vCPU VPS on the same model.

Two consequences worth planning around:

1. Smaller models are not just cheaper, they're proportionally faster. Dropping from 8B to 3B roughly halves memory traffic per token and roughly doubles speed.
2. Prompt processing scales differently. Reading a long prompt is compute-bound and parallelizes across cores, so more vCPU genuinely speeds up the "thinking" pause before the first token. If you feed long documents, cores matter; if you send one-line chat messages, they matter much less.

Before committing to a model size, run llama-bench from llama.cpp on the server itself. It reports prompt-processing and token-generation speed separately for your actual hardware in a couple of minutes. If the numbers disappoint, move down a model size before you move up a plan.

Workloads that suit CPU-only inference

Plenty of useful work doesn't need interactive chat speed:

• Embeddings and semantic search. Models like bge-small or all-MiniLM are tens of megabytes and run fast on CPU. A private document search or RAG index over your own notes is genuinely a good fit for an Iron 2.
• Classification, tagging, routing. Sentiment, spam scoring, ticket triage, "which category is this?" — short outputs, so generation speed barely matters.
• Scheduled summarization. Overnight digests of RSS feeds, logs, or scraped pages. Batch jobs don't care that each summary takes 40 seconds.
• Structured extraction. Pulling fields out of invoices or emails into JSON, where a 3B model with a constrained grammar does fine.
• Low-volume personal assistants. One user, no concurrency, patient with the latency.

What CPU-only inference does not do well: multi-user chat (every concurrent request competes for the same bandwidth), long code generation, agentic loops that make dozens of calls per task, or anything where you'd notice the gap between a 3B local model and a frontier hosted model. Be honest with yourself about which category your project is in — no VPS plan fixes a quality gap caused by model size.

What self-hosting an LLM actually keeps private

This is the real reason to do it, so it's worth being precise about the boundaries.

What you gain: prompts, documents and outputs are processed in your own server's RAM. No third-party AI provider retains them, trains on them, or holds them for a legal process against their records. For anyone running client documents, draft legal text, medical notes, or internal business data through a model, that's a meaningful difference from pasting the same text into a hosted chatbot.

What you don't gain: the server is still a machine in a datacenter. Your hosting provider has hypervisor-level access to the disk and, in principle, to memory — the model running there is not hidden from the infrastructure, which is the same honest caveat that applies to any self-hosted service. We covered that boundary in detail in what your VPS provider can actually see. What a no-KYC account changes is not the hypervisor; it's the paperwork. There's no name, email, phone or ID tied to the account, and payment arrives as Monero, Bitcoin or Litecoin rather than a card in your name.

If that trade-off is the one you're optimizing for, signing up at IronBalkans takes a random account ID and a recovery key — no email address, no verification step — and the VPS deploys in under 60 seconds once payment confirms. Full root on real KVM virtualization means you install whatever inference runtime you like with no restrictions on what runs in userspace.

What neither gains you: the network side still exists. If you expose an inference API to the internet and call it from your home connection, the model is private but the access pattern isn't. Keep the listener on localhost or behind a VPN — see anonymous VPS access and home IP correlation for the reasoning.

A sane setup, briefly

You don't need a complicated stack:

• Runtime: llama.cpp (via its server binary) or Ollama, which wraps it. Both run GGUF models on CPU with no GPU dependency.
• Model: download a 4-bit GGUF from the model's official repository. Verify the hash; check the model licence if the output is going anywhere commercial.
• Binding: bind the API to 127.0.0.1, never 0.0.0.0. Ollama in particular defaults to an unauthenticated API — there are plenty of internet-exposed instances precisely because people skipped this.
• Access: reach it over WireGuard or an SSH tunnel, or put a reverse proxy with authentication in front of it.
• Isolation: run it as a dedicated non-root user under a hardened unit file. systemd service sandboxing applies cleanly here, since an inference server needs almost no system access beyond reading its model directory.
• Memory safety: cap the unit's MemoryMax so a long-context request can't take the whole box down. On tight RAM, read the low-RAM VPS tuning notes first — a model that swaps doesn't run slowly, it effectively stops.

Common mistakes buyers make

• Buying by vCPU count. Eight cores won't rescue a model that exceeds your memory bandwidth. Size RAM for the model, then add cores for prompt processing.
• Forgetting the KV cache. Context isn't free. A long context window can add a gigabyte or more on top of the weights, and it's the usual cause of a model that loads fine then dies mid-conversation.
• Underestimating disk. Model files are 1–9 GB each. On a 15 GB NVMe, two models and a system install leave nothing. Iron 2's 35 GB is the practical floor if you want to try several.
• Expecting hosted-API quality. A 3B model is not a frontier model. It summarizes, classifies and extracts well; it does not reason through hard problems.
• Planning for concurrency. Two simultaneous users on a CPU-only server roughly halve each one's speed. If you need concurrency, you need a GPU, not a bigger VPS.

Honest pros and cons

Pros: your data stays on infrastructure you control; flat monthly cost instead of per-token billing; no rate limits, content filters or API key tied to an identity; the same server can host your app, your database and your model together.

Cons: meaningfully slower than GPU inference; limited to smaller models; no GPU option on a standard VPS; you own all the maintenance; and the hypervisor-level trust assumption doesn't disappear just because the model is local.

FAQ

Can I add a GPU to a VPS? Not to a standard KVM VPS — GPU inference needs physical hardware passed through to the instance. If your workload genuinely requires it, a GPU cloud is the right tool, and you'll generally be trading privacy for speed since most GPU providers require full account verification.

Which model should I start with? On 4 GB, try a 3B instruct model at Q4_K_M. On 8 GB or more, an 8B instruct model. Benchmark both if you can; the quality jump from 3B to 8B is usually more noticeable than the speed penalty.

Does Romania hosting affect inference? Not the compute. It affects latency for the API round-trip, which is irrelevant next to generation time. Bucharest is well-positioned for European users if you're also serving a web app from the same box.

Can I pay monthly and cancel if it's too slow? Billing is flat monthly with no card on file and no auto-renewal, so a single month is the natural trial. Start with Iron 2, benchmark honestly, and scale up only if the numbers justify it.

Get started

If you want a private inference box without handing over an identity to get one, pick a plan at ironbalkans.com/#pricing — Iron 2 at €9.99/mo for small models and embeddings, Iron 3 or Iron 4 if you want 8B-class output. Signup needs no email or ID, payment is in XMR, BTC or LTC, and the server is yours in under a minute. Benchmark before you build on it, and size the model to the hardware rather than the other way around.