For a long time, our job was refreshingly simple: provide reliable hardware in Switzerland and let you build whatever you needed on top of it. That's served a lot of businesses well, and for many use cases it always will. But over the past couple of years, what a server can do for a business has changed dramatically — and we've decided to grow right along with it.

Increasingly, the most valuable thing a server can offer isn't just quietly hosting a website. It's helping your business think — reasoning over everything your company knows and answering questions from it in seconds. Setting that up well, keeping it secure, and running it smoothly takes real expertise, though. So we're expanding what we offer: alongside the infrastructure you already trust us for, we now help design, build, and manage capabilities that deliver value from day one.

The flagship example — and the subject of this post — is private AI: a model that runs on infrastructure you control and has effectively read every document your company has ever produced. And to be clear up front, this isn't just a tool for IT teams. If your business runs on knowledge — a law firm's case files, a factory's manuals, an accountant's playbook, an HR handbook — there's something here for you. Let's walk through the whole picture, in plain language.

1. Two Different Kinds of AI

There's a quiet fork in the road here, and it's worth understanding before you invest anywhere.

The chatbot you rent. When you use a public AI chat service, you're talking to a brilliant stranger with a touch of amnesia. It knows an enormous amount about the world in general, and nothing at all about you. Every conversation starts from zero. It has never seen your pricing, your policies, your client history, or your internal procedures — and everything you type into it leaves your building and lands on someone else's servers. For a fintech, a crypto business, a law firm, or any company handling regulated or confidential data, that alone is often reason enough to look for something better.

The AI that knows your business. The second kind runs on your infrastructure and is connected to your knowledge. Instead of asking a question to a model floating in a data centre you don't control, you give the model a private, searchable memory of everything your company knows. Now when a salesperson asks "what's the best discount I can offer on a three-year deal?" or a brand-new hire asks "what's our standard process for onboarding a client?", the model answers from your actual documents — not from a plausible-sounding guess.

The mechanism that makes this possible has a name, and it's less mysterious than it sounds.

2. What RAG Actually Is (No Jargon)

RAG stands for Retrieval-Augmented Generation. Forget the acronym; here's the idea.

Imagine giving the model an open-book exam instead of a closed-book one. In a closed-book exam, the model answers from memory alone — and when its memory is fuzzy, it can fill the gaps with guesses (this is what people mean by "hallucination"). In an open-book exam, before the model answers, it's allowed to flip to the exact pages of your documents that are relevant to the question, read them, and then base its answer on what it just read.

RAG is simply the machinery that does the "flipping to the right pages" automatically. It works in three steps:

1. Retrieve — When you ask a question, the system searches your company's documents and pulls out the handful of passages most relevant to what you asked.
2. Augment — It quietly hands those passages to the model along with your question.
3. Generate — The model writes its answer using both your question and the real passages it was just given.

The clever part is how it finds the right passages. Rather than a keyword search like Google circa 2005, your documents get converted into vectors — think of them as coordinates that capture meaning rather than exact words. A question about "carrying unused vacation days into next year" will correctly find a document titled "Annual Leave & Carryover Policy" even though almost none of the words match, because the meaning is close. Those vectors live in a vector database, which is really just a memory organised by meaning instead of by filename.

That's the whole trick. No magic, no science fiction — you're building the model a private library and teaching it to look things up before it speaks.

3. What a Basic Setup Looks Like

Here's the reassuring news: the core stack is only three moving parts.

Part 1 — The engine (the model itself). This is the AI brain that does the reasoning and writing. A popular way to run one privately is Ollama, an open-source tool that downloads and runs open-weight models with a single command and handles the GPU for you.

Part 2 — The interface (where people and documents meet). Open WebUI is a clean, familiar web page your team logs into. It's also where you upload documents and configure retrieval. If it isn't quite your taste, the ecosystem has grown up fast: AnythingLLM is excellent for document-heavy, multi-workspace setups, LibreChat is a strong multi-user option, and LM Studio is a friendly desktop app for smaller deployments. They all sit on top of the same kind of engine.

Part 3 — The knowledge (your data, prepared properly). This is where the real work is, and it's the part most people underestimate. Your raw business material — documents, spreadsheets, PDFs, wiki pages — needs to be cleaned and reshaped into tidy, consistent text before it goes into the vector database. Messy input tends to produce a confused model; clean input produces a sharp one.

We can say this with some confidence because we run private AI inside SwissLayer ourselves, for internal use — not something we put in front of our customers. (The technology is genuinely good; people are simply still getting comfortable trusting AI, and that caution is perfectly healthy.) Building it taught us that the glamorous part — the model — is maybe 20% of the effort. The other 80% is quieter and very real: cleaning up messy data so the model stops getting confused, patiently tuning the retrieval settings until they return signal instead of noise, and refining the model's instructions until it stops filling gaps with guesses. We worked through every one of those challenges, and that hard-won experience is exactly what we now bring to the table for clients.

Where It Runs — and What It Costs

You have two comfortable homes for this:

• On a managed server at SwissLayer, in our Swiss facility, where we handle the hardware, security, and uptime and you simply log in. Your data stays in Switzerland.
• On-premise, on a machine inside your own building, when your compliance posture calls for the data never leaving the room.

Either way, the single number that shapes everything is VRAM — the memory on the GPU. VRAM decides how large, and therefore how capable, a model you can run. Here's the map, from a modest pilot right up to an always-on company brain — including the single- and dual-GPU configurations we build:

A helpful rule of thumb: a model at the common Q4 compression level needs roughly (billions of parameters ÷ 2) + about 2 GB of VRAM. So a 32B model lands around 18 GB and fits comfortably on our dual-T4 configuration. Newer "Mixture-of-Experts" models are even more efficient — they hold a lot of knowledge but only draw on a small slice of it per answer, so they punch above their memory footprint.

On cost — and why timing matters. Right now, GPUs are on the expensive side, and prices have been climbing. A structural memory-chip shortage has large AI providers buying up the fast memory these workloads need, which has pushed prices up across the board: flagship consumer cards that once listed around $2,000 have been trading well above $3,000, and this looks like a lasting squeeze rather than a blip, with meaningful new supply not expected until around 2028. Over a five-year horizon the picture is still encouraging: memory capacity will expand, and — just as importantly — the software keeps getting more efficient, so each year the same card quietly runs a smarter model at no extra cost. The practical takeaway is gentle but clear: this may not be the ideal moment to invest four figures in a GPU at the top of the market, whereas running your AI on a managed server you can resize over time lets you benefit as prices ease and models improve. That flexibility is a big part of why we're leaning into managed service.

4. What This Could Do for Your Business

This is the part worth sitting with, because it's where "interesting technology" becomes real time and money saved. A private model that has read your company's knowledge is, in effect, a new colleague who has absorbed every document you own and is always available. A few concrete pictures:

A law firm. A paralegal asks, "Have we argued a non-compete dispute like this before, and how did it go?" The assistant searches years of case files and briefs and surfaces the three most relevant matters in seconds — rather than an afternoon in the archive.

A manufacturer. A technician on the floor asks, "What's the reset sequence and torque spec for the line-3 packaging unit?" and gets the answer pulled straight from equipment manuals and past maintenance logs, without paging through a binder.

An accounting firm. During tax season, junior staff ask, "Which deductions apply to a client structured like this?" and the model answers from the firm's own internal guidance — helping everyone stay consistent with how the firm actually works.

And that only scratches the surface. The same pattern fits almost any business that runs on knowledge:

• Real estate agencies — agents checking property details, contract terms, and disclosure rules in a moment.
• Medical & dental clinics — staff checking clinical protocols and policies (with the strict data controls covered below).
• Architecture & engineering firms — pulling building codes and specifications from years of past projects.
• HR, in any company — employees self-serving the handbook, benefits, and leave policy instead of emailing HR.
• Hospitality & hotels — front-desk staff checking SOPs, guest policies, and vendor terms.
• Insurance — adjusters checking coverage rules and prior decisions quickly.
• Field services — engineers pulling install guides and safety procedures on-site, from a phone.
• Any growing team — new hires who can simply ask the company a question instead of interrupting a senior colleague.

If your people spend real time hunting for answers that already exist somewhere in your files, this technology is a way to give that time back.

5. A Helpful Habit: Build Departments, Not One Giant Brain

A common first instinct is to pour everything into a single knowledge base. In practice, a better pattern is segmentation — separate knowledge bases for separate functions.

• A Sales knowledge base with pricing, product details, and proposal templates.
• An Operations base with runbooks and procedures.
• A Support base with troubleshooting guides and resolved issues.
• A Legal / Compliance base with contracts, policies, and regulatory notes.

Why go to the trouble? Three good reasons:

Sharper answers. When someone asks a sales question, the system searches only sales material, so it isn't distracted by an unrelated operations document that happens to share a few words. Smaller, focused libraries retrieve more accurately.

Access control. Not everyone should query the legal or HR base. Segmentation lets you say "the support team's assistant sees support material and nothing else." That's a genuine security benefit, not just tidiness.

Cleaner maintenance. When a policy changes, you update one base rather than a monolith. Each department owns and curates its own knowledge.

Think of it as building a team of specialists rather than one generalist who has read everything and remembers none of it clearly.

6. Helping the Model Dig Deeper into Your Data

Out of the box, one of these systems does a fairly shallow lookup. Getting it to do a genuine deep-dive on your data takes a little tuning — and in a tool like Open WebUI, these are just settings rather than code. The ones that matter most:

Chunk size and overlap. Documents get sliced into pieces before storage. Slices too big and each is a blurry mix of topics; too small and you lose context. A little overlap between slices keeps ideas from being cut in half at the seam. This is the single biggest lever on answer quality.

Top K — how many passages to pull. This sets how many relevant chunks the model reads before answering. Too few and it misses context; too many and it can drown in noise. It's a dial you tune to your documents.

The embedding model — how "meaning" is measured. This is the component that turns text into those meaning-coordinates. The default is often modest; switching to a purpose-built embedding model noticeably sharpens what gets retrieved.

Hybrid search and re-ranking. The strongest setups combine meaning-based search with old-fashioned keyword search, then use a second small model to re-rank results so the best passage rises to the top. It's the difference between "found something related" and "found exactly the right paragraph."

The instruction template. A short instruction telling the model how to use what it retrieved — to lean on it, and to say "I don't know" rather than guessing when the documents don't cover something. This one change makes a big difference to reliability.

None of this is exotic. It's a handful of settings that separate a toy from a genuinely useful tool — and it's exactly the kind of tuning we're happy to handle for clients so they don't have to learn it the hard way.

7. Security: Two Things Worth Getting Right

Private AI can be more secure than the public alternative — but only when it's set up thoughtfully. Two areas deserve particular care.

7a. Keep the Engine Off the Open Internet

By default, the model engine will answer anyone who can reach it over the network. If it's placed on a public IP without protection, others could potentially use your GPU, see your prompts, or probe your system — and automated scanners look for exactly this all the time.

The good news is that the safeguards are standard, well-understood infrastructure practice:

• Keep the engine private and place it behind a gateway that handles authentication.
• Require a login. Access goes through the web interface with real user accounts rather than an open port.
• Encrypt the connection with HTTPS so traffic can't be read in transit.
• Scope the firewall so only trusted locations (your office, your VPN) can reach it.
• Add a second credential on the engine itself, as an extra layer of protection.

This is bread-and-butter hardening — firewalls, gateways, VPN-gated access, encryption — and it's the same discipline we apply to every production system we run. It's also one of the most common things to overlook in a self-built setup, which is a good reason to have it managed.

7b. Keep Private Data Out of the Model's Memory to Begin With

The subtler consideration isn't the network — it's the knowledge base itself. Once a document is in the vector database, the model can surface anything in it. So if you include a spreadsheet full of customers' names, ID numbers, or card details, the model may repeat them to whoever asks the right question.

The principle is simple: the vector database should hold the knowledge, not the secrets. A support assistant needs to know how a process works — it doesn't need real people's personal data to explain it. Personal information (PII) and sensitive internal material are best removed before anything is added, rather than filtered afterward.

8. Thinking About the "Safety Gate"

That removal happens in what's often called a transform step — a small piece of automation that sits between your raw data and the vector database and tidies every document on the way in. You don't need to build one yourself to understand what it should do; you just need to know it exists and to make sure it's there.

A good safety gate does three jobs, in one direction only:

1. Reshape — Turn messy source material into clean, uniform text the model can digest well. Consistent structure in, sharp answers out.

2. Redact — Detect and remove or mask personal and sensitive data as it passes through:

• Direct identifiers — names, emails, phone numbers, addresses, ID and passport numbers.
• Financial data — card numbers, account numbers, balances, wallet addresses.
• Credentials — anything resembling a password, key, or token (these should never be near a knowledge base).
• Internal-only material — figures or notes best kept out of a general-purpose assistant.

3. Verify — Spot-check the output before it's added. A useful mindset is "assume every document becomes readable the moment it enters the database." If a line wouldn't be appropriate read aloud to the wrong person, the gate should have removed it.

The guiding idea: teach the model your procedures, not your people. A knowledge base that explains how your business works — without carrying the private details of who it works with — is both far safer and, in practice, produces better answers, because it isn't cluttered with data the model never needed anyway.

We're Not Talking About This from the Sidelines

We've built exactly this kind of system ourselves — end to end, on our own hardware, working through every consideration described above. We'll keep what we use it for to ourselves, but we're glad to say it works well, and those hard-won lessons are now built into how we approach it for clients.

That's really what this expansion is about: growing from providing the infrastructure you rely on to also helping you get real value running on top of it. For private AI in particular, there's a clean and reassuring division of labour — we build and manage the infrastructure; you own and control the knowledge. The vector database, the documents, the knowledge base — those remain entirely yours. They live on your managed server, under your control, and never become ours. You're free to experiment, load your data, and shape your own company brain, while we keep the hardware sized right, the security sound, and everything running smoothly.

Whether you'd like a managed GPU server in our Swiss facility — resizable as the hardware market shifts and models improve — or help standing up a private AI on your own premises, we'll take care of the parts that are easy to get wrong and hand you the keys to the part that matters most: your knowledge.

Private AI isn't a research project anymore. It's becoming everyday infrastructure — and infrastructure is what we do best.

Curious what your own private AI would cost and run on? Let's talk.

Written by the team at SwissLayer — Swiss-hosted managed infrastructure for businesses that would rather not hand their data to someone else. GPU pricing and model details reflect the market as of mid-2026 and will change as the memory shortage eases.