Self-hosting AI means running the application yourself, on a server you control, instead of renting a seat in someone else’s product. A small Linux VPS can host a chat interface that works like ChatGPT, an automation tool that runs AI agents, and a full platform for building your own AI apps. Your chat history, documents, and workflows stay on your server.
This guide walks through the whole setup on a Linux virtual private server: choosing the right size, securing it, deploying with Docker Compose and automatic HTTPS, and avoiding the mistakes that waste most people’s first weekend.
Quick Outline
- How to Self-Host AI Applications: Choosing Where the AI Model Runs
- Best AI Applications to Self-Host on a Linux VPS
- How to Choose the Right Linux VPS for AI Applications
- How to Secure Self-Hosted AI Applications on a Linux VPS
- How to Back Up and Update Self-Hosted AI Applications
- Troubleshooting Common Linux VPS and AI Deployment Issues
- Conclusion
- Common Asked Questions
How to Self-Host AI Applications: Choosing Where the AI Model Runs
Before you pick a plan, answer one question. Will the AI model run on your VPS, or will your VPS call a model hosted somewhere else?
Most AI applications you self-host are really just the interface and the logic. They send prompts to a model through an API and show the reply. That workload is light, and a small server handles it fine. Running the model itself is a completely different job that needs a lot of RAM, and ideally a GPU. People buy a 1 GB server, install a local model, and wonder why nothing works.
For a first project, let the VPS run the application and call an outside model through an API key. Add a local model later if privacy or cost demands it. Our guides on how much RAM and CPU AI models need and on running a private ChatGPT-style assistant cover that second path in detail.
Best AI Applications to Self-Host on a Linux VPS
The best AI application depends on the job, so match the tool to the task.
Tool | What it does | Typical use |
|---|---|---|
A browser chat interface, an application like ChatGPT | Private team chat, document questions | |
Workflow automation with AI agent nodes | A self-hosted AI agent that reads input, calls tools, and acts | |
Dify | Visual platform for building LLM apps and agents | Internal assistants and retrieval apps |
Ollama | Runs open models locally | The model backend for any of the above |
Dify is the heaviest of these. Its published minimum is 2 CPU cores and 4 GiB of RAM, and a practical starting point is 4 vCPUs and 8 GB. Open WebUI and n8n are far lighter, which is why this guide builds them first.
How to Choose the Right Linux VPS for AI Applications
A Linux VPS gives you a slice of a physical machine with root access, and the right size depends on what you run.
VPS size | Good for |
|---|---|
1 vCPU, 1 to 2 GB RAM | One light app calling a hosted model API |
2 vCPU, 4 GB RAM | Open WebUI plus n8n for personal or small team use |
4 vCPU, 8 GB RAM | Dify, or several apps plus a small local model |
8+ vCPU, 16+ GB RAM, or a GPU | Local models you actually rely on |
When comparing Linux VPS providers, look past the headline price. Check that the plan uses KVM virtualization, offers NVMe storage, includes snapshots, gives you a data center near your users, and allows hourly or monthly billing so you can resize.
Most Linux VPS hosting plans let you pick Debian or Ubuntu, and both work well here. Debian VPS hosting suits you if you prefer a lean, stable base, and a VPS with Ubuntu LTS has the widest set of tutorials and GPU driver support. Every command below works on either.
Can You Self-Host AI on a Free Linux VPS?
A free VPS server on Linux exists, with limits worth knowing. Oracle’s Always Free tier includes Arm-based Ampere A1 capacity, commonly described as up to 4 cores and 24 GB of RAM, though Oracle documents it in monthly usage hours and what you actually receive depends on region capacity and account approval.
Expect to provide payment details, and expect idle instances to be reclaimed. Google Cloud offers an always-free e2-micro with 1 GB of RAM in select US regions. AWS and Azure now lean on time-limited plans and credits rather than permanent free servers.
Free Linux VPS hosting works for learning and for light apps that call an outside model. It is a poor fit for running models. If you use an Arm instance, confirm each Docker image publishes a linux/arm64 build before you plan around it.
Step 1: Secure Your Linux VPS Before Deployment
Log in over SSH and update the system:
sudo apt update |
|---|
Create a normal user, copy your SSH key to it, and disable root login and password authentication in /etc/ssh/sshd_config. Then set up the firewall:
sudo ufw allow OpenSSH sudo ufw allow 80/tcp sudo ufw allow 443/tcp sudo ufw enable |
|---|
Optional: Small servers also run out of memory quickly, and the kernel kills processes when that happens. A modest swap file is a cheap safety net for the application layer:
sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile && sudo swapon /swapfile echo ‘/swapfile none swap sw 0 0’ | sudo tee -a /etc/fstab |
|---|
Step 2: Install Docker and Docker Compose on Linux
Let’s install Docker and Docker Compose on a Linux VPS.
curl -fsSL https://get.docker.com | sudo sh sudo usermod -aG docker $USER |
|---|
Log out and back in, then confirm Docker and Compose work:
docker –version docker compose version |
|---|
Step 3: Configure DNS for Your Self-Hosted AI Applications
Create two DNS A records, such as chat.example.com and agents.example.com, both pointing to your VPS IP address. Automatic HTTPS in the next step needs these to resolve before you start.
Step 4: Deploy AI Applications Using Docker Compose
Now, deploy AI applications using Docker Compose on Linux.
1. Configure Open WebUI and n8n
Create a project folder and a compose file:
mkdir -p ~/ai-stack && cd ~/ai-stack nano docker-compose.yml |
|---|
services: caddy: image: caddy:2 restart: unless-stopped ports: – “80:80” – “443:443” volumes: – ./Caddyfile:/etc/caddy/Caddyfile – caddy_data:/data – caddy_config:/config open-webui: image: ghcr.io/open-webui/open-webui:main restart: unless-stopped environment: – OPENAI_API_BASE_URL=https://api.openai.com/v1 – OPENAI_API_KEY=OPENAIAPIKEY-WEBUISECRETKEY={WEBUI_SECRET_KEY} volumes: – webui_data:/app/backend/data n8n: image: docker.n8n.io/n8nio/n8n restart: unless-stopped environment: – N8N_HOST=agents.example.com – N8N_PROTOCOL=https – WEBHOOK_URL=https://agents.example.com/ volumes: – n8n_data:/home/node/.n8n volumes: caddy_data: caddy_config: webui_data: n8n_data: |
|---|
2. Configure Caddy as a Reverse Proxy
Notice that Open WebUI and n8n publish no ports at all. Only Caddy faces the internet, and it reaches the other containers by name over the internal Docker network. That choice matters, and the security section below explains why.
Swap in the base URL and key for whichever OpenAI-compatible model provider you use. Then create the Caddyfile:
chat.example.com { reverse_proxy open-webui:8080 } agents.example.com { reverse_proxy n8n:5678 } |
|---|
3. Store API Keys and Application Secrets Securely
Store your secrets in a .env file next to the compose file:
echo “OPENAI_API_KEY=your-key-here” > .env echo “WEBUI_SECRET_KEY=$(openssl rand -hex 32)” >> .env chmod 600 .env |
|---|
Step 5: Start Your AI Applications and Configure Admin Accounts
Let’s start your AI applications and configure admin accounts.
docker compose up -d && docker compose ps |
|---|
Caddy requests and renews Let’s Encrypt certificates automatically, so visit https://chat.example.com after a minute. The first account you create in Open WebUI becomes the administrator.
Once your own account exists, turn off open registration in the admin settings so strangers cannot sign up. Do the same owner account setup in n8n at https://agents.example.com.
Step 6: Build Your First AI Agent with n8n
An AI agent in n8n is a workflow where a model decides which tool to use. Start with something small and useful. Add a webhook trigger, connect an AI Agent node to your model provider, give it one tool such as an HTTP request or a database lookup, and send the result to email or chat.
A support inbox summarizer or a daily report generator makes a good first project. Keep the agent’s permissions narrow, since it acts on whatever you connect it to.
How to Secure Self-Hosted AI Applications on a Linux VPS
Docker publishes container ports by writing its own firewall rules, and those rules bypass UFW. If you had published port 5678 for n8n, it would be open to the whole internet even though UFW never allowed it. That is why the compose file above exposes nothing except Caddy. If you ever need a direct port for testing, bind it to localhost only, written as 127.0.0.1:5678.
Two more habits protect you. Set a monthly spending limit on your model provider account, because a leaked API key can run up a large bill within hours. And remember that webhook URLs in n8n are public by design, so add authentication to any workflow that does something sensitive.
How to Back Up and Update Self-Hosted AI Applications
Your data lives in Docker volumes, so back them up on a schedule. Run docker volume ls to see the exact names, since Compose prefixes them with the folder name, then archive each one:
docker run –rm -v ai-stack_n8n_data:/data -v $(pwd):/backup alpine \ tar czf /backup/n8n-$(date +%F).tar.gz -C /data . |
|---|
Back up the n8n volume without fail. It holds the encryption key for your saved credentials, and losing it means re-entering every connection. Copy the archives off the server too, because a backup on the same disk is not a real backup.
Once everything works, pin image tags to specific versions so a surprise update cannot break a working setup.
Troubleshooting Common Linux VPS and AI Deployment Issues
- Caddy cannot get a certificate. The DNS records have not propagated, or ports 80 and 443 are blocked at the provider’s firewall. Check Docker Compose logs for Caddy.
- Containers keep restarting or disappearing. The server is probably out of memory. Check free -h and docker stats, then move to a bigger plan or drop a service.
- n8n says it cannot reach the webhook URL. The WEBHOOK_URL value is missing or does not match your real domain, including the trailing slash.
- Open WebUI shows no models. The API key or base URL is wrong. Confirm both in .env and recreate the container.
Conclusion
Self-hosting AI applications on a Linux VPS is far more approachable than it looks, as long as you separate the application from the model. Run the interface and agents on a modest server, let a hosted API do the heavy thinking, and put everything behind Caddy so nothing else touches the internet. Secure the box before you deploy, set a spending cap on your API account, and back up your volumes. Once that foundation is solid, you can swap in local models later without rebuilding anything.
Common Asked Questions
1. Can I self-host AI on a free VPS?
You can run light AI applications on a free Linux VPS, especially ones that call an outside model through an API. Free tiers are too small to run useful models locally, and terms such as idle reclaim, capacity limits, and credits can change, so check the provider’s current rules.
2. How much RAM does a Linux VPS need to self-host AI applications?
Plan on 2 GB for a single light app, 4 GB for Open WebUI plus n8n, and 8 GB for Dify. Running a language model on the server needs far more, often 16 GB or a GPU, depending on the model.
3. Is Self-Hosting AI Cheaper Than a ChatGPT Subscription?
It can be. A small VPS costs a few dollars a month, and you pay your model provider only for the tokens you use. Running the model locally removes per-token fees but needs expensive hardware, so the savings depend on how heavily you use it.
4. Should I Choose Debian or Ubuntu for an AI VPS?
Either works. Ubuntu LTS has the broadest tutorials and the easiest GPU driver support. Debian is a lean, stable choice if you prefer a minimal base. The Docker commands in this guide are identical on both.
5. How Do I Secure a Self-Hosted AI Application?
Use SSH keys, keep UFW on, and publish only ports 80 and 443 through a reverse proxy, since Docker ports bypass UFW. Disable open sign-ups, store keys in a locked .env file, set a spending cap with your model provider, and back up your volumes.
Leave feedback about this