Sovereign AI infrastructure — open source demonstration

High-performance On-Premise AI
platform

Design, deployment and operation of a local and sovereign AI infrastructure — from low-level hardware up to LLMs and RAG pipelines. An open source, documented and reproducible build: what is shown here works, and demonstrates that an organization can host its own AI while keeping control of its data, with no cloud dependency.

Proxmox VE 9.2 RTX 5090 VFIO Passthrough Ollama + LLM RAG Vectoriel Docker ZFS 192 Go DDR5

Approach & philosophy

📖
Shared reference notes
This portfolio is my technical logbook. Each page documents what I learned, how I solved a problem, and the resources that helped me. Freely accessible.
🌿
Open Source first
Every tool chosen in this portfolio has a free and open alternative. Git, Proxmox, Ollama, Python, nginx — transparency and community are at the heart of every technical decision.
🔧
From the field to production
26 years of experience — 7 years in systems security, 19 years as a project manager in an analysis laboratory. What is presented here was designed, deployed and maintained in real conditions.

Areas of expertise

Architecture — Overview
Overview
The platform in layers · from hardware to consumer AI
Overview→
Hardware Infrastructure
Infrastructure
AMD Ryzen 9 9950X3D · RTX 5090 · 192 Go DDR5
Advanced→
OSS
Proxmox VE Hypervisor
Virtualization
Proxmox 9.2.2 · KVM · QEMU · Templates
Advanced→
OSS
Reverse Proxy
Infrastructure
VM200 · Nginx · TLS · Certbot · Fail2ban · HTTPS publishing
Advanced→
GPU Passthrough VFIO
Virtualization
VFIO · IOMMU · PCIe Passthrough RTX 5090
Advanced→
OSS
ZFS Storage
Storage
AI datasets · Snapshots · Storage architecture
Intermediate+→
OSS
Operating Systems
OS
Debian · Ubuntu · Alpine · Windows 11
Intermediate+→
OSS
Local AI & LLM
AI / ML
Ollama · Open WebUI · Llama · Mistral · Gemma
Intermediate+→
OSS
RAG & Vector Databases
AI / ML
PostgreSQL · pgvector · bge-m3 · HNSW
Basic+→
OSS
Generative Image AI
AI / ML
ComfyUI · Flux · GPU Computing
Basic+→
OSS
Speech AI — Whisper
AI / ML
Speech-to-Text · multimodal AI · audio processing
Basic+→
OSS
AI Agents — Hermes & OpenCode
AI / Agents
VM301 · Linux agentic station · OpenCode · Hermes · remote GPU inference
Advanced→
AI Dev Station — DeepSeek Harness
Dev / AI
VM302 · Windows 11 · VSCodium · Python · Node.js · DeepSeek Harness (preliminary)
Advanced→
MIT
Second Brain OpenCode
AI / Agents
Markdown kit · durable memory · OpenCode · human validation · MIT license (prototype)
Intermediate→
OSS
Docker & Containerization
DevOps
Docker Compose · Isolation · AI services
Intermediate→
OSS
Nextcloud
Services
Self-hosted files · Docker · MariaDB
Intermediate→
FAIR-CODE
n8n — Automation
Services
Workflows · Docker · external PostgreSQL · Ollama API
Intermediate→
Network
Network
Bridge Proxmox · SSH · Firewall · NAT
Intermediate→
OSS
Python
Dev
venv · pip · requirements.txt · Scripts IA
Basic+→
OSS
Git
DevOps
GitHub vs GitLab · self-hosted GitLab · repository hygiene
Intermediate→

Levels of mastery

Virtualization / ProxmoxAdvanced
GPU Passthrough VFIOAdvanced
AI infrastructureAdvanced
Systems security (Antivirus, Firewall)Advanced
IT project managementExpert
Linux (Debian/Ubuntu)Intermediate+
Windows Server/DesktopIntermediate+
Local AI (LLM / Ollama)Intermediate+
Docker / ContainerizationIntermediate
Nextcloud / Self-hostingIntermediate
DevOps (IaC, templates)Intermediate
ZFS / StorageIntermediate
CUDA / NVIDIAIntermediate
Git / GitLab / GitHubIntermediate
RAG / pgvector / EmbeddingsBasic+
Python / venv / pipBasic+

Project presentation

Context: Design of a high-performance local AI platform based on Proxmox VE, an NVIDIA RTX 5090 GPU in VFIO passthrough, Windows/Linux virtualization, ZFS storage, Docker infrastructure, hosting of local LLMs (Ollama), vector RAG (PostgreSQL/pgvector), n8n automation and multimodal tools (Whisper, ComfyUI).

Purpose: to demonstrate that an organization — a local authority (town hall) or an SME — can host its own AI and automation services while remaining sovereign over its infrastructure and data. Some building blocks are in production, others validated on the design platform or being deployed.

infrastructure → virtualization → deployment → AI operations, built entirely on open source tools.