Full deployment of an Artificial Intelligence ecosystem on Linux and Docker. No dependency on external services, no data sent to the cloud. 100% private, 100% yours.
Today, data sovereignty and privacy have become fundamental pillars of technological development. This project was born from the need to deploy a high-performance AI ecosystem that operates 100% locally, eliminating cloud service dependency and guaranteeing information security.
The deployment covers the complete configuration process on Linux, using Ollama as the LLM inference engine, Docker + Open WebUI for the multi-user interface, and ComfyUI + Flux.1 for advanced image generation. The goal is to demonstrate that with consumer hardware, professional AI workflows are achievable with minimal latency and total control.
No data leaves the local network. Models run entirely on the local GPU, without external APIs or paid subscriptions.
GPU VRAM is the critical factor. With 16GB, models like Mistral 12B or Flux.1 can be fully loaded into GPU memory, yielding inference speeds far superior to using system RAM.
| Component | Specification | Role |
|---|---|---|
| CPU | Intel i5-13600KF | Auxiliary compute and Docker |
| RAM | 2 × 8GB Kingston Fury 3200MHz | System and containers |
| GPU | NVIDIA RTX 4060 Ti 16GB VRAM | AI inference (CUDA) |
| Storage | 1TB NVMe M.2 Kingston | Models and containers |
| Operating System | Ubuntu / Debian (recommended) | Main host |
For isolating the user interface (Open WebUI) in containers.
Required for ComfyUI virtual environments and PyTorch dependencies.
Updated drivers with CUDA support for GPU acceleration during inference.
Ollama acts as the backend "brain" responsible for running LLM models locally. It installs as a system service and automatically detects the available NVIDIA GPU.
The installer downloads binaries, creates the ollama system user, configures a systemd service with auto-start, and detects NVIDIA graphics hardware.
Make sure you have enough VRAM before downloading large models. Mistral 12B requires ~8GB VRAM at Q4 quantization.
By default, Ollama only listens on 127.0.0.1. To allow the Open WebUI container or other network devices to connect, you need to change the bind address:
Open WebUI is the web interface that connects to the Ollama backend. It deploys as an isolated Docker container with persistent data and starts automatically on system reboot.
The interface is available at http://localhost:8080. On first access, a local admin account is created — no data is sent to external servers.
Active — Port 8080In the Admin Panel → Settings → Connections, configure the Ollama URL:
Open WebUI supports multiple user accounts. Each user can have independent conversations, their own files (RAG), and custom configurations.
ComfyUI is used for image generation — a highly modular node-based interface — together with the Flux.1 model in its quantized (GGUF) version to optimize VRAM usage.
The original Flux.1 Dev model is too large for many configurations. The quantized Q4_K_S version (6.81 GB) is used from HuggingFace, reducing VRAM consumption without noticeable quality loss.
Generation time varies by resolution and sampling steps. On an RTX 4060 Ti 16GB, a 1024×1024 image takes approximately 60 seconds.
With all components installed, the AI ecosystem is fully operational on the local network. Any device connected to the same network (including mobile) can access the chat and image generation interface.
Open WebUI at http://[LOCAL-IP]:8080 — Select Mistral, Llama or another installed model. Upload PDF documents or images to activate RAG capabilities.
ComfyUI at http://[LOCAL-IP]:8188 — Configure the workflow with Flux.1 GGUF, CLIP/T5 and VAE. Write prompts in English for best results.
Connect your phone to the same WiFi and access via the server IP. Create a user account in Open WebUI to have your own independent conversation history.
The system is 100% locally operational. No API keys, no recurring costs, no data sent to third parties. Privacy is the primary feature, not an afterthought.