Compare commits
2 Commits
3c645cb215
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| cad658fd5d | |||
| 8bd45f7e07 |
255
Docker/proxmox-node/Proxmox.md
Normal file
255
Docker/proxmox-node/Proxmox.md
Normal file
@@ -0,0 +1,255 @@
|
|||||||
|
# Runbook: Docker Host Setup with RBAC and Node Exporter on Proxmox
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This runbook describes how to prepare a Proxmox host to run Docker services safely and predictably, using:
|
||||||
|
|
||||||
|
* ZFS-backed storage (not the OS disk)
|
||||||
|
* Role-based access control (RBAC)
|
||||||
|
* Non-root daily administration
|
||||||
|
* Docker Engine installed from the official repository
|
||||||
|
* Node Exporter as a first monitoring workload
|
||||||
|
|
||||||
|
The goal is to support **multiple administrators**, minimise risk, and avoid VM sprawl for simple host-level services.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
This runbook covers:
|
||||||
|
|
||||||
|
* Storage decisions on a Proxmox host
|
||||||
|
* Admin group and user setup
|
||||||
|
* Docker installation and configuration
|
||||||
|
* Docker data-root relocation to ZFS
|
||||||
|
* Node Exporter deployment via Docker Compose
|
||||||
|
|
||||||
|
This runbook does **not** cover:
|
||||||
|
|
||||||
|
* Prometheus or Grafana configuration
|
||||||
|
* Firewall rules
|
||||||
|
* Proxmox cluster configuration
|
||||||
|
* Advanced Docker security hardening
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Assumptions
|
||||||
|
|
||||||
|
* Proxmox VE is already installed and operational
|
||||||
|
* A ZFS pool exists (e.g. `/tank`)
|
||||||
|
* SSH access to the Proxmox host is available
|
||||||
|
* Root access is available for initial setup
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## High-Level Design
|
||||||
|
|
||||||
|
### Storage Model
|
||||||
|
|
||||||
|
* Proxmox OS and system services remain on the OS disk
|
||||||
|
* ZFS pool is used for:
|
||||||
|
|
||||||
|
* Docker engine data
|
||||||
|
* Docker service directories
|
||||||
|
* Proxmox-managed VM storage remains isolated (e.g. `tank/vmdata`)
|
||||||
|
|
||||||
|
### Access Model (RBAC)
|
||||||
|
|
||||||
|
* One top-level admin group with full control
|
||||||
|
* Sub-admin groups for scoped access (Docker, VMs, monitoring)
|
||||||
|
* Users operate as themselves, not as root
|
||||||
|
* Root access is only used when required
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Admin Groups
|
||||||
|
|
||||||
|
Create the following Unix groups:
|
||||||
|
|
||||||
|
* `sysadmin` – full administrative access (sudo)
|
||||||
|
* `docker-admin` – Docker administration
|
||||||
|
* `vm-admin` – VM and Proxmox-related tasks
|
||||||
|
* `monitoring-admin` – monitoring-related services
|
||||||
|
|
||||||
|
> Groups represent **roles**, not individuals.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## User Setup
|
||||||
|
|
||||||
|
* Create named user accounts for administrators
|
||||||
|
* Add full administrators to:
|
||||||
|
|
||||||
|
* `sysadmin`
|
||||||
|
* `docker-admin`
|
||||||
|
* Sub-admins may be added only to the groups they require
|
||||||
|
|
||||||
|
### Sudo Access
|
||||||
|
|
||||||
|
* Grant full sudo access to the `sysadmin` group via `/etc/sudoers.d/`
|
||||||
|
* Avoid per-user sudo rules
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Base Directory
|
||||||
|
|
||||||
|
All Docker-related data must live under the ZFS pool.
|
||||||
|
|
||||||
|
Recommended structure:
|
||||||
|
|
||||||
|
```
|
||||||
|
/tank/docker
|
||||||
|
├── engine/ # Docker internal data-root
|
||||||
|
├── node-exporter/ # Monitoring exporter
|
||||||
|
└── <future-services>/
|
||||||
|
```
|
||||||
|
|
||||||
|
Ownership and permissions:
|
||||||
|
|
||||||
|
* Owner: `root`
|
||||||
|
* Group: `docker-admin`
|
||||||
|
* Permissions: `2775` (setgid enabled)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Installation
|
||||||
|
|
||||||
|
Install Docker Engine from the **official Docker repository**.
|
||||||
|
|
||||||
|
Reasons:
|
||||||
|
|
||||||
|
* Predictable updates
|
||||||
|
* Supported versions
|
||||||
|
* Includes Compose plugin
|
||||||
|
|
||||||
|
Packages installed:
|
||||||
|
|
||||||
|
* docker-ce
|
||||||
|
* docker-ce-cli
|
||||||
|
* containerd.io
|
||||||
|
* docker-buildx-plugin
|
||||||
|
* docker-compose-plugin
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Data-Root Configuration
|
||||||
|
|
||||||
|
Docker’s internal data-root must be moved off the OS disk.
|
||||||
|
|
||||||
|
Configure Docker to use:
|
||||||
|
|
||||||
|
```
|
||||||
|
/tank/docker/engine
|
||||||
|
```
|
||||||
|
|
||||||
|
via `/etc/docker/daemon.json`:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"data-root": "/tank/docker/engine",
|
||||||
|
"group": "docker-admin"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker Socket Permissions
|
||||||
|
|
||||||
|
Docker access is controlled via the Unix socket:
|
||||||
|
|
||||||
|
```
|
||||||
|
/var/run/docker.sock
|
||||||
|
```
|
||||||
|
|
||||||
|
To align with RBAC:
|
||||||
|
|
||||||
|
* Override the systemd socket unit
|
||||||
|
* Set socket group to `docker-admin`
|
||||||
|
|
||||||
|
This allows Docker administration without:
|
||||||
|
|
||||||
|
* root shells
|
||||||
|
* sudo for every command
|
||||||
|
* use of the default `docker` group
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Verification (Docker Access)
|
||||||
|
|
||||||
|
As a non-root admin user:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker version
|
||||||
|
docker info
|
||||||
|
```
|
||||||
|
|
||||||
|
Success criteria:
|
||||||
|
|
||||||
|
* No `permission denied` errors
|
||||||
|
* Docker daemon responds
|
||||||
|
* Commands run without sudo
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Node Exporter Deployment
|
||||||
|
|
||||||
|
Node Exporter is used to expose host-level metrics.
|
||||||
|
|
||||||
|
### Deployment Model
|
||||||
|
|
||||||
|
* Runs directly on the Proxmox host
|
||||||
|
* Deployed via Docker Compose
|
||||||
|
* Uses host networking
|
||||||
|
* Read-only filesystem access
|
||||||
|
|
||||||
|
### Service Directory
|
||||||
|
|
||||||
|
```
|
||||||
|
/tank/docker/node-exporter
|
||||||
|
```
|
||||||
|
|
||||||
|
### Compose Characteristics
|
||||||
|
|
||||||
|
* `network_mode: host`
|
||||||
|
* `pid: host`
|
||||||
|
* Read-only root filesystem
|
||||||
|
* Bind mounts for `/proc`, `/sys`, and `/`
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:9100/metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
A successful response returns a large metrics output.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operational Notes
|
||||||
|
|
||||||
|
* No monitoring services (Prometheus, Grafana) should run on the Proxmox host
|
||||||
|
* Exporters only — no stateful services
|
||||||
|
* Docker services should remain minimal and infrastructure-focused
|
||||||
|
* ZFS allows future snapshotting and rollback if required
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
After completing this runbook, the Proxmox host will have:
|
||||||
|
|
||||||
|
* Clean separation between OS, VM storage, and Docker workloads
|
||||||
|
* ZFS-backed Docker storage
|
||||||
|
* Role-based admin access
|
||||||
|
* Non-root Docker administration
|
||||||
|
* A working Node Exporter endpoint ready for Prometheus
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Future Extensions
|
||||||
|
|
||||||
|
* Add Prometheus scrape configuration
|
||||||
|
* Add firewall rules for exporter ports
|
||||||
|
* Add additional exporters (SMART, Proxmox)
|
||||||
|
* Automate via Ansible or Terraform
|
||||||
|
* Convert into a standard “Host Baseline” template
|
||||||
Reference in New Issue
Block a user