server-manager-bot.

active

Self-hosted Discord control plane bot for host telemetry, Docker operations, alerts, and webhook-driven automation.

$ cat github.com/arnonsang/server-manager-bot/README.mdsynced 2026-11-19 10:33 ICT · rebuilt daily

Server Manager avatar

Discord Bot - Server Manager

Self-hosted Discord control plane for host telemetry, Docker operations, alerts, and webhook-driven automation.

Python 3.12+ discord.py FastAPI Docker 35 passed tests

Overview

  • Real-time hardware monitoring (CPU, RAM, disk, network, GPU, storage SMART)
  • Full Docker container operations from Discord (list, start, stop, restart, stats, images, inspect, pull, prune, exec, logs)
  • Controlled host actions (reboot, shutdown, uptime, snapshots, package updates)
  • Port visibility — list all listening ports or find what owns a specific port
  • Background alerting with disk threshold, snooze, quiet window, and escalation controls
  • Audit log of all command interactions with /audit log
  • MOTD (Message of the Day) that posts on every bot startup
  • A FastAPI webhook receiver to forward local service alerts to Discord
  • Webhook ingress with per-service API keys and per-service rate limits
  • Persistent SQLite-backed state for dashboard recovery, snapshots, audit history, and webhook auth

The bot is designed for self-hosted environments where operational control from Discord is more convenient than direct SSH access. This project was developed primarily for internal use and is shared as-is for reference and practical reuse. It has not been packaged or supported as a general-purpose product, so if you choose to use or adapt it in your own environment, you should review the code, validate the behavior against your requirements, and operate it at your own discretion. Contributions, improvements, and fixes are welcome from anyone interested in helping make it better.

Table of Contents

Features

Monitoring

  • Host status snapshot: CPU, RAM, uptime, health
  • Detailed GPU readout: temperature, utilization, VRAM usage, power draw, ECC status, running processes
  • Disk usage per mount with ASCII progress bars and inode status
  • SMART health check for physical drives
  • Network speed and transfer totals
  • Live dashboard message that auto-refreshes on an interval
  • Host port map — all listening TCP/UDP ports with owning process names
  • Server uptime history with reboot journal context

Control

  • Full Docker lifecycle: list, start, stop, restart, stats, images, inspect, pull, prune, exec, logs
  • Tunnel helper command for cloudflared status/restart
  • Check and preview host package updates
  • Reboot and shutdown host through a confirmation flow
  • Save and compare server metric snapshots by label
  • Run commands inside containers via modal input

Alerting and Integrations

  • Background polling loop for thermal, memory, disk threshold, and GPU availability checks
  • Configurable disk alert threshold, snooze window, quiet hours, and escalation interval
  • Alert cooldown to prevent notification spam
  • Audit log of all slash command interactions
  • MOTD (Message of the Day) displayed on every bot startup
  • HTTP webhook endpoint to ingest alerts from local services with manual /webhook send modal

Architecture

  • Language: Python 3.11+
  • Runtime: discord.py + FastAPI + Uvicorn
  • Metrics: psutil + nvidia-smi wrapper
  • Container management: Docker SDK for Python
  • Project layout: src layout under src/bot

High-level flow:

Discord User
  |
  v
Slash Commands ---> Bot Cogs -----------------> Host Metrics / Docker / Process / Port Controls
           |
           +-----------------------> Alert Loop ---------------------> Discord Alert Channel
           |
External Services --> Webhook API -----------> Webhook Formatter -------------> Discord Alert Channel

Prerequisites

Host requirements:

  • Ubuntu server
  • Docker Engine + Docker Compose plugin
  • NVIDIA drivers and NVIDIA Container Toolkit (for GPU access)
  • lm-sensors installed and configured (for CPU temperature availability)
  • Discord bot application and bot token

Developer requirements (local run):

  • Python 3.11+
  • uv

Environment Setup

  1. Copy environment template:
cp .env.example .env
  1. Fill required values in .env:
DISCORD_BOT_TOKEN=your_discord_bot_token
OWNER_IDS=123456789,987654321
ALERT_CHANNEL_ID=123456789
ALERT_PING=
SQLITE_DB_PATH=/data/server-manager.db
DASHBOARD_REFRESH_SECONDS=60
ALERT_COOLDOWN_SECONDS=300
DISK_ALERT_THRESHOLD=90
ALERT_DIGEST_HOUR=-1
ALERT_ESCALATION_SECONDS=0
WEBHOOK_PORT=8000
WEBHOOK_RATE_LIMIT_PER_MINUTE=60

Environment Variables

Variable Description
DISCORD_BOT_TOKEN Discord bot token from the developer portal.
OWNER_IDS Comma-separated Discord user IDs allowed to run protected commands.
ALERT_CHANNEL_ID Target channel ID for alerts and webhook forwards.
ALERT_PING Critical alert mention value: @here, @everyone, or empty for no ping.
SQLITE_DB_PATH SQLite database path for persistent state.
DASHBOARD_REFRESH_SECONDS Refresh interval for the live dashboard.
ALERT_COOLDOWN_SECONDS Cooldown window for repeated alert types.
DISK_ALERT_THRESHOLD Disk usage percentage that triggers a disk alert (default: 90).
ALERT_DIGEST_HOUR Hour (0–23) to send a daily alert digest. Set to -1 to disable.
ALERT_ESCALATION_SECONDS Re-alert interval in seconds if a condition persists. Set to 0 to disable.
WEBHOOK_PORT FastAPI webhook listener port.
WEBHOOK_RATE_LIMIT_PER_MINUTE Maximum accepted webhook events per service per rolling minute.

Installation and Local Run

  1. Recommended: install uv first using the official guide: Astral uv installation

  2. Install dependencies:

uv sync
  1. Run the bot and API server:
uv run python -m bot.main
  1. Run tests:
uv run pytest -q

Docker Deployment

  1. Ensure .env is filled.
  2. Optional: set IMAGE_TAG in .env (default is latest).
  3. Pull image:
docker compose pull
  1. Start:
docker compose up -d
  1. Check logs:
docker compose logs -f
  1. Stop services:
docker compose down

Deployment Notes

  • Compose pulls ghcr.io/arnonsang/server-manager-bot:${IMAGE_TAG:-latest}.
  • Container uses host PID namespace and elevated privileges to access host process and reboot controls.
  • Docker socket is mounted read-only for container discovery and management operations.
  • D-Bus system socket is mounted to support reboot command path.
  • GPU utility access requires NVIDIA runtime/toolkit support.

Discord Setup

In Discord Developer Portal:

  1. Create application and bot.
  2. Enable required bot scopes and permissions.
  3. Invite bot to server.
  4. Ensure slash commands are registered on startup.

Recommended permission set:

  • Send Messages
  • Embed Links
  • Use Application Commands
  • Read Message History

Command Guide

All commands are slash commands. Commands marked Owner-only are restricted to user IDs in OWNER_IDS.

Monitoring

Command Parameters Purpose Access
/dashboard — Create a live status embed that auto-refreshes on an interval. All users
/status — Quick host health summary: CPU, RAM, uptime, health label. All users
/network — Current bandwidth and transfer totals (RX/TX). All users
/ping — Gateway heartbeat latency check. All users
/ping-target <host> [count] ICMP probe from the host network namespace to any IP or domain. Owner-only

GPU

Command Parameters Purpose Access
/gpu status — Per-GPU temperature, utilization, VRAM, and power draw. Includes refresh button. All users
/gpu processes — Processes currently using GPU memory. Owner-only
/gpu ecc — ECC memory error counts per GPU. Owner-only

Storage

Command Parameters Purpose Access
/storage info — Per-mount disk usage with ASCII bars. All users
/storage largest [path] [count] Top N largest files or directories under a path. Owner-only
/storage inodes — Inode usage per mount. Owner-only
/storage smart <device> SMART health check for a physical drive (e.g. sda). Owner-only

Docker

Command Parameters Purpose Access
/docker list — All containers with status, image, and port map. Owner-only
/docker start <container> Start a container. Owner-only
/docker stop <container> Stop a container. Owner-only
/docker restart <container> Restart a container. Owner-only
/docker stats — Live CPU, memory, network I/O, and block I/O per container. Owner-only
/docker images — Locally cached Docker images with size and age. Owner-only
/docker inspect <container> Container env, ports, mounts, and health state. Owner-only
/docker pull <image> Pull an image with confirmation. Owner-only
/docker prune — Remove stopped containers and dangling images with confirmation. Owner-only
/docker exec <container> Run a command inside a container via modal input. Owner-only
/docker logs <container> [tail] [since_minutes] [follow] Stream recent container logs with optional auto-refresh. Owner-only
/tunnel <action> Manage cloudflared tunnel container (status or restart). Owner-only

Server

Command Parameters Purpose Access
/server updates — Preview pending host package upgrades. Owner-only
/server uptime — Host uptime, last reboot history, and previous boot journal tail. Owner-only
/server snapshot <label> [notes] Save a named metrics snapshot (CPU, RAM, disk). Owner-only
/server compare <label> Compare current metrics against a saved snapshot. Owner-only
/server reboot — Reboot host OS with confirm/cancel buttons. Owner-only
/server shutdown — Shut down host OS with confirm/cancel buttons. Owner-only

Ports

Command Parameters Purpose Access
/ports list [proto] All listening ports on the host (tcp, udp, or all). Owner-only
/ports find <port> Find which process owns a specific port number. Owner-only

Alerts

Command Parameters Purpose Access
/alert config — Show current alert thresholds and settings. Owner-only
/alert snooze <minutes> Suppress all alerts for N minutes. Owner-only
/alert quiet <start_hour> <end_hour> Set a quiet window (no alerts between two hours). Owner-only

Audit

Command Parameters Purpose Access
/audit log [limit] Show recent slash command audit entries. Owner-only

MOTD

Command Parameters Purpose Access
/motd set <message> Set the Message of the Day (shown on every bot startup). Owner-only
/motd clear — Clear the current MOTD. Owner-only

Webhook

Command Parameters Purpose Access
/webhook key-generate <service> Create or rotate the API key for a service. Owner-only
/webhook key-delete <service> Delete the API key for a service. Owner-only
/webhook key-list — List all active service keys with masked previews. Owner-only
/webhook key-status — Show whether auth is active and key count. Owner-only
/webhook send — Manually send a webhook alert to the alert channel via modal. Owner-only

Help

Command Parameters Purpose Access
/help — Display full command summary. All users
/help <command> Detailed help for a specific command. All users

Webhook API

Endpoint:

  • Method: POST
  • Path: /webhook
  • Port: WEBHOOK_PORT (default 8000)

Authentication behavior:

  • If no active service keys exist, requests are accepted without auth.
  • If one or more active service keys exist, include one of:
    • X-Webhook-Key: <key>
    • X-API-Key: <key>
    • Authorization: Bearer <key>
  • Incoming key is mapped to a service name and that mapped name is shown in the Discord embed source.
  • Requests are rate-limited per service key (WEBHOOK_RATE_LIMIT_PER_MINUTE). Exceeding limit returns HTTP 429.

API key lifecycle commands:

  • /webhook key-generate <service>: rotate/create key for one service and return one-time plaintext key
  • /webhook key-delete <service>: delete active key for one service
  • /webhook key-list: list all active service keys with masked previews
  • /webhook key-status: show whether auth is active and active key count

Schema:

Required fields:

  • level (info | warning | critical)

Conditional field:

  • service (string): required when auth is open (no active keys). When key auth is active, mapped service from API key is used.

Content fields (at least one required):

  • message (string)
  • raw_text (string)
  • raw_json (object/array/value)
  • logs (string)

Optional fields:

  • title (string): custom embed title
  • source (string): source label shown in signal block
  • ping (@here | @everyone | empty): mention prefix for this event
  • tags (array[string]): tags shown as a comma-separated field
  • details (object[string,string]): key/value diagnostics block
  • attachment_name (string): output filename shown in Discord
  • attachment_base64 (string): base64-encoded file payload (max 8MB)

Request body:

{
  "service": "ServiceName",
  "level": "info",
  "message": "The alert message",
  "title": "Optional custom title",
  "source": "Optional source label",
  "ping": "",
  "tags": ["host-a", "scheduler"],
  "details": {
    "queue_depth": "17",
    "retry_count": "2"
  },
  "raw_text": "optional raw text block",
  "raw_json": {"event": "heartbeat", "ok": true},
  "logs": "optional multiline logs",
  "attachment_name": "service.log",
  "attachment_base64": "<base64 bytes>"
}

Allowed level values:

  • info
  • warning
  • critical

Example:

curl -X POST http://localhost:8000/webhook \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Key: YOUR_API_KEY" \
  -d '{"service":"superchat","message":"disk above threshold","level":"warning","tags":["disk","ops"],"details":{"path":"/var","usage":"91%"}}'

Raw JSON example:

curl -X POST http://localhost:8000/webhook \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Key: YOUR_API_KEY" \
  -d '{"service":"build-agent","level":"info","raw_json":{"job":"nightly","status":"ok","duration_s":422}}'

Attachment example (base64):

LOG_B64=$(base64 -w 0 ./latest.log)
curl -X POST http://localhost:8000/webhook \
  -H "Content-Type: application/json" \
  -H "X-Webhook-Key: YOUR_API_KEY" \
  -d "{\"service\":\"collector\",\"level\":\"warning\",\"raw_text\":\"attached runtime log\",\"attachment_name\":\"latest.log\",\"attachment_base64\":\"${LOG_B64}\"}"

Persistence notes:

  • SQLite stores dashboard state so /dashboard survives process/container restarts.
  • SQLite stores webhook API key hash, service mapping, masked key preview, and creation timestamp.
  • Default path is /data/server-manager.db; compose maps ./data to /data.

Security Model

  • Protected commands do not rely on Discord roles.
  • Authorization is based on user ID membership in OWNER_IDS.
  • Reboot requires explicit confirmation interaction.
  • Alert and command errors are returned as ephemeral responses where appropriate.

Operational warning:

  • This bot has high host-level capabilities by design. Restrict access carefully and keep OWNER_IDS minimal.

Production Checklist

Use this checklist before treating the bot as production-ready.

Access and Secrets

  • Keep OWNER_IDS limited to trusted admins only.
  • Store .env securely and never commit it.
  • Rotate DISCORD_BOT_TOKEN on any suspected leak.
  • Review bot invite permissions and remove unnecessary scopes.

Reliability and Restart Behavior

  • Set restart: unless-stopped in Compose (already configured).
  • Verify bot process auto-recovers after host reboot.
  • Confirm dashboard recreation workflow after bot restarts.
  • Validate alert loop resumes and sends messages after reconnect.

Monitoring and Logs

  • Aggregate docker compose logs into your existing logging stack if available.
  • Set up host-level disk and memory monitoring to avoid silent failures.
  • Watch for repeated command failures or alert spam patterns.
  • Periodically verify webhook health with a test POST.

Backup and Recovery

  • Back up deployment files: docker-compose.yml, .env, and any local overrides.
  • Document a quick redeploy path: pull code, restore .env, run docker compose pull && docker compose up -d.
  • Keep a rollback plan to the previous image or commit if a release regresses.

Upgrade Workflow

  • Update dependencies with uv lock --upgrade in a controlled branch.
  • Run tests before deployment: uv run pytest -q.
  • Deploy during a maintenance window if reboot features are actively used.
  • After deployment, verify /ping, /status, /gpu, /docker list, and webhook delivery.

Troubleshooting

Imports marked unresolved in editor

  • Ensure VS Code uses .venv interpreter.
  • Ensure dependencies are installed with uv sync.
  • Reload VS Code window if diagnostics are stale.

Bot starts but slash commands do not appear

  • Confirm bot has application command scope in invite URL.
  • Confirm startup logs indicate command tree sync succeeded.

GPU metrics unavailable

  • Verify nvidia-smi works on host.
  • Verify NVIDIA Container Toolkit is configured.
  • Verify compose deployment includes GPU utility capability.

Docker operations fail

  • Verify /var/run/docker.sock is mounted.
  • Verify bot container can access Docker daemon.

Reboot command fails

  • Verify privileged mode and host PID namespace are enabled.
  • Verify D-Bus system socket mount exists and is accessible.

Testing

Run unit tests:

uv run pytest -q

Current test coverage includes config parsing, owner checks, formatting helpers, and webhook behavior.


Made with ❤️ by arnonsang

continue reading on github →