Sandbar/README.md

Sandbar

Self-hosted notifications, monitoring, on-call and incident response in one small program. One binary, one SQLite file, and everything is done in the web UI.

Sandbar covers the ground of four tools:

Area	Like	What it does
Topics	ntfy	Publish a message to a topic with one HTTP request. Subscribe in the browser, with curl, or with the ntfy phone app
Monitors	Tianji, Uptime Kuma	Website, port, DNS and certificate checks, heartbeats for cron jobs, and a public status page
On-call	Opsgenie	Rotating schedules, escalation policies, and alerts that are acknowledged, snoozed or resolved
Incidents	incident.io	Declare, assign, update and close an incident, with a timeline and a post-mortem

It is built in the same style as Hammerhead, Blacktip and Bamboo and shares Reef with them (dotenv, netguard, secretbox).

Quick start

go build -o bin/ ./cmd/...

cp .env.example .env
bin/sandbar -genkey               # paste the output into .env as SANDBAR_MASTER_KEY=

bin/sandbar -listen :8090 -url https://sandbar.example.com

Open http://localhost:8090 and create the first administrator. Or set SANDBAR_ADMIN_EMAIL and SANDBAR_ADMIN_PASSWORD in .env to create it at startup.

Deploying

Docker

Pushing a version tag publishes a multi-architecture (amd64 and arm64) image to the registry on the same server as the repository, as git.veillax.com/veillax/sandbar:<version> and :latest. Generate a master key once with docker run --rm git.veillax.com/veillax/sandbar -genkey, then keep it in a .env file next to this compose file:

services:
  sandbar:
    image: git.veillax.com/veillax/sandbar:latest
    restart: unless-stopped
    ports:
      - "127.0.0.1:8090:8090"
    env_file: .env
    environment:
      SANDBAR_URL: https://sandbar.example.com
      SANDBAR_AUTO_UPDATE: "1"   # optional, see Updating below
    volumes:
      - sandbar-data:/data

volumes:
  sandbar-data:

docker compose up -d

• The container runs as user 65532 and keeps its database in /data. Use a named volume as above. If you bind-mount a folder instead, make it writable by 65532 (chown 65532:65532 ./data).
• Set SANDBAR_MASTER_KEY yourself. Without it a key is generated inside /data, next to the database it protects.
• To build the image locally, see the comment at the top of Dockerfile.
• Update by hand with docker compose pull && docker compose up -d, or let Sandbar do it (see Updating below). Your data stays in the volume. Copy the data folder before upgrading, since there is no migration tooling yet.

Binary

Download an archive from the release page (Linux amd64 and arm64, Windows amd64) or build it (go build -o bin/ ./cmd/sandbar), copy it to the server with a .env file, and run it under a service manager. A minimal systemd unit:

[Unit]
Description=Sandbar
After=network-online.target

[Service]
User=sandbar
WorkingDirectory=/var/lib/sandbar
ExecStart=/usr/local/bin/sandbar -listen 127.0.0.1:8090 -env /etc/sandbar/.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

Put SANDBAR_DATA=/var/lib/sandbar in the env file and make /etc/sandbar/.env readable only by the sandbar user.

Updating

Pushing a version tag publishes a release with archives and SHA256SUMS as well as the image. Sandbar can install new releases itself, in Docker and as a plain binary, on Linux and macOS.

• Automatic: set SANDBAR_AUTO_UPDATE=1 (or pass -auto-update). Sandbar checks a few minutes after start and then every few hours, downloads the archive for its platform, checks it against SHA256SUMS, checks that the new binary runs, replaces itself and restarts. Pre-releases (v1.0.0-rc1) are never installed.
• On demand: an administrator can see the newest release and click Update on the Account page, with or without automatic updates on.
• In Docker: the image is read-only to Sandbar, and a change inside the container would be lost when it is recreated. So the new binary is written to <data>/bin/sandbar on the data volume. At every start Sandbar runs that file when it is a newer release than the image, and deletes it once the image has caught up. The container keeps running through the update (it re-executes in place, so the process id does not change), and docker compose pull still works: a newer image takes over from the volume copy.
• Pin a version: leave automatic updates off and use an image tag such as :v0.2.0 instead of :latest.
• Requirements: the release page must be readable without signing in, so the repository has to be public. A development build (dev) and Windows never update themselves. SANDBAR_UPDATE_SERVER and SANDBAR_UPDATE_REPO point it at a different Gitea server or fork. If a bad update leaves Sandbar unable to start, delete <data>/bin/sandbar (or set SANDBAR_NO_OVERLAY=1) and it starts from the image again.

HTTPS and a reverse proxy

Sandbar serves plain HTTP, so put a reverse proxy in front of it and set SANDBAR_URL to the public address, since links in notifications use it. Caddy needs one line:

sandbar.example.com {
    reverse_proxy 127.0.0.1:8090
}

With nginx, turn off response buffering for the live topic feeds (/<topic>/sse and /<topic>/json), or subscribers see messages late: proxy_buffering off; proxy_read_timeout 1h;.

After the first start

1. Open the address and create the administrator, or set SANDBAR_ADMIN_EMAIL and SANDBAR_ADMIN_PASSWORD first.
2. Add an email or webhook channel and use its test button, so you know alerts will reach you.
3. Create an API key on the Account page for scripts and monitoring agents.
4. Set up backups (see Database below), including the master key, which must be stored apart from the data.

How the pieces fit

monitor fails ─┐
API request ───┼─> alert ─> escalation policy ─> person ─> topic + email
Alertmanager ──┘        │                                 (repeat until acknowledged)
                        └─> broadcast channels (Slack, Discord, ntfy, email)
                        └─> optionally an incident, with a timeline

Every alert is deduplicated by its key, so a monitor that stays down is one alert, not one per check.

Topics

# publish (ntfy-style headers)
curl -H "Authorization: Bearer sb_..." -H "Title: Disk full" -H "Priority: urgent" -H "Tags: warning,disk" \
     -d "web-1 is at 98%" https://sandbar.example.com/ops

# subscribe: server-sent events, or JSON lines; ?since=all replays stored messages
curl -N -H "Authorization: Bearer sb_..." https://sandbar.example.com/ops/sse
curl -N -H "Authorization: Bearer sb_..." "https://sandbar.example.com/ops/json?since=all"

A topic is 1 to 64 letters, digits, - or _. Publishing and subscribing both need a signed-in session or an API key (create one on the Account page). Clients that use basic auth can send the key as the password. Messages are kept for SANDBAR_MESSAGE_DAYS (default 30).

?since= also takes unix seconds, a duration such as 10m, or the id of the last message you have, and ?poll=1 returns just that replay and closes, which is how the ntfy apps catch up.

Phone app (ntfy)

1. Create an API key on the Account page and copy it.
2. In the ntfy app, open Settings, then Default server, and enter your Sandbar address (https://sandbar.example.com). Or tick "Use another server" when you add each subscription.
3. Add a subscription. For a page when an escalation reaches you, use your personal topic from the Account page (u-<id>). For everything, use alerts and incidents.
4. Under Manage users, add a user for the server. The username can be anything and the password is the API key.

The Android app keeps a connection open, so messages arrive at once. The iPhone app depends on ntfy.sh's push relay, which Sandbar does not use, so there messages arrive only when the app polls.

Two topics are filled for you: alerts gets every alert and its recovery, and incidents gets every incident update. Each person also has a personal topic (u-<id>, shown on the Account page). Subscribe to it in the ntfy app to be paged on your phone when an escalation policy reaches you.

Monitors

Type	Passes when	Notes
Website (HTTP)	The status is in the expected range (default 2xx) and the page contains the text you set	Redirects are not followed
Port (TCP)	A connection opens
DNS lookup	The name resolves
Certificate expiry	The certificate has more than N days left (default 14)
Heartbeat	Your job requests /api/v1/hb/<token> within its interval plus the allowed delay	A request on a down heartbeat recovers it at once

A monitor goes down after the set number of failures in a row (default 2), raises an alert with the monitor's escalation policy, and recovers on the first success. Monitors marked public appear on /status, together with incidents marked public. The status page never shows staff names or private monitors.

The status page shows each public monitor as 90 daily uptime bars, like most hosted status pages. A day is green with no failed checks, amber when at least 99% of checks passed, red below that, and grey with no data. Hovering a bar (or tapping it on a phone) gives the date, the uptime and the number of failed checks. Days are UTC. Counts are kept in a small per-day table, so the bars go back 90 days even though raw check results are kept for SANDBAR_CHECK_DAYS only.

Monitors cannot probe private or internal addresses unless you tick Allow targets on private networks on that monitor, so a user who can add a monitor cannot use Sandbar to scan your network.

On-call

• A schedule rotates through people, handing over every N hours from a start time. Overrides cover a holiday or swap a shift.
• An escalation policy is a list of steps. Each step waits some minutes, then tells people and/or whoever is on call for a schedule. It can start over after the last step until someone acknowledges.
• Alerts are stored with the time their next step is due, so restarting Sandbar loses nothing and repeats nothing.
• Acknowledge stops escalation. Snooze pauses it and resumes at the same step. Resolve closes the alert.

Incidents

Declare one from the UI or POST /api/v1/incidents, or from an open alert (which links them). An incident has a severity (sev1 to sev3), a status (investigating, identified, monitoring, resolved), a lead and a communications owner, a timeline of everything that happened, comments, and a post-mortem with a template filled from the timeline. Every change is announced on the incidents topic and on all channels.

Sending alerts in

# generic
curl -H "Authorization: Bearer sb_..." -H "Content-Type: application/json" \
     -d '{"key":"cpu-web-1","summary":"CPU above 90%","severity":"warning","policy":"<policy id>"}' \
     https://sandbar.example.com/api/v1/alerts

# resolve it later with the same key
curl ... -d '{"key":"cpu-web-1","status":"resolved"}' https://sandbar.example.com/api/v1/alerts

Prometheus Alertmanager and Grafana can post their webhook to /api/v1/integrations/alertmanager?policy=<policy id> with an API key as the bearer token.

Channels

A channel receives every alert and incident update: an email address list, or a webhook in Slack, Discord, ntfy or plain JSON format. The address is stored encrypted and is never sent back to the browser. Webhooks cannot point at internal addresses.

Configuration and secrets

.env.example lists every setting. Secrets at rest use AES-256-GCM with SANDBAR_MASTER_KEY; passwords are bcrypt hashes and API keys and sessions are stored as SHA-256 hashes. With the wrong key, saved channel addresses cannot be read. If you do not set a key, one is generated into <data>/master.key; move it into .env, and back it up separately.

Prometheus metrics are served at /metrics when SANDBAR_METRICS_TOKEN is set.

Database

SQLite (pure Go, no C compiler), in WAL mode, in <data>/sandbar.db. Messages, monitor results and alerts older than their retention are pruned hourly. Back up by copying the data folder while Sandbar is stopped, plus the key.

Limits and notes

• One instance, one organization. SQLite means no clustering.
• No ICMP ping monitor, no SMS or voice, and no two-factor or single sign-on yet.
• Sandbar serves plain HTTP. Put it behind HTTPS and set -url, or keep it on a private network.
• Sign-in attempts are limited to 10 per 10 minutes per address.
• Tests: go test ./.... Style: node scripts/lint-design.mjs (see docs/design.md).

License

AGPL-3.0. See LICENSE.