Sandbar/README.md

Sandbar

Self-hosted notifications, monitoring, on-call and incident response in one small program. One binary, one SQLite file, and everything is done in the web UI.

Sandbar covers the ground of five tools:

Area	Like	What it does
Topics	ntfy	Publish a message to a topic with one HTTP request. Subscribe in the browser, with curl, or with the ntfy phone app
Monitors	Tianji, Uptime Kuma	Website, port, DNS and certificate checks, heartbeats for cron jobs, and a status page with groups, incident posts, maintenance and subscribers
Websites	Tianji, Plausible, Umami	Page views, visitors, referrers, browsers, countries and custom events for your own sites, with no cookies and no personal data
On-call	Opsgenie	Rotating schedules, escalation policies, and alerts that are acknowledged, snoozed or resolved
Incidents	incident.io	Declare, assign, update and close an incident, with a timeline and a post-mortem

It is built in the same style as Hammerhead, Blacktip and Bamboo and shares Reef with them (dotenv, netguard, secretbox).

Quick start

go build -o bin/ ./cmd/...

cp .env.example .env
bin/sandbar -genkey               # paste the output into .env as SANDBAR_MASTER_KEY=

bin/sandbar -listen :8090 -url https://sandbar.example.com

Open http://localhost:8090 and create the first administrator. Or set SANDBAR_ADMIN_EMAIL and SANDBAR_ADMIN_PASSWORD in .env to create it at startup.

Deploying

Docker

Pushing a version tag publishes a multi-architecture (amd64 and arm64) image to the registry on the same server as the repository, as git.veillax.com/veillax/sandbar:<version> and :latest. Generate a master key once with docker run --rm git.veillax.com/veillax/sandbar -genkey, then keep it in a .env file next to this compose file:

services:
  sandbar:
    image: git.veillax.com/veillax/sandbar:latest
    restart: unless-stopped
    ports:
      - "127.0.0.1:8090:8090"
    env_file: .env
    environment:
      SANDBAR_URL: https://sandbar.example.com
      SANDBAR_AUTO_UPDATE: "1"   # optional, see Updating below
    volumes:
      - sandbar-data:/data

volumes:
  sandbar-data:

docker compose up -d

• The container runs as user 65532 and keeps its database in /data. Use a named volume as above. If you bind-mount a folder instead, make it writable by 65532 (chown 65532:65532 ./data).
• Set SANDBAR_MASTER_KEY yourself. Without it a key is generated inside /data, next to the database it protects.
• To build the image locally, see the comment at the top of Dockerfile.
• Update by hand with docker compose pull && docker compose up -d, or let Sandbar do it (see Updating below). Your data stays in the volume. Copy the data folder before upgrading, since there is no migration tooling yet.

Binary

Download an archive from the release page (Linux amd64 and arm64, Windows amd64) or build it (go build -o bin/ ./cmd/sandbar), copy it to the server with a .env file, and run it under a service manager. A minimal systemd unit:

[Unit]
Description=Sandbar
After=network-online.target

[Service]
User=sandbar
WorkingDirectory=/var/lib/sandbar
ExecStart=/usr/local/bin/sandbar -listen 127.0.0.1:8090 -env /etc/sandbar/.env
Restart=on-failure

[Install]
WantedBy=multi-user.target

Put SANDBAR_DATA=/var/lib/sandbar in the env file and make /etc/sandbar/.env readable only by the sandbar user.

Updating

Pushing a version tag publishes a release with archives and SHA256SUMS as well as the image. Sandbar can install new releases itself, in Docker and as a plain binary, on Linux and macOS.

• Automatic: set SANDBAR_AUTO_UPDATE=1 (or pass -auto-update). Sandbar checks a few minutes after start and then every few hours, downloads the archive for its platform, checks it against SHA256SUMS, checks that the new binary runs, replaces itself and restarts. Pre-releases (v1.0.0-rc1) are never installed.
• On demand: an administrator can see the newest release and click Update on the Account page, with or without automatic updates on.
• In Docker: the image is read-only to Sandbar, and a change inside the container would be lost when it is recreated. So the new binary is written to <data>/bin/sandbar on the data volume. At every start Sandbar runs that file when it is a newer release than the image, and deletes it once the image has caught up. The container keeps running through the update (it re-executes in place, so the process id does not change), and docker compose pull still works: a newer image takes over from the volume copy.
• Pin a version: leave automatic updates off and use an image tag such as :v0.2.0 instead of :latest.
• Requirements: the release page must be readable without signing in, so the repository has to be public. A development build (dev) and Windows never update themselves. SANDBAR_UPDATE_SERVER and SANDBAR_UPDATE_REPO point it at a different Gitea server or fork. If a bad update leaves Sandbar unable to start, delete <data>/bin/sandbar (or set SANDBAR_NO_OVERLAY=1) and it starts from the image again.

HTTPS and a reverse proxy

Sandbar serves plain HTTP, so put a reverse proxy in front of it and set SANDBAR_URL to the public address, since links in notifications use it. Also set SANDBAR_TRUST_PROXY=1, so website analytics and the sign-in limit see the real client address instead of the proxy's. Caddy needs one line:

sandbar.example.com {
    reverse_proxy 127.0.0.1:8090
}

With nginx, turn off response buffering for the live topic feeds (/<topic>/sse and /<topic>/json), or subscribers see messages late: proxy_buffering off; proxy_read_timeout 1h;.

After the first start

1. Open the address and create the administrator, or set SANDBAR_ADMIN_EMAIL and SANDBAR_ADMIN_PASSWORD first.
2. Add an email or webhook channel and use its test button, so you know alerts will reach you.
3. Create an API key on the Account page for scripts and monitoring agents.
4. Set up backups (see Database below), including the master key, which must be stored apart from the data.

How the pieces fit

monitor fails ─┐
API request ───┼─> alert ─> escalation policy ─> person ─> topic + email
Alertmanager ──┘        │                                 (repeat until acknowledged)
                        └─> broadcast channels (Slack, Discord, ntfy, email)
                        └─> optionally an incident, with a timeline

Every alert is deduplicated by its key, so a monitor that stays down is one alert, not one per check.

Topics

# publish (ntfy-style headers)
curl -H "Authorization: Bearer sb_..." -H "Title: Disk full" -H "Priority: urgent" -H "Tags: warning,disk" \
     -d "web-1 is at 98%" https://sandbar.example.com/ops

# subscribe: server-sent events, or JSON lines; ?since=all replays stored messages
curl -N -H "Authorization: Bearer sb_..." https://sandbar.example.com/ops/sse
curl -N -H "Authorization: Bearer sb_..." "https://sandbar.example.com/ops/json?since=all"

A topic is 1 to 64 letters, digits, - or _. Publishing and subscribing both need a signed-in session or an API key (create one on the Account page). Clients that use basic auth can send the key as the password. Messages are kept for SANDBAR_MESSAGE_DAYS (default 30).

?since= also takes unix seconds, a duration such as 10m, or the id of the last message you have, and ?poll=1 returns just that replay and closes, which is how the ntfy apps catch up.

Phone app (ntfy)

1. Create an API key on the Account page and copy it.
2. In the ntfy app, open Settings, then Default server, and enter your Sandbar address (https://sandbar.example.com). Or tick "Use another server" when you add each subscription.
3. Add a subscription. For a page when an escalation reaches you, use your personal topic from the Account page (u-<id>). For everything, use alerts and incidents.
4. Under Manage users, add a user for the server. The username can be anything and the password is the API key.

The Android app keeps a connection open, so messages arrive at once. The iPhone app depends on ntfy.sh's push relay, which Sandbar does not use, so there messages arrive only when the app polls.

Two topics are filled for you: alerts gets every alert and its recovery, and incidents gets every incident update. Each person also has a personal topic (u-<id>, shown on the Account page). Subscribe to it in the ntfy app to be paged on your phone when an escalation policy reaches you.

Monitors

Type	Passes when	Notes
Website (HTTP)	The status is in the expected range (default 2xx) and the page contains the text you set	Redirects are not followed
Port (TCP)	A connection opens
DNS lookup	The name resolves
Certificate expiry	The certificate has more than N days left (default 14)
Heartbeat	Your job requests /api/v1/hb/<token> within its interval plus the allowed delay	A request on a down heartbeat recovers it at once

A monitor goes down after the set number of failures in a row (default 2), raises an alert with the monitor's escalation policy, and recovers on the first success. Monitors marked public appear on the status page (see below), together with incidents marked public. The status page never shows staff names or private monitors.

The status page shows each public monitor as 90 daily uptime bars, like most hosted status pages. A day is green with no failed checks, amber when at least 99% of checks passed, red below that, and grey with no data. Hovering a bar (or tapping it on a phone) gives the date, the uptime and the number of failed checks. Days are UTC. Counts are kept in a small per-day table, so the bars go back 90 days even though raw check results are kept for SANDBAR_CHECK_DAYS only.

Monitors cannot probe private or internal addresses unless you tick Allow targets on private networks on that monitor, so a user who can add a monitor cannot use Sandbar to scan your network.

Status page

The public page is at /status (and at the root of its own domain, see below). Everything is set up under Status page in the sidebar.

• Components and groups. Every monitor with Show on the public status page ticked is a component, shown as 90 daily uptime bars (see Monitors). Create groups on the Groups tab, then pick a group, and optionally a different public name, in the monitor's form. Groups are ordered by their order number; components without a group come first.
• Incidents. An incident marked public is posted with its status, the components it affects, and the updates you write. Declare it as public, or switch it on later. Post an update (or changing the status) adds a post with your text, or standard wording if you leave it empty. Customers see only the posts. The internal timeline, comments, names and post-mortem never leave Sandbar. The page shows the last 14 days; everything older is on /status/history, and each incident has its own page with the full list of updates.
• What the page says. A component is Major outage when its monitor is down or a sev1 incident names it, Partial outage for sev2, Degraded performance for sev3, and Under maintenance during a maintenance window for it. The banner shows the worst of them.
• Scheduled maintenance. Give a window (your local time in the form, stored as UTC) and the affected components. It is announced when scheduled, listed on the page until it ends, and subscribers are told when it starts and when it ends. During the window its components show as under maintenance instead of as an outage. Windows can be moved or canceled, and subscribers are told. Alerts for failing checks still fire as usual.
• Branding. Title, description, logo (an https address), accent color, a contact link and footer text. The accent is checked so that white text on it stays readable (WCAG 4.5 to 1).
• Custom domain. Enter a host name such as status.example.com, point its DNS at your server and have the reverse proxy forward that host to Sandbar with the Host header unchanged (Caddy does this by default; with nginx add proxy_set_header Host $host;). Sandbar then serves the status page at / on that host, and nothing else: the app, API and topics are not reachable through it. Links in emails and feeds use https://<domain>. The proxy provides the certificate. With SANDBAR_TRUST_PROXY=1 the host is read from X-Forwarded-Host when present.

Following the page

Visitors choose Subscribe to updates on the page:

• Email. Needs SANDBAR_SMTP_HOST; the form is hidden without it. A visitor gets a confirmation link, and no mail is sent until it is opened. Every message has an unsubscribe link, which asks for one click on the page, so mail scanners cannot unsubscribe people. Unconfirmed addresses are forgotten after 7 days.
• Webhook. The address must answer a test request with a 2xx status before it is subscribed. Requests to private networks are refused. Each update is a JSON POST: {"page","event","title","status","message","url","time","unsubscribe"}. The events are incident.created, incident.updated, incident.resolved, maintenance.scheduled, maintenance.rescheduled, maintenance.started, maintenance.completed and maintenance.canceled. The unsubscribe link works with a POST of {"token": ...} to /status/unsubscribe, or can be opened in a browser.
• RSS. /status/feed.rss has every incident update and maintenance announcement, newest first.

Subscribers can be seen and removed on the Subscribers tab (webhook addresses show only the host). Addresses are stored encrypted. Subscribing is limited to 6 requests per hour per address.

Status page API

Request
GET /status.json	Settings, groups, components with their 90 days, current incidents, upcoming and running maintenance
GET /status/api/history?before=<time>&limit=<n>	Past incidents and maintenance, newest first, with the time to pass as before for the next page
GET /status/api/incidents/<id>	One public incident with all its updates
GET/PUT /api/v1/statuspage	Settings (PUT is for administrators)
/api/v1/status-groups	Groups, as the other configuration lists
/api/v1/maintenance, POST .../<id>/cancel	Scheduled maintenance (writes are for administrators)
PATCH /api/v1/incidents/<id>	Also takes components (monitor ids) and note (the text of the public post)

Websites (analytics)

Add a website on the Websites page, then paste its one-line tracking code into the head of your pages:

<script defer src="https://sandbar.example.com/script.js" data-website="WEBSITE_ID"></script>

The page for each website shows visitors, page views, visits, bounce rate and who is online now, a chart over the last 24 hours, 7, 30 or 90 days, and top lists for pages, referrers, browsers, operating systems, devices, countries, languages and events.

• Single-page apps are counted as people move between pages. Add data-hash to count #hash changes as pages, or data-auto="false" to send nothing until your code calls sandbar.view().
• Events: call sandbar.track("signup", { plan: "pro" }) from your code.
• Private by design: no cookies, nothing stored in the browser, no IP address or user agent kept (only the labels "Chrome", "macOS", "Mobile" and so on), and nothing is sent when the visitor has Do Not Track on. A visitor is a hash of the address, the browser and the website with a secret and the date, so the same person is one visitor for a day and cannot be followed beyond it. Query strings are dropped from paths and only the referrer's host is kept.
• Domain: a website's optional domain makes Sandbar ignore visits from other hosts. Bots, and anything sending more than 120 hits a minute from one address, are ignored.
• Countries come from the CF-IPCountry or CloudFront-Viewer-Country header, so they appear only behind Cloudflare or CloudFront. Sandbar bundles no location database. Without the header, visits count as Unknown.
• Behind a reverse proxy, set SANDBAR_TRUST_PROXY=1 so the client address comes from X-Forwarded-For. Without it every visitor looks like the proxy. Only set it when the proxy is the only way to reach Sandbar.
• Hits are kept for SANDBAR_ANALYTICS_DAYS (default 365).

Not included: funnels, goals, session replay, a public share link, and comparison with the previous period.

On-call

• A schedule rotates through people, handing over every N hours from a start time. Overrides cover a holiday or swap a shift.
• An escalation policy is a list of steps. Each step waits some minutes, then tells people and/or whoever is on call for a schedule. It can start over after the last step until someone acknowledges.
• Alerts are stored with the time their next step is due, so restarting Sandbar loses nothing and repeats nothing.
• Acknowledge stops escalation. Snooze pauses it and resumes at the same step. Resolve closes the alert.

Incidents

Declare one from the UI or POST /api/v1/incidents, or from an open alert (which links them). An incident has a severity (sev1 to sev3), a status (investigating, identified, monitoring, resolved), a lead and a communications owner, a timeline of everything that happened, comments, and a post-mortem with a template filled from the timeline. Every change is announced on the incidents topic and on all channels.

Sending alerts in

# generic
curl -H "Authorization: Bearer sb_..." -H "Content-Type: application/json" \
     -d '{"key":"cpu-web-1","summary":"CPU above 90%","severity":"warning","policy":"<policy id>"}' \
     https://sandbar.example.com/api/v1/alerts

# resolve it later with the same key
curl ... -d '{"key":"cpu-web-1","status":"resolved"}' https://sandbar.example.com/api/v1/alerts

Prometheus Alertmanager and Grafana can post their webhook to /api/v1/integrations/alertmanager?policy=<policy id> with an API key as the bearer token.

Channels

A channel receives every alert and incident update: an email address list, or a webhook in Slack, Discord, ntfy or plain JSON format. The address is stored encrypted and is never sent back to the browser. Webhooks cannot point at internal addresses.

Configuration and secrets

.env.example lists every setting. Secrets at rest use AES-256-GCM with SANDBAR_MASTER_KEY; passwords are bcrypt hashes and API keys and sessions are stored as SHA-256 hashes. With the wrong key, saved channel addresses cannot be read. If you do not set a key, one is generated into <data>/master.key; move it into .env, and back it up separately.

Prometheus metrics are served at /metrics when SANDBAR_METRICS_TOKEN is set.

Database

SQLite (pure Go, no C compiler), in WAL mode, in <data>/sandbar.db. Messages, monitor results and alerts older than their retention are pruned hourly. Back up by copying the data folder while Sandbar is stopped, plus the key.

Limits and notes

• One instance, one organization. SQLite means no clustering.
• No ICMP ping monitor, no SMS or voice, and no two-factor or single sign-on yet.
• Sandbar serves plain HTTP. Put it behind HTTPS and set -url, or keep it on a private network.
• Sign-in attempts are limited to 10 per 10 minutes per address (the client address, see SANDBAR_TRUST_PROXY).
• Tests: go test ./.... Style: node scripts/lint-design.mjs (see docs/design.md).

License

AGPL-3.0. See LICENSE.