Personal Data Platform
A small self-hosted warehouse that collects data from several parts of ordinary life into one database: environmental readings from indoor sensors and an outdoor PurpleAir node, health and activity data from Fitbit and Strava, and hardware telemetry from the machines running the system. A TimescaleDB cluster on a NAS holds the data; Grafana and the API pollers run on a Raspberry Pi alongside it.
The motivation was practical rather than analytical. Consumer services each keep their own slice of the record behind their own app and their own export button. Putting all of it in a database under my control means the whole record is queryable with ordinary SQL — joins, window functions, and time-series functions — instead of being read one dashboard at a time. The Fitbit archive now reaches back roughly ten years, which makes the long horizon more interesting than the recent one.
Five Grafana dashboards are provisioned from the repository — an overview, a trends view, the scoreboard above, a morning report, and a running view backed by Strava. A 30-day window is useful for immediate feedback; the value of the backfill is being able to widen it to years.
Design notes
- Deliberately boring ingestion — historical exports do the backfill; small pollers keep the database current. Both paths are idempotent upserts, so re-running one is always safe.
- A dead-man switch — the pollers check in with a heartbeat service on every run, each with a grace period matched to how often it runs: between 30 minutes and 18 hours. Miss the window and an alert goes out, so a poller that has stopped silently surfaces within a day rather than as a gap discovered months later.
- Least-privilege roles — separate owner, read-write, and read-only roles, with a schema per project so unrelated domains can share one cluster without reaching into each other.
- Extensions doing the heavy lifting — TimescaleDB provides hypertables and continuous aggregates; the TimescaleDB Toolkit adds statistical hyperfunctions; PostGIS stores and queries the GPS tracks from Strava.
- Built for the API migration — the health poller targets the Google Health API rather than the legacy Fitbit Web API, which sunsets in September 2026.
- Nightly offsite backups — the cluster is dumped and copied off the NAS on a nightly schedule.
Architecture
Sources fan into a column of small ingest services on the Raspberry Pi, each writing into the shared TimescaleDB cluster on the NAS. Grafana reads back from the same cluster; the pollers report to a heartbeat monitor, and the database is backed up offsite nightly.
Source
The health and activity half of the platform is public: database schema, historical backfill tools, API pollers, Docker configuration, and the provisioned Grafana dashboards. The environmental half lives in its own repository, linked below. The system is in active development and its design is still settling.