Commit graph

2 commits

Author SHA1 Message Date
Viktor Barzin
602103ede1 [owntracks] Strip face avatar from hook payload + drop orphan PVC
Bundles two small follow-ups to the live bridge + port-fix work:

## Face avatar fix (dawarich-hook.lua)

After the Recorder ran in production for a while it began enriching
publish payloads with a `face` field — the base64-encoded user avatar
uploaded via the Recorder's web UI (~120 KB). Our Lua hook builds a
curl command that embeds the JSON payload as `-d '<payload>'`, which
hit `E2BIG` / `Argument list too long` (os.execute reason=code=7) on
Linux's `execve` argv limit (~128 KB). Every live POST stopped making
it to Dawarich, even though the HTTP POST from the phone to Owntracks
still returned 200 and the .rec write still happened.

Fix: `data.face = nil` before serializing. Dawarich doesn't use it
anyway (not persisted into any column — `raw_data` stored without it).

Also upgraded the debug log: on failure we now emit
`dawarich-bridge: FAIL tst=... reason=... code=... cmd=...` so any
future variant of this problem (next big field surfaced upstream, etc.)
is one log tail away from a diagnosis.

```
$ kubectl -n owntracks logs deploy/owntracks --tail=5 | grep dawarich-bridge
+ dawarich-bridge: init
+ dawarich-bridge: ok tst=1776600238
```

## Orphan PVC removal (main.tf)

`owntracks-data-proxmox` (1 Gi, proxmox-lvm, unencrypted) was a leftover
from the encrypted-migration attempt; the Deployment has been mounting
`owntracks-data-encrypted` the whole time. Verified `Used By: <none>`
on the live PVC before removal. Removing the resource from Terraform
destroys the PVC — harmless, no data loss.

## Test Plan

### Automated

```
$ ../../scripts/tg plan
Plan: 0 to add, 1 to change, 1 to destroy.

$ ../../scripts/tg apply --non-interactive
Apply complete! Resources: 0 added, 1 changed, 1 destroyed.

$ kubectl -n owntracks get pvc
NAME                       STATUS   VOLUME ...
owntracks-data-encrypted   Bound    ...
(owntracks-data-proxmox gone)
```

### Manual Verification

```
$ VIKTOR_PW=$(vault kv get -field=credentials secret/owntracks | jq -r .viktor)
$ TST=$(date +%s)
$ kubectl -n owntracks run t --rm -i --image=curlimages/curl -- \
    curl -s -w 'HTTP %{http_code}\n' -X POST -u "viktor:$VIKTOR_PW" \
    -H 'Content-Type: application/json' \
    -H 'X-Limit-U: viktor' -H 'X-Limit-D: iphone-15pro' \
    -d "{\"_type\":\"location\",\"lat\":51.5074,\"lon\":-0.1278,\"tst\":$TST,\"tid\":\"vb\"}" \
    https://owntracks.viktorbarzin.me/pub
HTTP 200

$ sleep 3 && kubectl -n dbaas exec pg-cluster-1 -c postgres -- \
    psql -U postgres -d dawarich -tAc \
    "SELECT ST_AsText(lonlat::geometry) FROM points WHERE user_id=1 AND timestamp=$TST"
POINT(-0.1278 51.5074)
```

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-19 12:05:18 +00:00
Viktor Barzin
17a3e03e07 [owntracks] Bridge Recorder → Dawarich via Lua hook script
## Context

Viktor wanted live forwarding from Owntracks to Dawarich so his map
stays in sync without a periodic backfill. The original plan assumed
ot-recorder honoured an `OTR_HTTPHOOK` environment variable — but
Recorder 1.0.1 (latest on Docker Hub as of Aug 2025) has no such
feature:

```
$ kubectl -n owntracks exec deploy/owntracks -- \
    strings /usr/bin/ot-recorder | grep -iE 'hook|webhook|http_post'
(no matches)
```

Lua hooks, on the other hand, are first-class: `--lua-script` loads a
file and calls the `otr_hook(topic, _type, data)` function for every
publish. That is the pivot this commit makes.

## This change

Mount a Lua script via ConfigMap and tell ot-recorder to load it:

```
Phone POST /pub ---> Traefik ---> Recorder pod
                                     |
                                     | handle_payload() writes .rec
                                     | otr_hook(topic,_type,data)
                                     |   |
                                     |   +---> os.execute("curl … &")
                                     |             |
                                     |             v
                                     |         Dawarich /api/v1/owntracks/points
                                     |
                                     +---> HTTP 200 to phone
```

Per-publish cost: one `curl` subprocess, `--max-time 5`, backgrounded
with `&` so it doesn't block the HTTP response to the phone. A
Dawarich 5xx drops exactly one point — the `.rec` write still happens,
so the one-shot backfill Job can always re-play.

`DAWARICH_API_KEY` is injected from K8s Secret `owntracks-secrets`
(sourced from Vault `secret/owntracks.dawarich_api_key` via the
existing `dataFrom.extract` ExternalSecret). The Lua reads it with
`os.getenv()` so the key never lands in Terraform state.

### Key discoveries in the verification loop (why iteration count > 1)

1. The hook function must be named `otr_hook`, not `hook` (recorder's
   `luasupport.c` calls `lua_getglobal(L, "otr_hook")`). The recorder
   logs `cannot invoke otr_hook in Lua script` when missing — the
   plan's `hook()` naming was wrong.
2. Dawarich's `latitude`/`longitude` scalar columns are legacy and
   always NULL; the authoritative geometry is in the `lonlat` PostGIS
   column (`ST_AsText(lonlat::geometry)`). Early "it's broken" readings
   were me querying the wrong columns.
3. Default Recreate-strategy rollouts cause ~30s 502/503 windows on
   the ingress — tolerable, but every apply is visible as an outage
   to the phone. Batching edits is important.

## What is NOT in this change

- **Not** OTR_HTTPHOOK. Removed with this commit (dead env var).
- **Not** the one-shot backfill Job — that comes after the phone
  buffer has flushed to avoid racing against incoming hook POSTs
  (follow-up: code-h2r).
- **Not** Anca's bridge — a second Recorder instance or a smarter
  hook is needed to route her posts under her own Dawarich api_key
  (follow-up: code-72g).
- No Ingress or Service change — Commit 1 (`a21d4a44`) already landed
  those.

## Test Plan

### Automated

```
$ ../../scripts/tg apply --non-interactive
Apply complete! Resources: 1 added, 1 changed, 0 destroyed.

$ kubectl -n owntracks logs deploy/owntracks --tail=5
+ initializing Lua hooks from `/hook/dawarich-hook.lua'
+ dawarich-bridge: init
+ HTTP listener started on 0.0.0.0:8083, without browser-apikey
...
+ dawarich-bridge: tst=1 lat=0 lon=0 ok=true
```

### Manual Verification

```
$ VIKTOR_PW=$(vault kv get -field=credentials secret/owntracks | jq -r .viktor)
$ TST=$(date +%s)
$ kubectl -n owntracks run t --rm -i --image=curlimages/curl -- \
    curl -s -w 'HTTP %{http_code}\n' -X POST -u "viktor:$VIKTOR_PW" \
    -H 'Content-Type: application/json' \
    -H 'X-Limit-U: viktor' -H 'X-Limit-D: iphone-15pro' \
    -d "{\"_type\":\"location\",\"lat\":51.5074,\"lon\":-0.1278,\"tst\":$TST,\"tid\":\"vb\"}" \
    https://owntracks.viktorbarzin.me/pub
HTTP 200

$ sleep 3 && kubectl -n dbaas exec pg-cluster-1 -c postgres -- \
    psql -U postgres -d dawarich -c \
    "SELECT timestamp, ST_AsText(lonlat::geometry) FROM points \
     WHERE user_id=1 AND timestamp=$TST"
 timestamp  |        st_astext
------------+-------------------------
 1776555707 | POINT(-0.1278 51.5074)
```

Real phone traffic (from in-flight buffer flush) lands in Dawarich too:
`traefik logs -l app.kubernetes.io/name=traefik | grep 'POST /api/v1/owntracks/points'`
shows ingress POSTs from `owntracks` namespace to `dawarich` backend
with status 200.

### Reproduce locally

1. `vault login -method=oidc`
2. `kubectl -n owntracks logs deploy/owntracks --tail=20` — expect
   `dawarich-bridge: init` after the Lua loader line.
3. Do the curl above, poll the DB, expect `POINT(lon lat)`.

Closes: code-z9b

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 23:47:22 +00:00