21d20dccf8
Some checks failed
ci/woodpecker/push/default Pipeline was canceled
Emo's ~13.7k-document import was going through the API upload path, which stages each file on the pod's EPHEMERAL scratch before queuing it. Any paperless pod or redis restart therefore destroyed all in-flight work (the "File not found" failures we hit) and required manual re-uploads. Move bulk ingest to paperless's consume directory placed on the encrypted PVC, with PAPERLESS_CONSUMER_POLLING so the whole folder is re-scanned periodically (and on startup) with a file-stability check. Files now live on durable storage and survive any restart — the folder is the queue and self-heals, so we can copy everything in fast and let it process over time with zero retry/integrity risk. RECURSIVE preserves the source tree (avoids basename collisions); owner+tag come from a consumption workflow. Bump TASK_WORKERS 4->6 to speed the OCR/convert-bound processing (node6 has the core headroom for one pod) and mem limit 8->10Gi for the extra workers. Revert workers/mem/consume envs to defaults once the import ends. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| .terraform.lock.hcl | ||
| main.tf | ||
| providers.tf | ||
| secrets | ||
| terragrunt.hcl | ||