How it works

The pipeline

    flowchart TD
  A[walk library roots] --> B[parse path:<br>ids, title, year, season/episode]
  B --> C{sidecar<br>already there?}
  C -- yes --> H[record: have]
  C -- no --> P[record: pending]
  P --> Q[pick a batch:<br>limit, filters, backoff]
  Q --> S[provider chain:<br>search by id, rank candidates]
  S --> N[Normalize:<br>is this really SRT?]
  N --> W[Write:<br>temp file + atomic rename]
  W --> D[record: downloaded<br>+ count against quota]
  

Scanning and fetching are separate on purpose: scan never touches the network, so it is safe to run repeatedly while you tune ignore patterns or naming.

The walk

Roots are walked depth-first. One directory listing is cached at a time — because the walk finishes a directory before moving on, a single-entry cache is enough to avoid re-listing a folder once per video file in it. On a network mount that is the difference between one round trip per folder and N.

An unreadable directory is skipped rather than aborting the whole library. A root that cannot be read at all is an error, because it usually means a mount is missing.

Choosing what to fetch

Each (path, language) pair is a row. A run selects pending rows ordered by fewest attempts, then oldest attempt, then path — so work rotates through the library instead of retrying the head of the list forever.

Eligible rows are:

• everything pending, plus
• not_found / failed rows whose last attempt is older than run.retry_after_hours and whose attempts is under run.max_attempts

--force ignores both conditions.

The batch is then worked by run.concurrency workers over a jobs channel. The quota counter and run statistics are mutex-guarded.

Ranking candidates

Search returns candidates; they are scored and sorted best-first:

Exact-id matching happens earlier, at query time: the provider is asked about one specific title, so ranking only has to choose between that title’s releases.

The top three candidates are tried in order, because the best-scoring one is occasionally a dead link or a file that fails validation.

Validation and writing

Downloaded bytes go through Normalize before anything touches the filesystem. It strips a UTF-8 BOM, converts CRLF/CR to LF, ensures a trailing newline, and rejects the content unless it contains an SRT timecode line (00:01:02,345 --> 00:01:04,567). An empty body or an HTML error page — the two things providers actually return when something goes wrong — is rejected with a message that says which.

The write is atomic: a temp file is created in the destination directory and renamed into place, so the rename is a same-filesystem operation. A dropped mount mid-write leaves a stray temp file, never a half-written .srt that a later scan would count as done. Files are written 0644, and chown’d to PUID:PGID when running as root.

A write failure marks the item failed and stops the chain for that item — it is not the provider’s fault and would hit every other candidate too.

Item status lifecycle

    stateDiagram-v2
  [*] --> pending: scan finds no sidecar
  pending --> have: sidecar appears on disk
  pending --> downloaded: provider delivered
  pending --> not_found: no provider had it
  pending --> failed: error during fetch
  not_found --> pending: retry_after_hours elapsed<br>(under max_attempts)
  failed --> pending: retry_after_hours elapsed<br>(under max_attempts)
  downloaded --> have: later scan sees the file
  

A rescan never regresses progress: once a row is downloaded, not_found, or failed, a scan leaves the status alone — with one exception, finding a sidecar on disk always wins, so dropping a subtitle in by hand is recognized.

Rows whose video file no longer exists are deleted on the next scan, so a library that churns does not grow the database forever.

State on disk

SQLite, WAL mode, at run.db_path. Two tables matter:

• items — primary key (path, lang), carrying ids, status, attempts, last error.
• downloads — (day, provider) → count, the local quota counter.

WAL is what makes subtrace status safe to run during a long fetch.

Shutdown

fetch and watch cancel on SIGINT/SIGTERM. Cancellation is checked between items, so a run stops cleanly and never leaves a partial write behind.