Skip to content

feat: EPG improvements for better catch-up playback - #1164

Open
Alanon202 wants to merge 1 commit into
4gray:masterfrom
Alanon202:pr/epg-archive
Open

feat: EPG improvements for better catch-up playback#1164
Alanon202 wants to merge 1 commit into
4gray:masterfrom
Alanon202:pr/epg-archive

Conversation

@Alanon202

Copy link
Copy Markdown
Contributor

Improves EPG data reliability with three changes. First, EPG_FORCE_FETCH bypasses the 12-hour freshness check so manual refresh always re-fetches the XMLTV feed (needed when using different epg sources with different update times).

Second, changes the EPG import to preserve past programmes across refreshes — instead of wiping all old data before inserting new data, the parser now only replaces programmes starting today or later, keeping yesterday's archive intact. This is needed because many EPG sources only provide data looking forward, not backwards, and this enables supporting catch-up even in those instances.

Third, adds a rolling 7-day retention cap for this archive, so storage doesn't grow unbounded. Together these ensure catch-up programmes remain visible for their full archive window across daily EPG updates and most IPTV providers.

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown

Greptile Summary

This PR improves EPG catch-up reliability with three changes: EPG_FORCE_FETCH now bypasses the 12-hour freshness check correctly (and was fixed in this revision to send the queued UI notification); the database import switches from a full wipe to a selective delete that preserves yesterday-through-7-days-ago archive data; and a 7-day retention cap is enforced via the same delete predicate.

  • EPG_FORCE_FETCH (epg.events.ts): Adds the previously missing queued progress notification and wraps fetchEpgFromUrl in a try/catch returning a typed success/failure — no issues.
  • Selective delete + unique index (epg-database.ts): The deleteTodayAndFutureStmt logic is correct and runs atomically inside insertChannels; however, the CREATE UNIQUE INDEX executed in the constructor will throw for existing multi-source databases where two sources share the same channel, and the unique key omits source_url, causing INSERT OR REPLACE to silently drop per-source programme attribution on overlap.
  • Parser worker (epg-parser.worker.ts): The hasClearedSource → needsClear rename is equivalent and the non-atomic-delete concern from the previous review round is resolved.

Confidence Score: 3/5

The selective-delete and retention logic are sound, but the unique index creation in the EpgDatabase constructor will throw for any existing multi-source database where two EPG providers carry overlapping channels, breaking all EPG on upgrade.

Two distinct defects in the database layer affect real-world upgrade paths. First, creating the unique index inside the constructor rather than a migration means any user with duplicate (channel_id, start, title) rows — common with multi-source setups — gets a constructor throw and a dead EPG worker until they clear their database manually. Second, omitting source_url from the unique key means the last-parsed source overwrites earlier source_url values, silently breaking per-source programme filtering for channels covered by more than one feed.

apps/electron-backend/src/app/workers/epg-database.ts — specifically the CREATE UNIQUE INDEX call in the constructor (line 34-37) and the INSERT OR REPLACE unique key choice (line 39-42).

Important Files Changed

Filename Overview
apps/electron-backend/src/app/workers/epg-database.ts Adds unique index on (channel_id, start, title), switches to INSERT OR REPLACE, and adds selective delete for today+future+old-7-days; index creation in constructor risks crashing on upgrade for existing multi-source databases with overlapping channel data
apps/electron-backend/src/app/workers/epg-parser.worker.ts Simplifies source-clear flag from hasClearedSource to needsClear; logic is equivalent and the atomic-delete concern from previous review is resolved by delegating into insertChannels transaction
apps/electron-backend/src/app/events/epg.events.ts EPG_FORCE_FETCH handler now correctly sends queued progress before calling fetchEpgFromUrl directly, with proper try/catch returning success/failure shape

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant R as Renderer
    participant E as EpgEvents (Main)
    participant W as EpgWorkerService
    participant DB as EpgDatabase (Worker)

    R->>E: EPG_FORCE_FETCH(url)
    E->>W: deleteFetchedUrl(url)
    E->>W: sendProgressToRenderer(url, 'queued')
    W-->>R: progress: queued
    E->>W: fetchEpgFromUrl(url)
    W->>DB: new EpgDatabase() [constructor runs CREATE UNIQUE INDEX]
    Note over DB: Throws if existing rows have duplicate channel_id+start+title
    DB->>DB: deleteTodayAndFutureStmt in insertChannels txn
    Note over DB: Deletes start >= today OR start < now-7d
    loop Per channel batch
        DB->>DB: insertChannels
    end
    loop Per program batch
        DB->>DB: INSERT OR REPLACE - source_url not in unique key
    end
    W-->>R: EPG_COMPLETE / EPG_ERROR
    E-->>R: success true or false
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant R as Renderer
    participant E as EpgEvents (Main)
    participant W as EpgWorkerService
    participant DB as EpgDatabase (Worker)

    R->>E: EPG_FORCE_FETCH(url)
    E->>W: deleteFetchedUrl(url)
    E->>W: sendProgressToRenderer(url, 'queued')
    W-->>R: progress: queued
    E->>W: fetchEpgFromUrl(url)
    W->>DB: new EpgDatabase() [constructor runs CREATE UNIQUE INDEX]
    Note over DB: Throws if existing rows have duplicate channel_id+start+title
    DB->>DB: deleteTodayAndFutureStmt in insertChannels txn
    Note over DB: Deletes start >= today OR start < now-7d
    loop Per channel batch
        DB->>DB: insertChannels
    end
    loop Per program batch
        DB->>DB: INSERT OR REPLACE - source_url not in unique key
    end
    W-->>R: EPG_COMPLETE / EPG_ERROR
    E-->>R: success true or false
Loading
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
apps/electron-backend/src/app/workers/epg-database.ts:34-37
**Unique index creation in constructor crashes on multi-source upgrade**

`CREATE UNIQUE INDEX … ON epg_programs(channel_id, start, title)` will throw `SQLITE_CONSTRAINT_UNIQUE` and abort the `EpgDatabase` constructor if any two rows already share the same `(channel_id, start, title)` triplet. This is a realistic scenario for users who have configured more than one EPG source that both carry the same channel (e.g., `BBC.uk` at 22:00 "News at Ten" from Source A and Source B). Old code inserted both rows with different `source_url` values and no unique constraint, so duplicates on this triplet are present in the majority of multi-source databases. A failed constructor means the EPG worker cannot start at all until the user manually clears the database. The index should be created via a schema migration that de-dupes rows first, not inside the hot-path constructor.

Reviews (2): Last reviewed commit: "feat: add archive EPG feature" | Re-trigger Greptile

Comment thread apps/electron-backend/src/app/workers/epg-database.ts
Comment thread apps/electron-backend/src/app/workers/epg-parser.worker.ts Outdated
Comment thread apps/electron-backend/src/app/events/epg.events.ts
@Alanon202

Copy link
Copy Markdown
Contributor Author

@greptileai Please review the changes.

Comment thread apps/electron-backend/src/app/workers/epg-database.ts
@Alanon202

Copy link
Copy Markdown
Contributor Author

@greptileai Is this good now?

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown

Largely yes — all four previous review issues are resolved:

  • ✅ Duplicate accumulation: unique index + INSERT OR REPLACE handles it
  • ✅ Non-atomic delete: the deleteTodayAndFutureStmt now runs inside the insertChannels transaction
  • ✅ Missing queued UI notification: added before fetchEpgFromUrl
  • ✅ Constructor crash on upgrade: dedup DELETE runs before CREATE UNIQUE INDEX

One minor residual concern worth being aware of: the unique key is (channel_id, start, title) without source_url. In multi-source setups where two feeds carry the exact same programme, INSERT OR REPLACE will silently overwrite the source_url field to whichever source was processed last. Since deleteTodayAndFutureStmt filters by source_url, this means the first source's refresh won't reliably find and replace programmes that got re-attributed to the second source. In practice this only matters when two feeds have identical (channel_id, start, title) tuples — which may be uncommon — but worth noting for multi-source users.

Also, deleteTodayAndFuturePrograms() (the standalone public method at line 152) appears unused now that deletion is handled inside insertChannels. It can be removed unless you intend to call it from somewhere else.

Otherwise the logic is solid and the PR achieves its stated goals cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant