Use this checklist when isolating each desk from the shared research-pipeline.db. Every item must pass before signing off on a desk.
| # | Item | What to verify |
|---|---|---|
| 1 | Databases isolated | All tables in their own .db files, not in research-pipeline.db |
| 2 | Scraper queue independent | Desk has its own cron job, not sharing the 11 AM batch with other desks |
| 3 | Recovery queue | Desk's scrapers are in recovery list (or excluded if ToDo group) |
| 4 | Freshness tracking | Every table has a freshness config entry with correct schedule and timestamp column |
| 5 | Dashboard views | All source-table entries point to isolated .db files |
| 6 | Edge scanner | Reads from isolated .db files, not shared database |
| 7 | Scan windows | All data tagged with scan_type (6am/8am/10am/2pm/6pm/close) |
| 8 | Data cleanliness | No settled prices, no live data, same-day only for game-day tables |
| 9 | Column formatting | Human-readable tickers, game date/time, exec price where applicable |
| 10 | Filters | Column filters enabled on dashboard views |
| 11 | No cross-desk dependencies | Desk doesn't break if another desk's scraper crashes |
| 12 | Doubleheader handling | Game numbers tracked correctly for sports with doubleheaders |
| 13 | Alerts | Telegram notification if any table in the desk goes stale |
| # | Item | Status | Notes |
|---|---|---|---|
| 1 | Databases isolated | PARTIAL | kalshi-mlb-prices.db and pinnacle-mlb.db done. Futures, props, stats, pitching, batting, weather, model data still in research-pipeline.db |
| 2 | Scraper queue independent | NOT DONE | MLB scrapers still in the shared 11 AM batch. Need own cron. |
| 3 | Recovery queue | DONE | MLB scrapers in recovery list, ToDo desks removed |
| 4 | Freshness tracking | PARTIAL | Pinnacle MLB freshness added. Kalshi uses old system. Full poller not built yet (council ruling pending). |
| 5 | Dashboard views | PARTIAL | Kalshi game prices and Pinnacle odds point to isolated .db. Rest still shared. |
| 6 | Edge scanner | DONE | Reads Kalshi from kalshi-mlb-prices.db, Pinnacle from pinnacle-mlb.db |
| 7 | Scan windows | DONE | 6am/8am/10am/2pm/6pm/close for odds tables |
| 8 | Data cleanliness | DONE | Settled/live data purged, same-day filter, scan window gate |
| 9 | Column formatting | DONE | Human-readable Kalshi tickers, game date/time, exec price, spread |
| 10 | Filters | DONE | Column filters on kalshi-mlb and pinnacle-mlb views |
| 11 | No cross-desk dependencies | PARTIAL | Odds isolated. Stats still share 11 AM batch queue. |
| 12 | Doubleheader handling | DONE | game_number column, G1/G2 detection from Pinnacle game_id |
| 13 | Alerts | NOT DONE | Waiting on freshness tracker council ruling |
Next steps for MLB: