Mattias Thall e6faf8e08e Drastically reduce discovery yt-dlp call count: 64 → ~21 ...

Each yt-dlp call is a separate subprocess that opens a new HTTP session with
YouTube. 64 sessions in a row looks like a bot regardless of rate limiting.

Changes:
- crawl_by_search: 30 queries → 10 (top 5 tags, 4 channel names, 1 serendipity)
- update_liked_signal: 10 queries → 4
- update_watch_signal: removed (tags already included in crawl_by_search)
- update_trending_signal: 2 regions → 1 (first region only)
- update_graph_signal: 12 sampled channels → 6

New total: ~21 yt-dlp calls per run (~105s with 5s gaps) vs ~320s before.
Signal quality is preserved — the removed queries were low-marginal-value
duplicates of content already covered by the remaining ones.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-27 02:14:25 +02:00

25 KiB

Raw Blame History

View Raw