Fix YouTube hammering, sync rate limiting, and Following load time
Sync throttling: - sync-all now skips channels crawled within the last 6 hours (prevents re-scraping 1266 channels on every button press) - Channels are queued into a single _index_channels_batch task that runs with 1.5s delay between each yt-dlp call instead of firing 1266 background tasks simultaneously - Startup enrich task reduced from 10 to 3 videos (3 yt-dlp calls on each container restart) - Enrich task adds 2s sleep between metadata fetches SQLite stability: - busy_timeout=5000 prevents SQLITE_BUSY errors under concurrent load - synchronous=NORMAL speeds up writes without data loss risk (safe with WAL) Following page: - staleTime: 60s on channels query so cached data is reused immediately on revisit; gcTime keeps it in memory for 5 min Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -152,7 +152,7 @@ def on_startup():
|
||||
# Backfill descriptions for videos that don't have them yet (runs in background)
|
||||
import threading
|
||||
from .routers.channels import _enrich_missing_task
|
||||
threading.Thread(target=_enrich_missing_task, args=(10,), daemon=True).start()
|
||||
threading.Thread(target=_enrich_missing_task, args=(3,), daemon=True).start()
|
||||
|
||||
|
||||
@app.get("/api/health")
|
||||
|
||||
Reference in New Issue
Block a user