Возрастные YouTube-видео не скачивались: с cookies yt-dlp отбрасывает клиент
android и остаётся web, которому нужно решить n-challenge, а в образе стоял
Node 20 при требуемом минимуме 22 ("JS runtimes: node-20.19.2 (unsupported)").
Ставим Deno — рекомендованный yt-dlp рантайм и один статический бинарник.
Instagram падал с "empty media response" одинаково с cookies и без них —
дело было не в сессии, а в устаревшем экстракторе: слой pip был закеширован
на yt-dlp 2026.06.09. Поднимаем нижнюю границу до 2026.7.4.
/cookies/check помечал проблемой с cookies ЛЮБОЙ сбой, из-за чего на поломку
JS-рантайма прилетел алерт про протухшие cookies и увёл разбор не туда.
Теперь ответ содержит status: ok | cookies_invalid | extraction_failed |
no_cookies, и админ-бот шлёт разные сообщения. Разбор ответа в bot.py
сохраняет совместимость со старым форматом без поля status.
Добавлен smoke_test.py — гоняет реальные ссылки (включая обе регрессии выше)
через запущенные сервисы и печатает таблицу. Запускать после каждой правки.
Схема получения cookies переведена с крона на разовый ручной экспорт:
cookies-cron/ -> cookies/, удалены скрипты с анти-паттерном
`--cookies-from-browser BROWSER --cookies FILE`, который wiki yt-dlp прямо
запрещает и который сам ломал YouTube-сессию ротацией. Ключевые cookies живут
около года, поэтому обновление по расписанию не нужно — триггером служит алерт
health-check. deliver_cookies.sh только доставляет файлы по scp.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add /cookies/check to youtube-downloader and instagram-downloader,
polled every 30 min from bot.py; alerts the admin bot on
healthy->broken transitions (with a 24h re-reminder cap while still
broken) and on recovery. Also stop leaking raw exception text to end
users - they now see a plain, friendly English message while the
admin bot still gets full technical detail via notify_admin_error.
Notify admin (URL + error text + user info) on any download failure
across all sources, and on the file-too-large case, mirroring the
existing successful-download notification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
YouTube's SABR-only streaming rollout produces errors like "Signature
solving failed" / "Requested format is not available" that don't
mention cookies, so the existing no-cookies fallback never triggered.
Cookies also disable the android client, which is often the only one
still returning usable formats under this restriction.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
chat_id is shared by every member of a group chat, so all group
messages incremented one shared row and username kept getting
overwritten by whoever sent last. Track link counts in a new
user_links table keyed by the real from_user.id instead.
Per-line strip() only stripped the first line's leading spaces, leaving
subsequent lines indented differently. RTL names (Arabic etc.) also
flipped the number/counter to the right via Telegram's bidi handling.
Adds users.link_count, incremented in bot.py whenever a user submits
a recognized link (regardless of eventual download success), and
surfaces it in admin_bot's /stat as a count of users who ever sent a
link plus a top-5 leaderboard by link count.
- Remove get_cookies.sh (targets a container without yt-dlp, already
broken), get_cookies_local.sh, start_all.sh/stop_all.sh — these
implement the old per-service-compose startup that now conflicts
with the unified root docker-compose.yml.
- Add cookies-cron/ — a reference copy of the scripts actually running
via cron on the separate browser-equipped machine, with setup
requirements and instructions for standing up a new cron host.
- Update README.md/ARCHITECTURE.md to describe the current unified
docker-compose + CI/CD deploy flow and host-mounted cookies instead
of the old per-folder workflow.
Cookies baked into the image (COPY . .) required a full CI/CD rebuild
to refresh, incompatible with the cron job on the fedora machine that
re-extracts cookies from a logged-in browser every 20 minutes and
scp's them to prod. Volume-mount the cookies file instead — yt-dlp
reads it from disk per request, so a fresh scp takes effect
immediately with no container restart. The image-baked copy still
exists as a build-time fallback.
Also refresh the committed cookies snapshot (the cron's source yt-dlp
turned out to be missing on the fedora box, so prior commits had been
carrying years-stale cookies without anyone noticing).
Admin sends /send_messages, the next text message is broadcast to
every user in the users table. Sends via TELEGRAM_BOT_TOKEN (the
client bot), not the admin bot's own token — broadcasting through
context.bot would land in the user's chat with the admin bot, which
they've never opened, instead of the client bot they actually use.
send_video_to_admin_bot() had no retry, unlike the client-facing
reply_video() path fixed earlier — a transient httpx.ReadError during
upload silently dropped the admin copy while the user still got their
video, since this function swallows its own exceptions.
EXTRACTOR_ARGS used ':' instead of ';' between player_client and skip
fields, so yt-dlp parsed "android:skip=translated_subs" and "hls" as
bogus player_client values and silently fell back to the web client
only, which fails the n-challenge and returns no formats.
--max-requests=1 fully restarted the gunicorn worker after every
single request, opening a race window that could drop the response
connection right after a successful download; raised to 20 with
jitter to keep the periodic-recycle safety net without doing it on
every request.
reply_video() to Telegram had no retry, so a transient httpx
transport error during upload discarded an already-downloaded video;
added the same 3-attempt retry pattern already used for the
downloader services.
Mirrors the LichessStatTgWeb pipeline: build all 7 service images,
push to the Gitea registry, transfer to the prod host via docker
save/load (registry isn't reachable from the external VPS), and
redeploy with docker-compose.prod.yml.
- Замена reply_video() на reply_document() в bot.py — Telegram больше не сжимает видео
- Исправление format_id в get_youtube_formats(): конкретные format codes + fallback best[height<=N]
- Замена bestvideo[height<=N]+bestaudio на best[height<=N] — гарантированно работает когда
YouTube не отдаёт отдельные video-only потоки для низких разрешений
- Добавлено логирование реально скачанного формата для диагностики
