aboutsummaryrefslogtreecommitdiffstats
path: root/youtube/util.py
Commit message (Collapse)AuthorAgeFilesLines
* feat: improve 429 handling with Tor support and clean CIAstounds15 hours1-3/+9
| | | | | | - Retry with new Tor identity on 429 - Improve error logging - Remove .build.yml and .drone.yml
* Release v0.4.0 - HD Thumbnails, YouTube 2024+ Support, and yt-dlp Integrationv0.4.0Astounds16 hours1-68/+179
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Major Features: - HD video thumbnails (hq720.jpg) with automatic fallback to lower qualities - HD channel avatars (240x240 instead of 88x88) - YouTube 2024+ lockupViewModel support for channel playlists - youtubei/v1/browse API integration for channel playlist tabs - yt-dlp integration for multi-language audio and subtitles Bug Fixes: - Fixed undefined `abort` import in playlist.py - Fixed undefined functions in proto.py (encode_varint, bytes_to_hex, succinct_encode) - Fixed missing `traceback` import in proto_debug.py - Fixed blurry playlist thumbnails using default.jpg instead of HD versions - Fixed channel playlists page using deprecated pbj=1 format Improvements: - Automatic thumbnail fallback system (hq720 → sddefault → hqdefault → mqdefault → default) - JavaScript thumbnail_fallback() handler for 404 errors - Better thumbnail quality across all pages (watch, channel, playlist, subscriptions) - Consistent HD avatar display for all channel items - Settings system automatically adds new settings without breaking user config Files Modified: - youtube/watch.py - HD thumbnails for related videos and playlist items - youtube/channel.py - HD thumbnails for channel playlists, youtubei API integration - youtube/playlist.py - HD thumbnails, fixed abort import - youtube/util.py - HD thumbnail URLs, avatar HD upgrade, prefix_url improvements - youtube/comments.py - HD video thumbnail - youtube/subscriptions.py - HD thumbnails, fixed abort import - youtube/yt_data_extract/common.py - lockupViewModel support, extract_lockup_view_model_info() - youtube/yt_data_extract/everything_else.py - HD playlist thumbnails - youtube/proto.py - Fixed undefined function references - youtube/proto_debug.py - Added traceback import - youtube/static/js/common.js - thumbnail_fallback() handler - youtube/templates/*.html - Added onerror handlers for thumbnail fallback - youtube/version.py - Bump to v0.4.0 Technical Details: - All thumbnail URLs now use hq720.jpg (1280x720) when available - Fallback handled client-side via JavaScript onerror handler - Server-side avatar upgrade via regex in util.prefix_url() - lockupViewModel parser extracts contentType, metadata, and first_video_id - Channel playlist tabs now use youtubei/v1/browse instead of deprecated pbj=1 - Settings version system ensures backward compatibility
* Fix NoneType inside comments.pyJesus2025-03-081-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Bug: Traceback (most recent call last): File "/home/rusian/yt-local/youtube/comments.py", line 180, in video_comments post_process_comments_info(comments_info) File "/home/rusian/yt-local/youtube/comments.py", line 81, in post_process_comments_info comment['author'] = strip_non_ascii(comment['author']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/rusian/yt-local/youtube/util.py", line 843, in strip_non_ascii stripped = (c for c in string if 0 < ord(c) < 127) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: 'NoneType' object is not iterable During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 900, in gevent._gevent_cgreenlet.Greenlet.run File "/home/rusian/yt-local/youtube/comments.py", line 195, in video_comments comments_info['error'] = 'YouTube blocked the request. IP address: %s' % e.ip ^^^^ AttributeError: 'TypeError' object has no attribute 'ip' 2025-03-08T01:25:47Z <Greenlet at 0x7f251e5279c0: video_comments('hcm55lU9knw', 0, lc='')> failed with AttributeError
* minor fix deprecation warningAstound2025-03-011-1/+1
| | | | | | tests/test_util.py: 14 warnings /home/runner/work/yt-local/youtube-local/youtube/util.py:321: DeprecationWarning: HTTPResponse.getheader() is deprecated and will be removed in urllib3 v2.1.0. Instead use HTTPResponse.headers.get(name, default). response.getheader('Content-Encoding', default='identity'))
* util: use visitorData for api requestAstound2025-02-281-26/+93
| | | | watch: use android_vr client to get player data
* Revert "Refactoring code and reuse INNERTUBE_CLIENTS"Jesus2024-10-111-61/+37
| | | | This reverts commit 8af98968dd4325d5686bfed109aa4ed18b17edbc.
* Switch to android test suite client by defaultJesus2024-06-111-0/+23
| | | | | | | Invidious' solution to the destruction of the android client: https://github.com/iv-org/invidious/pull/4650 Fixes #207
* set ios clientAstound2024-04-271-16/+16
|
* Refactoring code and reuse INNERTUBE_CLIENTSAstound2024-04-211-41/+36
|
* updateAstound2024-04-211-2/+2
| | | | update android_music client
* updateAstound2024-04-211-2/+2
| | | | update android client
* Update `android` paramsAstound2024-04-061-6/+2
| | | | Discovered by LuanRT - https://github.com/LuanRT/YouTube.js/pull/624
* Set 'ios' client to bypassAstound2024-03-311-0/+16
| | | | | | | absidue notes that blockage of the android client is collateral damage due to YouTube's war with ReVanced. Switching to iOS should keep us out of the line of fire for now: https://github.com/yt-dlp/yt-dlp/issues/9554#issuecomment-2026828421
* Temporal fix: all requests with ANDROID client get redirected to aQvGIIdgFDM ↵Astound2024-03-311-0/+19
| | | | | | video, hence the different "content not available" Set YTMUSIC_ANDROID client instead, but it's just the matter of time before youtube updates that one too :(
* Update user-agents and update android client parameters to fix blockageAstound2024-03-301-10/+12
|
* Use response.headers instead of response.getheaders()Astound2024-03-111-1/+1
| | | | response.getheaders() will be deprecated by urllib3.
* Update android client version to fix 400 Bad RequestAstound2024-03-101-3/+3
|
* Fix channel about tabAstound2024-01-221-0/+79
|
* Set hqdefault thumnail imagesJesus E2023-06-181-2/+2
|
* Fix failing exit node retry testJames Taylor2022-02-161-2/+1
| | | | | | The urllib3 retries.history wasn't working anyways Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix error during exit blockage detection when Set-Cookie missingJames Taylor2022-02-151-1/+1
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix MaxRetryErrors due to Tor exit node blockageJames Taylor2022-02-151-3/+16
| | | | | | | | Sometimes YouTube redirects to a google.com/sorry page, seemingly setting up redirect loops. Other times the url redirects to itself. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Prefix youtube URLs in video descriptions and channel about pageJames Taylor2021-08-091-0/+7
| | | | | | Closes #75 Signed-off-by: Jesús <heckyel@hyperbola.info>
* Switch to new comments api now that old one is being disabledJames Taylor2021-08-091-0/+13
| | | | | | | | | | | watch_comment api periodically gives the error "Top level comments mweb servlet is turned down." The continuation items for the new api are in a different arrangement in the json, so changes were necessary to the extract_items function. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Cleaner channel-checking console error messages when Tor is closedJesús2021-07-281-1/+5
| | | | by James Taylor <user234683@users.noreply.github.com>
* Restart tor connection pool when the tor port setting is changedJames Taylor2021-07-281-0/+4
| | | | | | | Otherwise the old port continued to be used until the program was restarted. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Friendlier error message when Tor is closed or network is downJames Taylor2021-07-041-3/+22
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Capitalize name appJesús2021-06-101-1/+1
|
* Save uploader id in local playlistsJames Taylor2021-05-061-2/+5
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* util.py: Encode data using utf-8, not asciiJames Taylor2021-03-031-2/+2
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* util.py: Fix data not being passed in requestsJames Taylor2021-03-031-2/+2
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Debugging response saving: save page even if it's an HTTP errorJames Taylor2021-02-261-8/+9
| | | | | | | This will help debug new types of exit node blockage or other errors. Signed-off-by: Jesús <heckyel@hyperbola.info>
* TorMgr: Change new identity cooldown to 7 seconds instead of 6James Taylor2021-02-131-4/+4
| | | | | | Because sometimes a new identity was not being respected Signed-off-by: Jesús <heckyel@hyperbola.info>
* fetch_url: 429: get new identity regardless of page contentJames Taylor2021-01-131-3/+1
| | | | | | | | | | | New 429 captcha page doesn't have IP. This new page appears to match the 429 code plus the json of {"redirect": ...} which would be occasionally received when the pbj json endpoint was used in the past. Closes #22 Signed-off-by: Jesús <heckyel@hyperbola.info>
* General theme: fix syntax W3C markup and add improve 'Published' dateJesús2020-12-301-0/+7
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* strip_non_ascii in comments-author-nameJesús2020-12-301-0/+6
|
* util: Remove obsolete parse_info_prepare_for_html functionJames Taylor2020-12-281-9/+0
| | | | | | | | Info parsing is handled by yt_data_extract, and html post-processing is done with util.prefix_urls and util.add_extra_html_info Signed-off-by: Jesús <heckyel@hyperbola.info>
* Remove obsolete uppercase_escape functionJames Taylor2020-12-281-6/+0
| | | | | | | This function was only necessary with the old ajax format, which was removed in 4d7bba92eb62518e2273d030235214f4a7605444 Signed-off-by: Jesús <heckyel@hyperbola.info>
* TorManager: Add delay before subsequent new identity retriesJames Taylor2020-12-251-1/+18
| | | | | | | | | | | | | | | The request can be retried immediately after the first new identity, but if we do more new identities, we have to wait for at least 6 seconds before doing the request, otherwise it won't be done on a new ip based on my experiments. Potential issue: If after getting third new identity, request takes > 12 seconds (since timeout is 15) and returns 429, then the Tor Manager will let it do a 4th try instead of giving up (meaning request is taking forever from user's perspective). Should be a very rare occurence however. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Exit node retrying: Retry 3 times. Also add tests for it.James Taylor2020-12-211-4/+17
| | | | | | Closes #20 Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix youtube mixesJames Taylor2020-12-181-2/+8
| | | | | | | They cannot be viewed on their own, so change url in items to go to the video+playlist instead Signed-off-by: Jesús <heckyel@hyperbola.info>
* pep8Jesús2020-12-151-26/+40
|
* fetch_url: Use time.monotonic for comptability w/ tor_mgrJames Taylor2020-11-291-5/+8
| | | | | Buggy exit node retry behavior due to fetch_url still using time.time() instead of the time.monotonic() used by new_identity
* use Tor to resolve domain namesuser2346842020-11-121-2/+2
|
* new_identity: authenticate with blank passwordJames Taylor2020-10-251-1/+1
| | | | See https://github.com/user234683/youtube-local/issues/20#issuecomment-716198162
* Automatically change tor circuit once if ip is blockedJames Taylor2020-10-251-45/+117
| | | | | | | Use stem library to send a new identity signal via the tor control port. See #20
* fetch_url: move content decoding to before 429 checkJames Taylor2020-10-251-1/+3
| | | | | Content should be decoded before we try reading page to check for 429. Lucky that it worked at all before.
* Put vid title at end of download urls so downloads w/ that filenameJames Taylor2020-10-221-0/+54
|
* fix stylezrose5842020-10-091-1/+1
|
* settings.py: add 'proxy_images'zrose5842020-10-071-6/+7
|
generated by cgit v1.2.3 (git 2.25.1) at 2026-03-23 17:25:39 +0000