aboutsummaryrefslogtreecommitdiffstats
path: root/youtube/util.py
Commit message (Collapse)AuthorAgeFilesLines
* Set 'ios' client to bypassAstound2024-03-311-0/+16
| | | | | | | absidue notes that blockage of the android client is collateral damage due to YouTube's war with ReVanced. Switching to iOS should keep us out of the line of fire for now: https://github.com/yt-dlp/yt-dlp/issues/9554#issuecomment-2026828421
* Temporal fix: all requests with ANDROID client get redirected to aQvGIIdgFDM ↵Astound2024-03-311-0/+19
| | | | | | video, hence the different "content not available" Set YTMUSIC_ANDROID client instead, but it's just the matter of time before youtube updates that one too :(
* Update user-agents and update android client parameters to fix blockageAstound2024-03-301-10/+12
|
* Use response.headers instead of response.getheaders()Astound2024-03-111-1/+1
| | | | response.getheaders() will be deprecated by urllib3.
* Update android client version to fix 400 Bad RequestAstound2024-03-101-3/+3
|
* Fix channel about tabAstound2024-01-221-0/+79
|
* Set hqdefault thumnail imagesJesus E2023-06-181-2/+2
|
* Fix failing exit node retry testJames Taylor2022-02-161-2/+1
| | | | | | The urllib3 retries.history wasn't working anyways Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix error during exit blockage detection when Set-Cookie missingJames Taylor2022-02-151-1/+1
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix MaxRetryErrors due to Tor exit node blockageJames Taylor2022-02-151-3/+16
| | | | | | | | Sometimes YouTube redirects to a google.com/sorry page, seemingly setting up redirect loops. Other times the url redirects to itself. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Prefix youtube URLs in video descriptions and channel about pageJames Taylor2021-08-091-0/+7
| | | | | | Closes #75 Signed-off-by: Jesús <heckyel@hyperbola.info>
* Switch to new comments api now that old one is being disabledJames Taylor2021-08-091-0/+13
| | | | | | | | | | | watch_comment api periodically gives the error "Top level comments mweb servlet is turned down." The continuation items for the new api are in a different arrangement in the json, so changes were necessary to the extract_items function. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Cleaner channel-checking console error messages when Tor is closedJesús2021-07-281-1/+5
| | | | by James Taylor <user234683@users.noreply.github.com>
* Restart tor connection pool when the tor port setting is changedJames Taylor2021-07-281-0/+4
| | | | | | | Otherwise the old port continued to be used until the program was restarted. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Friendlier error message when Tor is closed or network is downJames Taylor2021-07-041-3/+22
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Capitalize name appJesús2021-06-101-1/+1
|
* Save uploader id in local playlistsJames Taylor2021-05-061-2/+5
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* util.py: Encode data using utf-8, not asciiJames Taylor2021-03-031-2/+2
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* util.py: Fix data not being passed in requestsJames Taylor2021-03-031-2/+2
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* Debugging response saving: save page even if it's an HTTP errorJames Taylor2021-02-261-8/+9
| | | | | | | This will help debug new types of exit node blockage or other errors. Signed-off-by: Jesús <heckyel@hyperbola.info>
* TorMgr: Change new identity cooldown to 7 seconds instead of 6James Taylor2021-02-131-4/+4
| | | | | | Because sometimes a new identity was not being respected Signed-off-by: Jesús <heckyel@hyperbola.info>
* fetch_url: 429: get new identity regardless of page contentJames Taylor2021-01-131-3/+1
| | | | | | | | | | | New 429 captcha page doesn't have IP. This new page appears to match the 429 code plus the json of {"redirect": ...} which would be occasionally received when the pbj json endpoint was used in the past. Closes #22 Signed-off-by: Jesús <heckyel@hyperbola.info>
* General theme: fix syntax W3C markup and add improve 'Published' dateJesús2020-12-301-0/+7
| | | | Signed-off-by: Jesús <heckyel@hyperbola.info>
* strip_non_ascii in comments-author-nameJesús2020-12-301-0/+6
|
* util: Remove obsolete parse_info_prepare_for_html functionJames Taylor2020-12-281-9/+0
| | | | | | | | Info parsing is handled by yt_data_extract, and html post-processing is done with util.prefix_urls and util.add_extra_html_info Signed-off-by: Jesús <heckyel@hyperbola.info>
* Remove obsolete uppercase_escape functionJames Taylor2020-12-281-6/+0
| | | | | | | This function was only necessary with the old ajax format, which was removed in 4d7bba92eb62518e2273d030235214f4a7605444 Signed-off-by: Jesús <heckyel@hyperbola.info>
* TorManager: Add delay before subsequent new identity retriesJames Taylor2020-12-251-1/+18
| | | | | | | | | | | | | | | The request can be retried immediately after the first new identity, but if we do more new identities, we have to wait for at least 6 seconds before doing the request, otherwise it won't be done on a new ip based on my experiments. Potential issue: If after getting third new identity, request takes > 12 seconds (since timeout is 15) and returns 429, then the Tor Manager will let it do a 4th try instead of giving up (meaning request is taking forever from user's perspective). Should be a very rare occurence however. Signed-off-by: Jesús <heckyel@hyperbola.info>
* Exit node retrying: Retry 3 times. Also add tests for it.James Taylor2020-12-211-4/+17
| | | | | | Closes #20 Signed-off-by: Jesús <heckyel@hyperbola.info>
* Fix youtube mixesJames Taylor2020-12-181-2/+8
| | | | | | | They cannot be viewed on their own, so change url in items to go to the video+playlist instead Signed-off-by: Jesús <heckyel@hyperbola.info>
* pep8Jesús2020-12-151-26/+40
|
* fetch_url: Use time.monotonic for comptability w/ tor_mgrJames Taylor2020-11-291-5/+8
| | | | | Buggy exit node retry behavior due to fetch_url still using time.time() instead of the time.monotonic() used by new_identity
* use Tor to resolve domain namesuser2346842020-11-121-2/+2
|
* new_identity: authenticate with blank passwordJames Taylor2020-10-251-1/+1
| | | | See https://github.com/user234683/youtube-local/issues/20#issuecomment-716198162
* Automatically change tor circuit once if ip is blockedJames Taylor2020-10-251-45/+117
| | | | | | | Use stem library to send a new identity signal via the tor control port. See #20
* fetch_url: move content decoding to before 429 checkJames Taylor2020-10-251-1/+3
| | | | | Content should be decoded before we try reading page to check for 429. Lucky that it worked at all before.
* Put vid title at end of download urls so downloads w/ that filenameJames Taylor2020-10-221-0/+54
|
* fix stylezrose5842020-10-091-1/+1
|
* settings.py: add 'proxy_images'zrose5842020-10-071-6/+7
|
* Increase max redirects for video routingJames Taylor2020-09-221-3/+13
|
* Add setting for tor_portuser9381202020-09-191-3/+3
|
* Add tor video routingJames Taylor2020-09-181-14/+23
| | | | | | | | | | Includes non-tor video routing by default, so no more chances of the browser leaking headers or user agent to googlevideo Adjust settings upgrade system to facilitate change to route_tor setting. Add some more space on settings page for dropdown settings so does not overflow due to options with long names. Closes #7
* Don't use brotli if import failsJames Taylor2020-08-111-2/+9
|
* Fix failure to parse comments when there's one from deleted channelJames Taylor2020-03-081-0/+9
| | | | | | Specifically, fix failures when any of the fields from the parsed comment are None, such as author, author_url, etc. (failure due to string concatenation when building urls).
* Ensure urllib3 and python version not leaked during video file checkJames Taylor2020-02-181-1/+2
| | | | | This is likely not a big deal since it is already assumed that video file server logs are not plugged into Google's tracking infrastructure, but it doesn't hurt to give less info.
* Fix MaxRetryError when checking for video URL accessJames Taylor2020-02-181-2/+10
| | | | | | | | The default urllib3 max redirect amount was set to 3. Change it to 10 and do not fail if there is a problem with checking for URL access. Just print the error to the console and proceed. Also add an unrelated remark about the bcptr=9999999999 parameter in watch.py
* Check for 403 errors and fallback on InvidiousJames Taylor2020-02-011-0/+8
| | | | 403 errors on the video urls happen typically when a video has copyrighted content or was livestreamed originally. They appear to not happen (or at least happen less frequently) if the Tor exit node used ipv6, however.
* Give a proper error message for 429 errorsJames Taylor2020-01-311-0/+23
| | | | | | These occur when too many requests are coming from a Tor exit node. Before, there would be an error page with an exception instructing users to report the issue. But this is an expected and persistent issue.
* Extraction: Move html post processing stuff from yt_data_extract to utilJames Taylor2019-12-191-0/+38
|
* Extraction: Rewrite item_extraction for better error handling and ↵James Taylor2019-12-181-0/+2
| | | | readability, rename extracted names for more consistency
* Extraction: Replace youtube-dl with custom-built watch page extractionJames Taylor2019-10-171-1/+8
|
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-15 10:50:42 +0000