Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | yt_data_extract: Simplify extract_items so it needs only 1 while loop | James Taylor | 2019-12-26 | 1 | -32/+31 | |
| | ||||||
* | items: commatize channel video count and playlist video count | James Taylor | 2019-12-24 | 1 | -2/+2 | |
| | ||||||
* | extract_item_info: Don't extract author, author_id, etc. for channel items | James Taylor | 2019-12-24 | 1 | -7/+8 | |
| | | | | Philosophically, a channel doesn't create itself. | |||||
* | Fix extract_approx_int not working for non-approx ints, make extract_int ↵ | James Taylor | 2019-12-24 | 1 | -2/+2 | |
| | | | | | | | | more robust For example, "354 subscribers" wasn't being extracted correctly be extract_approx_int. Make extract_approx_int and extract_int only extract integers that are words. So e.g. 342 will not be extracted from internetuser342 | |||||
* | Channel searching: indicate if there's no results | James Taylor | 2019-12-23 | 1 | -1/+5 | |
| | ||||||
* | Regression: Fix channel extraction 'items' key not present when there's no ↵ | James Taylor | 2019-12-23 | 1 | -2/+3 | |
| | | | | | | items. Examples: Empty channels, no search results | |||||
* | Channel: Change search results to use next and previous page buttons | James Taylor | 2019-12-23 | 5 | -39/+83 | |
| | | | | Because youtube doesn't give the number of search results, so previous behavior would give an error if a page number out of range was selected. | |||||
* | Subscriptions: Cleaner error message when checking terminated channels | James Taylor | 2019-12-22 | 1 | -1/+3 | |
| | | | | Don't display a nasty traceback in that case. | |||||
* | Subscriptions: Make uploader name clickable, with link to channel | James Taylor | 2019-12-22 | 1 | -2/+4 | |
| | ||||||
* | Finally fix video count on channels accessed through general urls, rather ↵ | James Taylor | 2019-12-22 | 1 | -19/+34 | |
| | | | | | | | | | | | | | than just channel id. It was set to a fake value of 1000 previously in order to ensure there would be enough page buttons. This was because two sequential requests are necessary (one to get the channel id corresponding to the custom url, another to get the number of videos from the "all uploaded videos" playlist, the url for which can be generated from the channel id). Since Tor has a high latency, I thought at the time that this would be too slow, but in practice it's not too big of a deal. Introduces cachetools dependency in order to cache the function which gets the number of videos. The get_channel_id function has also been fixed since the ajax api seems to have been removed. | |||||
* | channel.py: Refactor channel_id route logic into general channel url logic. | James Taylor | 2019-12-22 | 1 | -53/+21 | |
| | | | | | Deduplicates the code. channel_id logic was previously separate because of the need to get the number of videos and different page numbers Also makes search work for general urls, not just channel_id urls | |||||
* | Rewrite channel extraction with proper error handling and new extraction ↵ | James Taylor | 2019-12-21 | 3 | -47/+48 | |
| | | | | | | names. Extract subscriber_count correctly. Don't just shove english strings into info['stats']. Actually give semantic names for the stats. | |||||
* | Fix extract_approx_int. Fixes incorrect subscriber count on channels. | James Taylor | 2019-12-21 | 1 | -2/+2 | |
| | | | | It wasn't working because decimals such as 15.1M weren't considered, so it was extracting "1M" | |||||
* | Watch: Add padding in description box and urlize links | James Taylor | 2019-12-20 | 1 | -1/+2 | |
| | ||||||
* | Watch: display comment count and whether comments are disabled | James Taylor | 2019-12-20 | 2 | -8/+19 | |
| | ||||||
* | Better error handling for incorrect watch page urls | James Taylor | 2019-12-20 | 1 | -2/+4 | |
| | | | | | - Correctly handle /embed, /watch with no video ids - Correctly report error for this and for too short video ids | |||||
* | Add custom 500 error page. Display the traceback. Center and format error ↵ | James Taylor | 2019-12-20 | 2 | -1/+39 | |
| | | | | | | page in general. Also add a link to github for reporting the exception. | |||||
* | Add support for /embed urls | James Taylor | 2019-12-20 | 1 | -2/+3 | |
| | ||||||
* | Subscriptions: Display currently selected tag in page title | James Taylor | 2019-12-20 | 1 | -1/+5 | |
| | ||||||
* | Watch: Add border around badges such as unlisted badge | James Taylor | 2019-12-20 | 1 | -1/+3 | |
| | | | | Especially for the light theme | |||||
* | Fix regression: date extraction broken. Move constants to correct file in ↵ | James Taylor | 2019-12-20 | 2 | -2/+2 | |
| | | | | yt_data_extract | |||||
* | Subscriptions: Display selected tag above videos. | James Taylor | 2019-12-20 | 1 | -0/+6 | |
| | | | | Otherwise, it wasn't clear enough that a tag was selected. | |||||
* | Merge branch 'modular-data-extract' | James Taylor | 2019-12-19 | 20 | -780/+1760 | |
|\ | | | | | | | | | | | | | | | | | | | | | Commits in this branch are prefixed with "Extraction:" This branch refactors data extraction. All such functionality has been moved to the yt_data_extract module. Responses from requests are given to the module and it parses them into a consistent, more useful format. The dependency on youtube-dl has also been dropped and this functionality has been built from scratch for these reasons: (1) I've noticed youtube-dl breaks more often than invidious (which uses watch page extraction built from scratch) in response to changes from Youtube, so I'm hoping what I wrote will also be less brittle. (2) Such breakage is inconvenient because I have to manually merge the fixes since I had to make changes to youtube-dl to make it do things such as extracting related videos. (3) I have no control over error handling and request pooling with youtube-dl, since it does all the requests (these would require intrusive changes I don't want to maintain). (4) I will now be able to finally display the number of comments and whether comments are disabled without making additional requests. | |||||
| * | Extraction: Move non-stateful signature decryption functionality into ↵ | James Taylor | 2019-12-19 | 3 | -86/+110 | |
| | | | | | | | | yt_data_extract | |||||
| * | Extraction: Move stuff around in files and put underscores in front of ↵ | James Taylor | 2019-12-19 | 3 | -38/+37 | |
| | | | | | | | | | | | | internal helper function names Move get_captions_url in watch_extraction to bottom next to other exported, public functions | |||||
| * | Extraction: Move html post processing stuff from yt_data_extract to util | James Taylor | 2019-12-19 | 9 | -52/+50 | |
| | | ||||||
| * | Extraction: Split yt_data_extract.py into multiple files | James Taylor | 2019-12-19 | 5 | -1190/+1188 | |
| | | ||||||
| * | Extraction: Rewrite comment extraction, remove author_id and rename ↵ | James Taylor | 2019-12-19 | 3 | -80/+67 | |
| | | | | | | | | | | | | | | | | | | author_channel_id to that, fix bug in extract_items author_id (an internal sql-like integer previously required for deleting and editing comments) has been removed by Youtube and is no longer required. Remove it for simplicity. Rename author_channel_id to author_id for consistency with other extraction attributes. extract_items returned None for items instead of [] for empty continuation responses. Fixes that. | |||||
| * | Extraction: Adjust related videos box to fit new time_published information well | James Taylor | 2019-12-19 | 1 | -8/+8 | |
| | | | | | | | | | | | | time_published will be put to the right of the view_count in related videos Author will now always be above the other stats, since it doesn't make a difference in the big search result boxes since the description snippet is always very short (However, it's important the author isn't inline with the other stats in related video boxes since those are so narrow and the author name can be very long) | |||||
| * | Extraction: Use accessibility data to get timestamp and to get views for ↵ | James Taylor | 2019-12-18 | 1 | -0/+10 | |
| | | | | | | | | recommended videos | |||||
| * | Extraction: rename multi_get functions to more descriptive names | James Taylor | 2019-12-18 | 3 | -68/+68 | |
| | | ||||||
| * | Extraction: Rewrite item_extraction for better error handling and ↵ | James Taylor | 2019-12-18 | 12 | -340/+305 | |
| | | | | | | | | readability, rename extracted names for more consistency | |||||
| * | Extraction: Fix thumbnail and remove badges on related videos | James Taylor | 2019-12-17 | 2 | -4/+10 | |
| | | ||||||
| * | Extraction: Fix mistake with age-restriction detection | James Taylor | 2019-12-17 | 1 | -1/+1 | |
| | | ||||||
| * | Extraction: Detect limited state and fix false detection as unlisted | James Taylor | 2019-12-17 | 3 | -2/+14 | |
| | | ||||||
| * | Extraction: Make limited state videos work | James Taylor | 2019-12-17 | 1 | -1/+1 | |
| | | ||||||
| * | Extraction: Extract info from microformat to get views for limited state ↵ | James Taylor | 2019-12-17 | 1 | -39/+60 | |
| | | | | | | | | videos, and as a fallback. Shorten some function names | |||||
| * | Extraction: Add fallback playability error extraction from renderers | James Taylor | 2019-12-14 | 1 | -17/+24 | |
| | | ||||||
| * | Extraction: Fix subtitles error when video has no automatic captions but has ↵ | James Taylor | 2019-12-14 | 1 | -1/+5 | |
| | | | | | | | | foreign language captions | |||||
| * | Extraction: Fix subtitles not working on certain videos which require more ↵ | James Taylor | 2019-12-14 | 1 | -5/+15 | |
| | | | | | | | | parameters in the captions url | |||||
| * | Extraction: Display that video is age-restricted | James Taylor | 2019-12-12 | 1 | -7/+21 | |
| | | ||||||
| * | Extraction: Bypass age-restriction | James Taylor | 2019-12-12 | 2 | -35/+90 | |
| | | ||||||
| * | Extraction: Add general subtitle extraction and translation | James Taylor | 2019-11-29 | 2 | -69/+126 | |
| | | ||||||
| * | Extraction: extract automatic captions | James Taylor | 2019-11-28 | 1 | -2/+32 | |
| | | ||||||
| * | Extraction: extract fields from visible webpage if missing from playerResposne | James Taylor | 2019-11-25 | 1 | -31/+61 | |
| | | ||||||
| * | Extraction: return and display any errors preventing video playback | James Taylor | 2019-11-22 | 3 | -21/+40 | |
| | | ||||||
| * | Extraction: Add signature decryption | James Taylor | 2019-10-18 | 2 | -84/+193 | |
| | | ||||||
| * | Extraction: Replace youtube-dl with custom-built watch page extraction | James Taylor | 2019-10-17 | 4 | -96/+523 | |
| | | ||||||
| * | Extraction: Fix did_you_mean and showing_results_for | James Taylor | 2019-09-27 | 3 | -14/+18 | |
| | | ||||||
| * | Extraction: Rename get_response to extract_response and check that returned ↵ | James Taylor | 2019-09-27 | 1 | -20/+11 | |
| | | | | | | | | type is dict |