diff options
56 files changed, 1569 insertions, 4500 deletions
diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml deleted file mode 100644 index 04de08708..000000000 --- a/.github/FUNDING.yml +++ /dev/null @@ -1,13 +0,0 @@ -# These are supported funding model platforms - -github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] -patreon: # Replace with a single Patreon username -open_collective: # Replace with a single Open Collective username -ko_fi: # Replace with a single Ko-fi username -tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel -community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry -liberapay: # Replace with a single Liberapay username -issuehunt: # Replace with a single IssueHunt username -otechie: # Replace with a single Otechie username - -custom: ['https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators'] diff --git a/.github/ISSUE_TEMPLATE/1_broken_site.yml b/.github/ISSUE_TEMPLATE/1_broken_site.yml deleted file mode 100644 index f8f5ab1ca..000000000 --- a/.github/ISSUE_TEMPLATE/1_broken_site.yml +++ /dev/null @@ -1,63 +0,0 @@ -name: Broken site -description: Report broken or misfunctioning site -labels: [triage, site-bug] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a broken site - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your issue in an arbitrary form. - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp **that clearly demonstrates the problem**. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2022.06.22.1 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2022.06.22.1) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/2_site_support_request.yml b/.github/ISSUE_TEMPLATE/2_site_support_request.yml deleted file mode 100644 index 15fd1b471..000000000 --- a/.github/ISSUE_TEMPLATE/2_site_support_request.yml +++ /dev/null @@ -1,74 +0,0 @@ -name: Site support request -description: Request support for a new site -labels: [triage, site-request] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a new site support request - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: example-urls - attributes: - label: Example URLs - description: | - Provide all kinds of example URLs for which support should be added - placeholder: | - - Single video: https://www.youtube.com/watch?v=BaW_jenozKc - - Single video: https://youtu.be/BaW_jenozKc - - Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc - validations: - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide any additional information - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output **using one of the example URLs provided above**. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2022.06.22.1 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2022.06.22.1) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/3_site_feature_request.yml b/.github/ISSUE_TEMPLATE/3_site_feature_request.yml deleted file mode 100644 index 936db669c..000000000 --- a/.github/ISSUE_TEMPLATE/3_site_feature_request.yml +++ /dev/null @@ -1,72 +0,0 @@ -name: Site feature request -description: Request a new functionality for a supported site -labels: [triage, site-enhancement] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm requesting a site-specific feature - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: example-urls - attributes: - label: Example URLs - description: | - Example URLs that can be used to demonstrate the requested feature - placeholder: | - https://www.youtube.com/watch?v=BaW_jenozKc - validations: - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your site feature request in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp that demonstrates the need for the enhancement. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2022.06.22.1 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2022.06.22.1) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/4_bug_report.yml b/.github/ISSUE_TEMPLATE/4_bug_report.yml deleted file mode 100644 index 46743445e..000000000 --- a/.github/ISSUE_TEMPLATE/4_bug_report.yml +++ /dev/null @@ -1,57 +0,0 @@ -name: Bug report -description: Report a bug unrelated to any particular site or extractor -labels: [triage, bug] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a bug unrelated to a specific site - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your issue in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp **that clearly demonstrates the problem**. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2022.06.22.1 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2022.06.22.1) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE/5_feature_request.yml b/.github/ISSUE_TEMPLATE/5_feature_request.yml deleted file mode 100644 index 88901ab44..000000000 --- a/.github/ISSUE_TEMPLATE/5_feature_request.yml +++ /dev/null @@ -1,53 +0,0 @@ -name: Feature request -description: Request a new functionality unrelated to any particular site or extractor -labels: [triage, enhancement] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm requesting a feature unrelated to a specific site - required: true - - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your site feature request in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - If your feature request involves an existing yt-dlp command, provide the complete verbose output of that command. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2021.12.01 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2021.12.01) - <more lines> - render: shell diff --git a/.github/ISSUE_TEMPLATE/6_question.yml b/.github/ISSUE_TEMPLATE/6_question.yml deleted file mode 100644 index 6661c8b11..000000000 --- a/.github/ISSUE_TEMPLATE/6_question.yml +++ /dev/null @@ -1,55 +0,0 @@ -name: Ask question -description: Ask yt-dlp related question -labels: [question] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm asking a question and **not** reporting a bug or requesting a feature - required: true - - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - required: true - - label: I've verified that I'm running yt-dlp version **2022.06.22.1** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: question - attributes: - label: Question - description: | - Ask your question in an arbitrary form. - Please make sure it's worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information and as much context and examples as possible. - If your question contains "isn't working" or "can you add", this is most likely the wrong template. - If you are in doubt if this is the right template, use another template! - placeholder: WRITE QUESTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - If your question involves a yt-dlp command, provide the complete verbose output of that command. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2021.12.01 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2021.12.01) - <more lines> - render: shell diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml deleted file mode 100644 index 9cdffa4b1..000000000 --- a/.github/ISSUE_TEMPLATE/config.yml +++ /dev/null @@ -1,8 +0,0 @@ -blank_issues_enabled: false -contact_links: - - name: Get help from the community on Discord - url: https://discord.gg/H5MNcFW63r - about: Join the yt-dlp Discord for community-powered support! - - name: Matrix Bridge to the Discord server - url: https://matrix.to/#/#yt-dlp:matrix.org - about: For those who do not want to use Discord diff --git a/.github/ISSUE_TEMPLATE_tmpl/1_broken_site.yml b/.github/ISSUE_TEMPLATE_tmpl/1_broken_site.yml deleted file mode 100644 index 3d7f9d04e..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/1_broken_site.yml +++ /dev/null @@ -1,63 +0,0 @@ -name: Broken site -description: Report broken or misfunctioning site -labels: [triage, site-bug] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a broken site - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your issue in an arbitrary form. - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp **that clearly demonstrates the problem**. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version %(version)s (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (%(version)s) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.yml b/.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.yml deleted file mode 100644 index fc7306b61..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.yml +++ /dev/null @@ -1,74 +0,0 @@ -name: Site support request -description: Request support for a new site -labels: [triage, site-request] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a new site support request - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that none of provided URLs [violate any copyrights](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) or contain any [DRM](https://en.wikipedia.org/wiki/Digital_rights_management) to the best of my knowledge - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and am willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: example-urls - attributes: - label: Example URLs - description: | - Provide all kinds of example URLs for which support should be added - placeholder: | - - Single video: https://www.youtube.com/watch?v=BaW_jenozKc - - Single video: https://youtu.be/BaW_jenozKc - - Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc - validations: - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide any additional information - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output **using one of the example URLs provided above**. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version %(version)s (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (%(version)s) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.yml b/.github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.yml deleted file mode 100644 index 77e9d3469..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.yml +++ /dev/null @@ -1,72 +0,0 @@ -name: Site feature request -description: Request a new functionality for a supported site -labels: [triage, site-enhancement] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm requesting a site-specific feature - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - label: I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required - - type: input - id: region - attributes: - label: Region - description: "Enter the region the site is accessible from" - placeholder: "India" - - type: textarea - id: example-urls - attributes: - label: Example URLs - description: | - Example URLs that can be used to demonstrate the requested feature - placeholder: | - https://www.youtube.com/watch?v=BaW_jenozKc - validations: - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your site feature request in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp that demonstrates the need for the enhancement. - Add the `-vU` flag to your command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version %(version)s (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (%(version)s) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml b/.github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml deleted file mode 100644 index bb9d94c33..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml +++ /dev/null @@ -1,57 +0,0 @@ -name: Bug report -description: Report a bug unrelated to any particular site or extractor -labels: [triage, bug] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm reporting a bug unrelated to a specific site - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've checked that all provided URLs are playable in a browser with the same IP and same login details - required: true - - label: I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/ytdl-org/youtube-dl#video-url-contains-an-ampersand-and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your issue in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - Provide the complete verbose output of yt-dlp **that clearly demonstrates the problem**. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version %(version)s (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (%(version)s) - <more lines> - render: shell - validations: - required: true diff --git a/.github/ISSUE_TEMPLATE_tmpl/5_feature_request.yml b/.github/ISSUE_TEMPLATE_tmpl/5_feature_request.yml deleted file mode 100644 index 4686c1dff..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/5_feature_request.yml +++ /dev/null @@ -1,53 +0,0 @@ -name: Feature request -description: Request a new functionality unrelated to any particular site or extractor -labels: [triage, enhancement] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm requesting a feature unrelated to a specific site - required: true - - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: description - attributes: - label: Description - description: | - Provide an explanation of your site feature request in an arbitrary form. - Please make sure the description is worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information, any suggested solutions, and as much context and examples as possible - placeholder: WRITE DESCRIPTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - If your feature request involves an existing yt-dlp command, provide the complete verbose output of that command. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2021.12.01 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2021.12.01) - <more lines> - render: shell diff --git a/.github/ISSUE_TEMPLATE_tmpl/6_question.yml b/.github/ISSUE_TEMPLATE_tmpl/6_question.yml deleted file mode 100644 index 8936f096b..000000000 --- a/.github/ISSUE_TEMPLATE_tmpl/6_question.yml +++ /dev/null @@ -1,55 +0,0 @@ -name: Ask question -description: Ask yt-dlp related question -labels: [question] -body: - - type: checkboxes - id: checklist - attributes: - label: Checklist - description: | - Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp: - options: - - label: I'm asking a question and **not** reporting a bug or requesting a feature - required: true - - label: I've looked through the [README](https://github.com/yt-dlp/yt-dlp#readme) - required: true - - label: I've verified that I'm running yt-dlp version **%(version)s** ([update instructions](https://github.com/yt-dlp/yt-dlp#update)) or later (specify commit) - required: true - - label: I've searched the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar questions including closed ones. DO NOT post duplicates - required: true - - label: I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue) - required: true - - type: textarea - id: question - attributes: - label: Question - description: | - Ask your question in an arbitrary form. - Please make sure it's worded well enough to be understood, see [is-the-description-of-the-issue-itself-sufficient](https://github.com/ytdl-org/youtube-dl#is-the-description-of-the-issue-itself-sufficient). - Provide any additional information and as much context and examples as possible. - If your question contains "isn't working" or "can you add", this is most likely the wrong template. - If you are in doubt if this is the right template, use another template! - placeholder: WRITE QUESTION HERE - validations: - required: true - - type: textarea - id: log - attributes: - label: Verbose log - description: | - If your question involves a yt-dlp command, provide the complete verbose output of that command. - Add the `-vU` flag to **your** command line you run yt-dlp with (`yt-dlp -vU <your command line>`), copy the WHOLE output and insert it below. - It should look similar to this: - placeholder: | - [debug] Command-line config: ['-vU', 'http://www.youtube.com/watch?v=BaW_jenozKc'] - [debug] Portable config file: yt-dlp.conf - [debug] Portable config: ['-i'] - [debug] Encodings: locale cp1252, fs utf-8, stdout utf-8, stderr utf-8, pref cp1252 - [debug] yt-dlp version 2021.12.01 (exe) - [debug] Python version 3.8.8 (CPython 64bit) - Windows-10-10.0.19041-SP0 - [debug] exe versions: ffmpeg 3.0.1, ffprobe 3.0.1 - [debug] Optional libraries: Cryptodome, keyring, mutagen, sqlite, websockets - [debug] Proxy map: {} - yt-dlp is up to date (2021.12.01) - <more lines> - render: shell diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index 14d4da52e..000000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,29 +0,0 @@ -<!-- -# Please follow the guide below - -- You will be asked some questions, please read them **carefully** and answer honestly -- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x]) -- Use *Preview* tab to see how your *pull request* will actually look like - ---> - -### Before submitting a *pull request* make sure you have: -- [ ] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions) -- [ ] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests -- [ ] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) - -### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check one of the following options: -- [ ] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/) -- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence) - -### What is the purpose of your *pull request*? -- [ ] Fix or improvement to an extractor (Make sure to add/update tests) -- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy)) -- [ ] Core bug fix/improvement -- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes)) - ---- - -### Description of your *pull request* and other information - -Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible. diff --git a/.github/banner.svg b/.github/banner.svg deleted file mode 100644 index 35dc93eae..000000000 --- a/.github/banner.svg +++ /dev/null @@ -1,31 +0,0 @@ -<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid" width="699.935" height="173.764" viewBox="0 0 717 178"> - <defs> - <style> - .cls-1, .cls-4 { - fill: red; - } - - .cls-1, .cls-2, .cls-3, .cls-4 { - fill-rule: evenodd; - } - - .cls-2 { - fill: #666; - } - - .cls-3 { - fill: #fff; - } - - .cls-3, .cls-4 { - stroke: #282828; - stroke-linejoin: round; - stroke-width: 1px; - } - </style> - </defs> - <path d="M89.846,166.601 L87.111,166.601 L87.111,172.000 L82.173,172.000 L82.173,153.812 L90.024,153.812 C94.064,153.812 96.773,156.370 96.773,160.242 C96.773,164.158 93.993,166.601 89.846,166.601 ZM88.851,157.755 L87.111,157.755 L87.111,162.764 L88.851,162.764 C90.583,162.764 91.622,161.796 91.622,160.242 C91.622,158.679 90.583,157.755 88.851,157.755 ZM67.898,153.812 L72.835,153.812 L72.835,168.021 L80.189,168.021 L80.189,172.000 L67.898,172.000 L67.898,153.812 ZM56.572,172.000 L49.574,172.000 L49.574,153.812 L56.501,153.812 C62.113,153.812 65.630,157.223 65.630,162.906 C65.630,168.590 62.113,172.000 56.572,172.000 ZM56.252,158.004 L54.511,158.004 L54.511,167.808 L56.394,167.808 C59.094,167.808 60.657,166.707 60.657,162.906 C60.657,159.105 59.094,158.004 56.252,158.004 ZM38.211,162.906 L46.736,162.906 L46.736,166.601 L38.211,166.601 L38.211,162.906 ZM31.253,172.000 L26.387,172.000 L26.387,157.791 L20.916,157.791 L20.916,153.812 L36.724,153.812 L36.724,157.791 L31.253,157.791 L31.253,172.000 ZM12.007,172.000 L7.104,172.000 L7.104,166.281 L0.426,153.812 L5.932,153.812 L9.484,161.201 L9.627,161.201 L13.179,153.812 L18.685,153.812 L12.007,166.281 L12.007,172.000 Z" class="cls-1"/> - <path d="M714.317,161.947 C714.104,160.988 713.536,159.993 711.689,159.993 C710.019,159.993 708.634,160.846 708.456,162.018 C708.278,163.048 708.918,163.617 710.445,164.007 L712.399,164.505 C714.743,165.109 715.738,166.281 715.418,168.199 C715.028,170.544 712.577,172.284 709.415,172.284 C706.609,172.284 704.904,171.041 704.797,168.732 L706.893,168.235 C707.000,169.691 707.959,170.437 709.664,170.437 C711.617,170.437 713.038,169.478 713.216,168.306 C713.394,167.347 712.861,166.707 711.511,166.387 L709.344,165.855 C706.928,165.251 706.005,164.007 706.325,162.125 C706.715,159.816 709.131,158.182 712.008,158.182 C714.708,158.182 715.951,159.461 716.306,161.414 L714.317,161.947 ZM702.671,165.890 L692.751,165.890 C692.245,169.229 693.648,170.401 696.276,170.401 C697.955,170.401 699.269,169.691 700.042,168.270 L701.960,168.838 C700.974,170.899 698.736,172.284 695.957,172.284 C692.023,172.284 690.069,169.478 690.770,165.286 C691.454,161.095 694.403,158.182 698.088,158.182 C700.939,158.182 703.674,159.922 702.813,165.002 L702.671,165.890 ZM697.768,160.064 C695.477,160.064 693.461,162.143 693.044,164.078 L700.823,164.078 C701.223,161.770 700.051,160.064 697.768,160.064 ZM687.862,172.000 L685.446,172.000 L683.066,166.707 L678.910,172.000 L676.494,172.000 L681.965,165.180 L678.768,158.359 L681.183,158.359 L683.528,163.936 L687.720,158.359 L690.135,158.359 L684.594,165.180 L687.862,172.000 ZM673.886,154.630 C673.886,153.848 674.560,153.209 675.377,153.209 C676.194,153.209 676.869,153.848 676.869,154.630 C676.869,155.411 676.194,156.050 675.377,156.050 C674.560,156.050 673.886,155.411 673.886,154.630 ZM673.513,172.000 L671.417,172.000 L673.690,158.359 L675.786,158.359 L673.513,172.000 ZM670.212,154.914 C668.826,154.914 668.151,155.624 667.903,156.974 L667.672,158.359 L670.745,158.359 L670.460,160.135 L667.379,160.135 L665.416,172.000 L663.320,172.000 L665.301,160.135 L663.107,160.135 L663.391,158.359 L665.603,158.359 L665.914,156.477 C666.269,154.132 668.365,152.960 670.318,152.960 C671.348,152.960 671.952,153.173 672.237,153.315 L671.348,155.127 C671.135,155.056 670.816,154.914 670.212,154.914 ZM649.225,172.000 L649.580,169.904 L649.332,169.904 C648.745,170.650 647.582,172.284 644.962,172.284 C641.543,172.284 639.616,169.549 640.327,165.215 C641.046,160.917 643.879,158.182 647.324,158.182 C649.989,158.182 650.539,159.816 650.877,160.526 L651.054,160.526 L652.173,153.812 L654.269,153.812 L651.250,172.000 L649.225,172.000 ZM647.182,160.064 C644.527,160.064 642.911,162.302 642.440,165.180 C641.952,168.093 642.849,170.401 645.477,170.401 C647.999,170.401 649.811,168.270 650.326,165.180 C650.832,162.125 649.749,160.064 647.182,160.064 ZM635.980,172.000 L633.884,172.000 L635.305,163.475 C635.660,161.343 634.701,160.064 632.747,160.064 C630.723,160.064 629.053,161.414 628.627,163.794 L627.277,172.000 L625.181,172.000 L627.454,158.359 L629.479,158.359 L629.124,160.491 L629.302,160.491 C630.154,159.105 631.611,158.182 633.671,158.182 C636.406,158.182 638.005,159.851 637.436,163.333 L635.980,172.000 ZM621.349,172.000 L619.253,172.000 L619.573,170.153 L619.466,170.153 C618.898,171.041 617.442,172.320 615.062,172.320 C612.468,172.320 610.657,170.792 611.083,168.128 C611.616,165.002 614.458,164.434 617.051,164.114 C619.573,163.794 620.603,163.865 620.781,162.871 L620.781,162.800 C621.065,161.059 620.354,160.029 618.436,160.029 C616.447,160.029 615.097,161.095 614.458,162.089 L612.611,161.379 C614.067,158.892 616.554,158.182 618.614,158.182 C620.354,158.182 623.551,158.679 622.841,163.013 L621.349,172.000 ZM616.660,165.926 C614.991,166.139 613.428,166.636 613.179,168.235 C612.930,169.691 613.996,170.437 615.665,170.437 C618.152,170.437 619.786,168.767 620.070,167.062 L620.390,165.144 C619.964,165.570 617.548,165.819 616.660,165.926 ZM597.804,159.993 C596.135,159.993 594.749,160.846 594.572,162.018 C594.394,163.048 595.033,163.617 596.561,164.007 L598.515,164.505 C600.859,165.109 601.854,166.281 601.534,168.199 C601.143,170.544 598.692,172.284 595.531,172.284 C592.724,172.284 591.019,171.041 590.913,168.732 L593.009,168.235 C593.115,169.691 594.074,170.437 595.779,170.437 C597.733,170.437 599.154,169.478 599.332,168.306 C599.509,167.347 598.976,166.707 597.627,166.387 L595.460,165.855 C593.044,165.251 592.121,164.007 592.440,162.125 C592.831,159.816 595.247,158.182 598.124,158.182 C600.824,158.182 602.067,159.461 602.422,161.414 L600.433,161.947 C600.220,160.988 599.651,159.993 597.804,159.993 ZM588.786,165.890 L578.866,165.890 C578.360,169.229 579.763,170.401 582.392,170.401 C584.071,170.401 585.385,169.691 586.157,168.270 L588.076,168.838 C587.090,170.899 584.852,172.284 582.072,172.284 C578.138,172.284 576.185,169.478 576.886,165.286 C577.570,161.095 580.518,158.182 584.204,158.182 C587.054,158.182 589.790,159.922 588.928,165.002 L588.786,165.890 ZM583.884,160.064 C581.593,160.064 579.577,162.143 579.160,164.078 L586.939,164.078 C587.339,161.770 586.166,160.064 583.884,160.064 ZM574.722,160.171 C572.733,160.171 571.046,161.530 570.744,163.368 L569.323,172.000 L567.227,172.000 L569.500,158.359 L571.525,158.359 L571.170,160.420 L571.312,160.420 C572.023,159.070 573.586,158.146 575.255,158.146 C576.001,158.146 576.534,158.324 576.889,158.644 L575.894,160.384 C575.646,160.242 575.255,160.171 574.722,160.171 ZM561.299,172.000 L561.690,169.691 L561.548,169.691 C560.695,171.076 559.132,172.178 557.072,172.178 C554.515,172.178 552.952,170.508 553.520,167.027 L554.976,158.359 L557.072,158.359 L555.651,166.885 C555.332,168.874 556.362,170.153 558.102,170.153 C559.665,170.153 561.797,168.981 562.223,166.423 L563.573,158.359 L565.669,158.359 L563.395,172.000 L561.299,172.000 ZM551.534,160.135 L548.594,160.135 L547.271,168.093 C546.987,169.869 547.839,170.153 548.763,170.153 C549.225,170.153 549.509,170.082 549.686,170.046 L549.829,171.929 C549.509,172.036 548.976,172.178 548.195,172.178 C546.418,172.178 544.713,171.041 545.104,168.661 L546.507,160.135 L544.465,160.135 L544.749,158.359 L546.800,158.359 L547.342,155.091 L549.438,155.091 L548.896,158.359 L551.818,158.359 L551.534,160.135 ZM539.780,172.000 L537.684,172.000 L538.004,170.153 L537.897,170.153 C537.329,171.041 535.873,172.320 533.493,172.320 C530.900,172.320 529.088,170.792 529.514,168.128 C530.047,165.002 532.889,164.434 535.482,164.114 C538.004,163.794 539.034,163.865 539.212,162.871 L539.212,162.800 C539.496,161.059 538.786,160.029 536.867,160.029 C534.878,160.029 533.528,161.095 532.889,162.089 L531.042,161.379 C532.498,158.892 534.985,158.182 537.045,158.182 C538.786,158.182 541.983,158.679 541.272,163.013 L539.780,172.000 ZM535.091,165.926 C533.422,166.139 531.859,166.636 531.610,168.235 C531.361,169.691 532.427,170.437 534.097,170.437 C536.583,170.437 538.217,168.767 538.501,167.062 L538.821,165.144 C538.395,165.570 535.979,165.819 535.091,165.926 ZM527.316,165.890 L517.397,165.890 C516.891,169.229 518.294,170.401 520.922,170.401 C522.601,170.401 523.915,169.691 524.688,168.270 L526.606,168.838 C525.620,170.899 523.382,172.284 520.603,172.284 C516.669,172.284 514.715,169.478 515.416,165.286 C516.100,161.095 519.049,158.182 522.734,158.182 C525.585,158.182 528.320,159.922 527.459,165.002 L527.316,165.890 ZM522.414,160.064 C520.123,160.064 518.107,162.143 517.690,164.078 L525.469,164.078 C525.869,161.770 524.697,160.064 522.414,160.064 ZM514.282,154.914 C512.897,154.914 512.222,155.624 511.973,156.974 L511.742,158.359 L514.815,158.359 L514.531,160.135 L511.449,160.135 L509.487,172.000 L507.391,172.000 L509.371,160.135 L507.178,160.135 L507.462,158.359 L509.673,158.359 L509.984,156.477 C510.339,154.132 512.435,152.960 514.389,152.960 C515.419,152.960 516.023,153.173 516.307,153.315 L515.419,155.127 C515.206,155.056 514.886,154.914 514.282,154.914 ZM493.506,172.000 L496.525,153.812 L498.621,153.812 L495.601,172.000 L493.506,172.000 ZM489.674,172.000 L487.578,172.000 L487.898,170.153 L487.791,170.153 C487.223,171.041 485.766,172.320 483.386,172.320 C480.793,172.320 478.981,170.792 479.408,168.128 C479.941,165.002 482.782,164.434 485.375,164.114 C487.898,163.794 488.928,163.865 489.105,162.871 L489.105,162.800 C489.390,161.059 488.679,160.029 486.761,160.029 C484.772,160.029 483.422,161.095 482.782,162.089 L480.935,161.379 C482.392,158.892 484.878,158.182 486.938,158.182 C488.679,158.182 491.876,158.679 491.166,163.013 L489.674,172.000 ZM484.985,165.926 C483.315,166.139 481.752,166.636 481.504,168.235 C481.255,169.691 482.321,170.437 483.990,170.437 C486.477,170.437 488.111,168.767 488.395,167.062 L488.715,165.144 C488.288,165.570 485.873,165.819 484.985,165.926 ZM475.576,172.000 L473.480,172.000 L474.901,163.475 C475.256,161.343 474.297,160.064 472.343,160.064 C470.319,160.064 468.649,161.414 468.223,163.794 L466.873,172.000 L464.777,172.000 L467.051,158.359 L469.075,158.359 L468.720,160.491 L468.898,160.491 C469.750,159.105 471.207,158.182 473.267,158.182 C476.002,158.182 477.601,159.851 477.032,163.333 L475.576,172.000 ZM455.511,172.284 C451.745,172.284 449.703,169.407 450.395,165.109 C451.070,160.917 453.948,158.182 457.571,158.182 C461.336,158.182 463.388,161.059 462.686,165.393 C462.011,169.549 459.134,172.284 455.511,172.284 ZM457.535,160.064 C454.658,160.064 452.873,162.587 452.420,165.393 C451.994,168.057 452.811,170.401 455.546,170.401 C458.423,170.401 460.208,167.924 460.661,165.109 C461.088,162.444 460.271,160.064 457.535,160.064 ZM446.401,154.630 C446.401,153.848 447.076,153.209 447.893,153.209 C448.710,153.209 449.385,153.848 449.385,154.630 C449.385,155.411 448.710,156.050 447.893,156.050 C447.076,156.050 446.401,155.411 446.401,154.630 ZM446.028,172.000 L443.932,172.000 L446.206,158.359 L448.301,158.359 L446.028,172.000 ZM442.763,160.135 L439.823,160.135 L438.500,168.093 C438.216,169.869 439.069,170.153 439.992,170.153 C440.454,170.153 440.738,170.082 440.916,170.046 L441.058,171.929 C440.738,172.036 440.205,172.178 439.424,172.178 C437.648,172.178 435.943,171.041 436.333,168.661 L437.736,160.135 L435.694,160.135 L435.978,158.359 L438.030,158.359 L438.571,155.091 L440.667,155.091 L440.125,158.359 L443.047,158.359 L442.763,160.135 ZM431.380,154.630 C431.380,153.848 432.055,153.209 432.872,153.209 C433.689,153.209 434.364,153.848 434.364,154.630 C434.364,155.411 433.689,156.050 432.872,156.050 C432.055,156.050 431.380,155.411 431.380,154.630 ZM431.007,172.000 L428.911,172.000 L431.184,158.359 L433.280,158.359 L431.007,172.000 ZM422.770,172.000 L423.126,169.904 L422.877,169.904 C422.291,170.650 421.128,172.284 418.508,172.284 C415.089,172.284 413.162,169.549 413.872,165.215 C414.591,160.917 417.424,158.182 420.870,158.182 C423.534,158.182 424.085,159.816 424.422,160.526 L424.600,160.526 L425.719,153.812 L427.815,153.812 L424.795,172.000 L422.770,172.000 ZM420.728,160.064 C418.073,160.064 416.456,162.302 415.986,165.180 C415.497,168.093 416.394,170.401 419.023,170.401 C421.545,170.401 423.357,168.270 423.872,165.180 C424.378,162.125 423.294,160.064 420.728,160.064 ZM407.216,172.000 L407.572,169.904 L407.323,169.904 C406.737,170.650 405.573,172.284 402.954,172.284 C399.535,172.284 397.608,169.549 398.318,165.215 C399.037,160.917 401.870,158.182 405.316,158.182 C407.980,158.182 408.531,159.816 408.868,160.526 L409.046,160.526 L410.165,153.812 L412.261,153.812 L409.241,172.000 L407.216,172.000 ZM405.174,160.064 C402.519,160.064 400.902,162.302 400.432,165.180 C399.943,168.093 400.840,170.401 403.469,170.401 C405.991,170.401 407.803,168.270 408.318,165.180 C408.824,162.125 407.740,160.064 405.174,160.064 ZM393.971,172.000 L391.875,172.000 L392.195,170.153 L392.088,170.153 C391.520,171.041 390.063,172.320 387.683,172.320 C385.090,172.320 383.279,170.792 383.705,168.128 C384.238,165.002 387.080,164.434 389.673,164.114 C392.195,163.794 393.225,163.865 393.403,162.871 L393.403,162.800 C393.687,161.059 392.976,160.029 391.058,160.029 C389.069,160.029 387.719,161.095 387.080,162.089 L385.232,161.379 C386.689,158.892 389.175,158.182 391.236,158.182 C392.976,158.182 396.173,158.679 395.463,163.013 L393.971,172.000 ZM389.282,165.926 C387.612,166.139 386.049,166.636 385.801,168.235 C385.552,169.691 386.618,170.437 388.287,170.437 C390.774,170.437 392.408,168.767 392.692,167.062 L393.012,165.144 C392.586,165.570 390.170,165.819 389.282,165.926 ZM372.842,172.000 L370.746,172.000 L372.167,163.475 C372.522,161.308 371.528,160.064 369.574,160.064 C367.513,160.064 365.773,161.414 365.347,163.794 L363.997,172.000 L361.901,172.000 L364.920,153.812 L367.016,153.812 L365.915,160.491 L366.093,160.491 C366.945,159.070 368.330,158.182 370.497,158.182 C373.268,158.182 374.867,159.816 374.298,163.333 L372.842,172.000 ZM360.448,160.135 L357.508,160.135 L356.185,168.093 C355.901,169.869 356.753,170.153 357.677,170.153 C358.139,170.153 358.423,170.082 358.601,170.046 L358.743,171.929 C358.423,172.036 357.890,172.178 357.109,172.178 C355.333,172.178 353.627,171.041 354.018,168.661 L355.421,160.135 L353.379,160.135 L353.663,158.359 L355.714,158.359 L356.256,155.091 L358.352,155.091 L357.810,158.359 L360.732,158.359 L360.448,160.135 ZM349.065,154.630 C349.065,153.848 349.740,153.209 350.557,153.209 C351.374,153.209 352.049,153.848 352.049,154.630 C352.049,155.411 351.374,156.050 350.557,156.050 C349.740,156.050 349.065,155.411 349.065,154.630 ZM348.692,172.000 L346.596,172.000 L348.869,158.359 L350.965,158.359 L348.692,172.000 ZM337.615,172.000 L336.372,161.521 L336.159,161.521 L331.434,172.000 L329.374,172.000 L327.491,158.359 L329.694,158.359 L330.901,168.803 L331.043,168.803 L335.697,158.359 L337.935,158.359 L339.072,168.767 L339.214,168.767 L343.903,158.359 L346.105,158.359 L339.675,172.000 L337.615,172.000 ZM316.983,172.000 L314.319,172.000 L310.296,165.526 L308.600,166.885 L307.747,172.000 L305.651,172.000 L308.671,153.812 L310.767,153.812 L308.999,164.434 L309.239,164.434 L316.237,158.359 L318.830,158.359 L312.090,164.203 L316.983,172.000 ZM303.559,160.171 C301.569,160.171 299.882,161.530 299.580,163.368 L298.159,172.000 L296.063,172.000 L298.337,158.359 L300.362,158.359 L300.006,160.420 L300.149,160.420 C300.859,159.070 302.422,158.146 304.091,158.146 C304.837,158.146 305.370,158.324 305.726,158.644 L304.731,160.384 C304.482,160.242 304.091,160.171 303.559,160.171 ZM286.797,172.284 C283.031,172.284 280.989,169.407 281.682,165.109 C282.356,160.917 285.234,158.182 288.857,158.182 C292.622,158.182 294.674,161.059 293.972,165.393 C293.297,169.549 290.420,172.284 286.797,172.284 ZM288.822,160.064 C285.944,160.064 284.159,162.587 283.706,165.393 C283.280,168.057 284.097,170.401 286.832,170.401 C289.710,170.401 291.495,167.924 291.948,165.109 C292.374,162.444 291.557,160.064 288.822,160.064 ZM280.512,154.914 C279.126,154.914 278.452,155.624 278.203,156.974 L277.972,158.359 L281.045,158.359 L280.760,160.135 L277.679,160.135 L275.716,172.000 L273.620,172.000 L275.601,160.135 L273.407,160.135 L273.691,158.359 L275.903,158.359 L276.214,156.477 C276.569,154.132 278.665,152.960 280.618,152.960 C281.649,152.960 282.252,153.173 282.537,153.315 L281.649,155.127 C281.435,155.056 281.116,154.914 280.512,154.914 ZM259.735,172.000 L262.755,153.812 L264.851,153.812 L261.831,172.000 L259.735,172.000 ZM253.595,172.000 L253.950,169.904 L253.701,169.904 C253.115,170.650 251.952,172.284 249.332,172.284 C245.913,172.284 243.986,169.549 244.696,165.215 C245.416,160.917 248.249,158.182 251.694,158.182 C254.358,158.182 254.909,159.816 255.246,160.526 L255.424,160.526 L256.543,153.812 L258.639,153.812 L255.619,172.000 L253.595,172.000 ZM251.552,160.064 C248.897,160.064 247.281,162.302 246.810,165.180 C246.321,168.093 247.218,170.401 249.847,170.401 C252.369,170.401 254.181,168.270 254.696,165.180 C255.202,162.125 254.119,160.064 251.552,160.064 ZM233.670,165.180 L233.990,163.226 L241.947,163.226 L241.627,165.180 L233.670,165.180 ZM230.478,165.890 L220.558,165.890 C220.052,169.229 221.455,170.401 224.084,170.401 C225.762,170.401 227.077,169.691 227.849,168.270 L229.768,168.838 C228.782,170.899 226.544,172.284 223.764,172.284 C219.830,172.284 217.876,169.478 218.578,165.286 C219.262,161.095 222.210,158.182 225.896,158.182 C228.746,158.182 231.482,159.922 230.620,165.002 L230.478,165.890 ZM225.576,160.064 C223.285,160.064 221.269,162.143 220.851,164.078 L228.631,164.078 C229.030,161.770 227.858,160.064 225.576,160.064 ZM209.063,172.284 C206.434,172.284 205.830,170.650 205.475,169.904 L205.226,169.904 L204.871,172.000 L202.846,172.000 L205.866,153.812 L207.962,153.812 L206.860,160.526 L207.038,160.526 C207.606,159.816 208.708,158.182 211.372,158.182 C214.817,158.182 216.736,160.917 216.025,165.215 C215.315,169.549 212.473,172.284 209.063,172.284 ZM210.803,160.064 C208.246,160.064 206.541,162.125 206.043,165.180 C205.546,168.270 206.576,170.401 209.098,170.401 C211.727,170.401 213.432,168.093 213.929,165.180 C214.391,162.302 213.468,160.064 210.803,160.064 ZM196.634,172.000 L197.025,169.691 L196.883,169.691 C196.031,171.076 194.468,172.178 192.407,172.178 C189.850,172.178 188.287,170.508 188.855,167.027 L190.311,158.359 L192.407,158.359 L190.986,166.885 C190.667,168.874 191.697,170.153 193.437,170.153 C195.000,170.153 197.132,168.981 197.558,166.423 L198.908,158.359 L201.004,158.359 L198.730,172.000 L196.634,172.000 ZM186.869,160.135 L183.929,160.135 L182.606,168.093 C182.322,169.869 183.174,170.153 184.098,170.153 C184.560,170.153 184.844,170.082 185.022,170.046 L185.164,171.929 C184.844,172.036 184.311,172.178 183.530,172.178 C181.754,172.178 180.048,171.041 180.439,168.661 L181.842,160.135 L179.800,160.135 L180.084,158.359 L182.135,158.359 L182.677,155.091 L184.773,155.091 L184.231,158.359 L187.153,158.359 L186.869,160.135 ZM173.020,172.000 L173.410,169.691 L173.268,169.691 C172.416,171.076 170.853,172.178 168.792,172.178 C166.235,172.178 164.672,170.508 165.240,167.027 L166.697,158.359 L168.792,158.359 L167.372,166.885 C167.052,168.874 168.082,170.153 169.823,170.153 C171.386,170.153 173.517,168.981 173.943,166.423 L175.293,158.359 L177.389,158.359 L175.115,172.000 L173.020,172.000 ZM155.157,172.284 C151.391,172.284 149.349,169.407 150.041,165.109 C150.716,160.917 153.594,158.182 157.217,158.182 C160.982,158.182 163.034,161.059 162.332,165.393 C161.657,169.549 158.780,172.284 155.157,172.284 ZM157.181,160.064 C154.304,160.064 152.519,162.587 152.066,165.393 C151.640,168.057 152.457,170.401 155.192,170.401 C158.069,170.401 159.854,167.924 160.307,165.109 C160.734,162.444 159.917,160.064 157.181,160.064 ZM136.511,177.293 C135.801,177.293 135.197,177.151 135.019,177.044 L135.836,175.197 C137.293,175.588 138.207,175.366 139.317,173.350 L140.063,172.000 L137.293,158.359 L139.495,158.359 L141.449,169.229 L141.591,169.229 L147.168,158.359 L149.512,158.359 L141.023,174.202 C139.886,176.298 138.429,177.293 136.511,177.293 ZM123.225,166.849 L115.871,166.849 L113.181,172.000 L110.872,172.000 L120.569,153.812 L122.843,153.812 L126.501,172.000 L124.192,172.000 L123.225,166.849 ZM121.315,156.690 L121.173,156.690 L116.893,164.895 L122.860,164.895 L121.315,156.690 Z" class="cls-2"/> - <path d="M252.245,116.350 L252.245,102.200 L309.303,102.200 L309.303,116.350 L252.245,116.350 ZM208.254,81.088 L245.342,59.291 L208.254,38.180 L216.242,25.227 L260.862,52.844 L260.862,65.739 L216.413,93.355 L208.254,81.088 Z" class="cls-3"/> - <path d="M508.108,52.635 C507.921,55.093 507.643,57.527 507.274,59.937 L504.214,78.017 C503.658,81.170 502.754,84.324 501.502,87.475 C500.250,90.628 498.464,93.479 496.147,96.028 C493.829,98.579 491.047,100.503 487.802,101.800 C484.556,103.097 481.311,103.747 478.067,103.747 C476.211,103.747 474.357,103.491 472.504,102.982 C470.648,102.474 469.072,101.615 467.775,100.409 C466.475,99.205 465.410,97.767 464.576,96.098 C463.741,94.429 463.092,92.714 462.629,90.952 L455.953,131.146 L436.482,131.146 L453.310,28.922 L472.921,28.922 L471.391,38.240 C472.504,36.665 473.777,35.180 475.216,33.790 C476.652,32.399 478.228,31.240 479.944,30.313 C481.659,29.387 483.467,28.737 485.369,28.365 C487.268,27.996 489.145,27.809 491.001,27.809 C493.411,27.809 495.706,28.226 497.886,29.061 C500.063,29.895 501.871,31.171 503.310,32.886 C504.746,34.602 505.835,36.549 506.578,38.727 C507.319,40.907 507.806,43.156 508.039,45.472 C508.269,47.791 508.293,50.179 508.108,52.635 ZM487.455,48.184 C486.851,46.841 485.877,45.798 484.534,45.055 C483.189,44.314 481.729,43.942 480.153,43.942 C478.762,43.942 477.393,44.151 476.050,44.568 C474.705,44.986 473.499,45.681 472.434,46.655 C471.367,47.628 470.556,48.765 470.000,50.062 C469.444,51.362 469.027,52.659 468.748,53.956 L465.828,72.037 C465.641,73.149 465.480,74.286 465.341,75.444 C465.202,76.605 465.178,77.717 465.271,78.782 C465.363,79.849 465.526,80.916 465.758,81.981 C465.988,83.048 466.384,84.022 466.940,84.902 C467.497,85.784 468.283,86.456 469.305,86.918 C470.324,87.383 471.391,87.614 472.504,87.614 C474.079,87.614 475.633,87.314 477.163,86.710 C478.693,86.108 480.036,85.204 481.196,83.998 C482.354,82.794 483.235,81.425 483.839,79.895 C484.441,78.365 484.882,76.859 485.160,75.375 L488.081,57.294 C488.359,55.719 488.474,54.143 488.428,52.565 C488.381,50.990 488.057,49.530 487.455,48.184 ZM433.422,86.501 L433.422,102.635 L384.744,102.635 L384.744,86.501 L401.433,86.501 L412.977,16.544 L400.460,16.544 L400.460,0.410 L435.230,0.410 L420.905,86.501 L433.422,86.501 ZM354.285,102.635 L355.815,93.177 C354.793,94.846 353.542,96.354 352.060,97.697 C350.575,99.042 348.976,100.179 347.261,101.105 C345.545,102.033 343.736,102.704 341.837,103.121 C339.936,103.539 338.058,103.747 336.204,103.747 C333.792,103.747 331.497,103.330 329.320,102.495 C327.140,101.661 325.332,100.388 323.896,98.671 C322.457,96.956 321.368,94.985 320.627,92.760 C319.884,90.535 319.397,88.264 319.167,85.945 C318.934,83.628 318.911,81.264 319.097,78.852 C319.282,76.442 319.606,74.032 320.071,71.620 L322.992,53.539 C323.548,50.295 324.452,47.096 325.704,43.942 C326.955,40.791 328.764,37.962 331.128,35.459 C333.492,32.955 336.274,31.056 339.473,29.756 C342.672,28.459 345.892,27.809 349.139,27.809 C351.086,27.809 352.964,28.066 354.772,28.574 C356.580,29.085 358.155,29.943 359.500,31.147 C360.843,32.353 361.934,33.790 362.769,35.459 C363.603,37.128 364.205,38.844 364.577,40.605 L371.253,0.410 L390.724,0.410 L373.895,102.635 L354.285,102.635 ZM362.004,52.705 C361.956,51.592 361.795,50.503 361.517,49.436 C361.239,48.371 360.822,47.419 360.265,46.585 C359.709,45.751 358.920,45.103 357.901,44.638 C356.880,44.175 355.815,43.942 354.702,43.942 C353.124,43.942 351.573,44.245 350.043,44.846 C348.513,45.451 347.168,46.355 346.010,47.559 C344.849,48.765 343.969,50.110 343.367,51.592 C342.763,53.076 342.370,54.606 342.185,56.182 L339.125,74.262 C338.847,75.840 338.730,77.415 338.777,78.991 C338.823,80.569 339.147,82.029 339.751,83.372 C340.353,84.717 341.326,85.760 342.672,86.501 C344.015,87.244 345.475,87.614 347.053,87.614 C348.443,87.614 349.810,87.405 351.156,86.988 C352.499,86.571 353.705,85.875 354.772,84.902 C355.836,83.928 356.671,82.794 357.275,81.494 C357.877,80.197 358.270,78.900 358.457,77.600 L361.517,59.520 C361.702,58.407 361.841,57.272 361.934,56.112 C362.026,54.954 362.049,53.817 362.004,52.705 Z" class="cls-4"/> -</svg> diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml deleted file mode 100644 index 7b26f1a44..000000000 --- a/.github/workflows/build.yml +++ /dev/null @@ -1,397 +0,0 @@ -name: Build -on: workflow_dispatch - -jobs: - create_release: - runs-on: ubuntu-latest - outputs: - version_suffix: ${{ steps.version_suffix.outputs.version_suffix }} - ytdlp_version: ${{ steps.bump_version.outputs.ytdlp_version }} - upload_url: ${{ steps.create_release.outputs.upload_url }} - steps: - - uses: actions/checkout@v2 - with: - fetch-depth: 0 - - uses: actions/setup-python@v2 - with: - python-version: '3.10' - - - name: Set version suffix - id: version_suffix - env: - PUSH_VERSION_COMMIT: ${{ secrets.PUSH_VERSION_COMMIT }} - if: "env.PUSH_VERSION_COMMIT == ''" - run: echo ::set-output name=version_suffix::$(date -u +"%H%M%S") - - name: Bump version - id: bump_version - run: | - python devscripts/update-version.py ${{ steps.version_suffix.outputs.version_suffix }} - make issuetemplates - - - name: Push to release - id: push_release - run: | - git config --global user.name github-actions - git config --global user.email github-actions@example.com - git add -u - git commit -m "[version] update" -m "Created by: ${{ github.event.sender.login }}" -m ":ci skip all :ci run dl" - git push origin --force ${{ github.event.ref }}:release - echo ::set-output name=head_sha::$(git rev-parse HEAD) - - name: Update master - env: - PUSH_VERSION_COMMIT: ${{ secrets.PUSH_VERSION_COMMIT }} - if: "env.PUSH_VERSION_COMMIT != ''" - run: git push origin ${{ github.event.ref }} - - name: Get Changelog - run: | - changelog=$(grep -oPz '(?s)(?<=### ${{ steps.bump_version.outputs.ytdlp_version }}\n{2}).+?(?=\n{2,3}###)' Changelog.md) || true - echo "changelog<<EOF" >> $GITHUB_ENV - echo "$changelog" >> $GITHUB_ENV - echo "EOF" >> $GITHUB_ENV - - - name: Create Release - id: create_release - uses: actions/create-release@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - tag_name: ${{ steps.bump_version.outputs.ytdlp_version }} - release_name: yt-dlp ${{ steps.bump_version.outputs.ytdlp_version }} - commitish: ${{ steps.push_release.outputs.head_sha }} - body: | - #### [A description of the various files]((https://github.com/yt-dlp/yt-dlp#release-files)) are in the README - - --- - - ### Changelog: - ${{ env.changelog }} - draft: false - prerelease: false - - - build_unix: - needs: create_release - runs-on: ubuntu-18.04 # Standalone executable should be built on minimum supported OS - outputs: - sha256_bin: ${{ steps.get_sha.outputs.sha256_bin }} - sha512_bin: ${{ steps.get_sha.outputs.sha512_bin }} - sha256_tar: ${{ steps.get_sha.outputs.sha256_tar }} - sha512_tar: ${{ steps.get_sha.outputs.sha512_tar }} - sha256_linux: ${{ steps.get_sha.outputs.sha256_linux }} - sha512_linux: ${{ steps.get_sha.outputs.sha512_linux }} - sha256_linux_zip: ${{ steps.get_sha.outputs.sha256_linux_zip }} - sha512_linux_zip: ${{ steps.get_sha.outputs.sha512_linux_zip }} - - steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - with: - python-version: '3.10' - - name: Install Requirements - run: | - sudo apt-get -y install zip pandoc man - python -m pip install --upgrade pip setuptools wheel twine - python -m pip install Pyinstaller -r requirements.txt - - - name: Prepare - run: | - python devscripts/update-version.py ${{ needs.create_release.outputs.version_suffix }} - python devscripts/make_lazy_extractors.py - - name: Build Unix executables - run: | - make all tar - python pyinst.py --onedir - (cd ./dist/yt-dlp_linux && zip -r ../yt-dlp_linux.zip .) - python pyinst.py - - name: Get SHA2-SUMS - id: get_sha - run: | - echo "::set-output name=sha256_bin::$(sha256sum yt-dlp | awk '{print $1}')" - echo "::set-output name=sha512_bin::$(sha512sum yt-dlp | awk '{print $1}')" - echo "::set-output name=sha256_tar::$(sha256sum yt-dlp.tar.gz | awk '{print $1}')" - echo "::set-output name=sha512_tar::$(sha512sum yt-dlp.tar.gz | awk '{print $1}')" - echo "::set-output name=sha256_linux::$(sha256sum dist/yt-dlp_linux | awk '{print $1}')" - echo "::set-output name=sha512_linux::$(sha512sum dist/yt-dlp_linux | awk '{print $1}')" - echo "::set-output name=sha256_linux_zip::$(sha256sum dist/yt-dlp_linux.zip | awk '{print $1}')" - echo "::set-output name=sha512_linux_zip::$(sha512sum dist/yt-dlp_linux.zip | awk '{print $1}')" - - - name: Upload zip binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./yt-dlp - asset_name: yt-dlp - asset_content_type: application/octet-stream - - name: Upload Source tar - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./yt-dlp.tar.gz - asset_name: yt-dlp.tar.gz - asset_content_type: application/gzip - - name: Upload standalone binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_linux - asset_name: yt-dlp_linux - asset_content_type: application/octet-stream - - name: Upload onedir binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_linux.zip - asset_name: yt-dlp_linux.zip - asset_content_type: application/zip - - - name: Build and publish on PyPi - env: - TWINE_USERNAME: __token__ - TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }} - if: "env.TWINE_PASSWORD != ''" - run: | - rm -rf dist/* - python setup.py sdist bdist_wheel - twine upload dist/* - - - name: Install SSH private key for Homebrew - env: - BREW_TOKEN: ${{ secrets.BREW_TOKEN }} - if: "env.BREW_TOKEN != ''" - uses: yt-dlp/ssh-agent@v0.5.3 - with: - ssh-private-key: ${{ env.BREW_TOKEN }} - - name: Update Homebrew Formulae - env: - BREW_TOKEN: ${{ secrets.BREW_TOKEN }} - if: "env.BREW_TOKEN != ''" - run: | - git clone git@github.com:yt-dlp/homebrew-taps taps/ - python devscripts/update-formulae.py taps/Formula/yt-dlp.rb "${{ needs.create_release.outputs.ytdlp_version }}" - git -C taps/ config user.name github-actions - git -C taps/ config user.email github-actions@example.com - git -C taps/ commit -am 'yt-dlp: ${{ needs.create_release.outputs.ytdlp_version }}' - git -C taps/ push - - - build_macos: - runs-on: macos-11 - needs: create_release - outputs: - sha256_macos: ${{ steps.get_sha.outputs.sha256_macos }} - sha512_macos: ${{ steps.get_sha.outputs.sha512_macos }} - sha256_macos_zip: ${{ steps.get_sha.outputs.sha256_macos_zip }} - sha512_macos_zip: ${{ steps.get_sha.outputs.sha512_macos_zip }} - - steps: - - uses: actions/checkout@v2 - # NB: In order to create a universal2 application, the version of python3 in /usr/bin has to be used - - name: Install Requirements - run: | - brew install coreutils - /usr/bin/python3 -m pip install -U --user pip Pyinstaller -r requirements.txt - - - name: Prepare - run: | - /usr/bin/python3 devscripts/update-version.py ${{ needs.create_release.outputs.version_suffix }} - /usr/bin/python3 devscripts/make_lazy_extractors.py - - name: Build - run: | - /usr/bin/python3 pyinst.py --target-architecture universal2 --onedir - (cd ./dist/yt-dlp_macos && zip -r ../yt-dlp_macos.zip .) - /usr/bin/python3 pyinst.py --target-architecture universal2 - - name: Get SHA2-SUMS - id: get_sha - run: | - echo "::set-output name=sha256_macos::$(sha256sum dist/yt-dlp_macos | awk '{print $1}')" - echo "::set-output name=sha512_macos::$(sha512sum dist/yt-dlp_macos | awk '{print $1}')" - echo "::set-output name=sha256_macos_zip::$(sha256sum dist/yt-dlp_macos.zip | awk '{print $1}')" - echo "::set-output name=sha512_macos_zip::$(sha512sum dist/yt-dlp_macos.zip | awk '{print $1}')" - - - name: Upload standalone binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_macos - asset_name: yt-dlp_macos - asset_content_type: application/octet-stream - - name: Upload onedir binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_macos.zip - asset_name: yt-dlp_macos.zip - asset_content_type: application/zip - - - build_windows: - runs-on: windows-latest - needs: create_release - outputs: - sha256_win: ${{ steps.get_sha.outputs.sha256_win }} - sha512_win: ${{ steps.get_sha.outputs.sha512_win }} - sha256_py2exe: ${{ steps.get_sha.outputs.sha256_py2exe }} - sha512_py2exe: ${{ steps.get_sha.outputs.sha512_py2exe }} - sha256_win_zip: ${{ steps.get_sha.outputs.sha256_win_zip }} - sha512_win_zip: ${{ steps.get_sha.outputs.sha512_win_zip }} - - steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - with: # 3.8 is used for Win7 support - python-version: '3.8' - - name: Install Requirements - run: | # Custom pyinstaller built with https://github.com/yt-dlp/pyinstaller-builds - python -m pip install --upgrade pip setuptools wheel py2exe - pip install "https://yt-dlp.github.io/Pyinstaller-Builds/x86_64/pyinstaller-4.10-py3-none-any.whl" -r requirements.txt - - - name: Prepare - run: | - python devscripts/update-version.py ${{ needs.create_release.outputs.version_suffix }} - python devscripts/make_lazy_extractors.py - - name: Build - run: | - python setup.py py2exe - Move-Item ./dist/yt-dlp.exe ./dist/yt-dlp_min.exe - python pyinst.py - python pyinst.py --onedir - Compress-Archive -Path ./dist/yt-dlp/* -DestinationPath ./dist/yt-dlp_win.zip - - name: Get SHA2-SUMS - id: get_sha - run: | - echo "::set-output name=sha256_py2exe::$((Get-FileHash dist\yt-dlp_min.exe -Algorithm SHA256).Hash.ToLower())" - echo "::set-output name=sha512_py2exe::$((Get-FileHash dist\yt-dlp_min.exe -Algorithm SHA512).Hash.ToLower())" - echo "::set-output name=sha256_win::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA256).Hash.ToLower())" - echo "::set-output name=sha512_win::$((Get-FileHash dist\yt-dlp.exe -Algorithm SHA512).Hash.ToLower())" - echo "::set-output name=sha256_win_zip::$((Get-FileHash dist\yt-dlp_win.zip -Algorithm SHA256).Hash.ToLower())" - echo "::set-output name=sha512_win_zip::$((Get-FileHash dist\yt-dlp_win.zip -Algorithm SHA512).Hash.ToLower())" - - - name: Upload py2exe binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_min.exe - asset_name: yt-dlp_min.exe - asset_content_type: application/vnd.microsoft.portable-executable - - name: Upload standalone binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp.exe - asset_name: yt-dlp.exe - asset_content_type: application/vnd.microsoft.portable-executable - - name: Upload onedir binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_win.zip - asset_name: yt-dlp_win.zip - asset_content_type: application/zip - - - build_windows32: - runs-on: windows-latest - needs: create_release - outputs: - sha256_win32: ${{ steps.get_sha.outputs.sha256_win32 }} - sha512_win32: ${{ steps.get_sha.outputs.sha512_win32 }} - - steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - with: # 3.7 is used for Vista support. See https://github.com/yt-dlp/yt-dlp/issues/390 - python-version: '3.7' - architecture: 'x86' - - name: Install Requirements - run: | - python -m pip install --upgrade pip setuptools wheel - pip install "https://yt-dlp.github.io/Pyinstaller-Builds/i686/pyinstaller-4.10-py3-none-any.whl" -r requirements.txt - - - name: Prepare - run: | - python devscripts/update-version.py ${{ needs.create_release.outputs.version_suffix }} - python devscripts/make_lazy_extractors.py - - name: Build - run: | - python pyinst.py - - name: Get SHA2-SUMS - id: get_sha - run: | - echo "::set-output name=sha256_win32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA256).Hash.ToLower())" - echo "::set-output name=sha512_win32::$((Get-FileHash dist\yt-dlp_x86.exe -Algorithm SHA512).Hash.ToLower())" - - - name: Upload standalone binary - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./dist/yt-dlp_x86.exe - asset_name: yt-dlp_x86.exe - asset_content_type: application/vnd.microsoft.portable-executable - - - finish: - runs-on: ubuntu-latest - needs: [create_release, build_unix, build_windows, build_windows32, build_macos] - - steps: - - name: Make SHA2-SUMS files - run: | - echo "${{ needs.build_unix.outputs.sha256_bin }} yt-dlp" >> SHA2-256SUMS - echo "${{ needs.build_unix.outputs.sha256_tar }} yt-dlp.tar.gz" >> SHA2-256SUMS - echo "${{ needs.build_unix.outputs.sha256_linux }} yt-dlp_linux" >> SHA2-256SUMS - echo "${{ needs.build_unix.outputs.sha256_linux_zip }} yt-dlp_linux.zip" >> SHA2-256SUMS - echo "${{ needs.build_windows.outputs.sha256_win }} yt-dlp.exe" >> SHA2-256SUMS - echo "${{ needs.build_windows.outputs.sha256_py2exe }} yt-dlp_min.exe" >> SHA2-256SUMS - echo "${{ needs.build_windows32.outputs.sha256_win32 }} yt-dlp_x86.exe" >> SHA2-256SUMS - echo "${{ needs.build_windows.outputs.sha256_win_zip }} yt-dlp_win.zip" >> SHA2-256SUMS - echo "${{ needs.build_macos.outputs.sha256_macos }} yt-dlp_macos" >> SHA2-256SUMS - echo "${{ needs.build_macos.outputs.sha256_macos_zip }} yt-dlp_macos.zip" >> SHA2-256SUMS - echo "${{ needs.build_unix.outputs.sha512_bin }} yt-dlp" >> SHA2-512SUMS - echo "${{ needs.build_unix.outputs.sha512_tar }} yt-dlp.tar.gz" >> SHA2-512SUMS - echo "${{ needs.build_unix.outputs.sha512_linux }} yt-dlp_linux" >> SHA2-512SUMS - echo "${{ needs.build_unix.outputs.sha512_linux_zip }} yt-dlp_linux.zip" >> SHA2-512SUMS - echo "${{ needs.build_windows.outputs.sha512_win }} yt-dlp.exe" >> SHA2-512SUMS - echo "${{ needs.build_windows.outputs.sha512_py2exe }} yt-dlp_min.exe" >> SHA2-512SUMS - echo "${{ needs.build_windows32.outputs.sha512_win32 }} yt-dlp_x86.exe" >> SHA2-512SUMS - echo "${{ needs.build_windows.outputs.sha512_win_zip }} yt-dlp_win.zip" >> SHA2-512SUMS - echo "${{ needs.build_macos.outputs.sha512_macos }} yt-dlp_macos" >> SHA2-512SUMS - echo "${{ needs.build_macos.outputs.sha512_macos_zip }} yt-dlp_macos.zip" >> SHA2-512SUMS - - - name: Upload SHA2-256SUMS file - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./SHA2-256SUMS - asset_name: SHA2-256SUMS - asset_content_type: text/plain - - name: Upload SHA2-512SUMS file - uses: actions/upload-release-asset@v1 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - with: - upload_url: ${{ needs.create_release.outputs.upload_url }} - asset_path: ./SHA2-512SUMS - asset_name: SHA2-512SUMS - asset_content_type: text/plain diff --git a/.github/workflows/core.yml b/.github/workflows/core.yml deleted file mode 100644 index 66e8ced53..000000000 --- a/.github/workflows/core.yml +++ /dev/null @@ -1,34 +0,0 @@ -name: Core Tests -on: [push, pull_request] -jobs: - tests: - name: Core Tests - if: "!contains(github.event.head_commit.message, 'ci skip')" - runs-on: ${{ matrix.os }} - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest] - # CPython 3.9 is in quick-test - python-version: ['3.6', '3.7', '3.10', 3.11-dev, pypy-3.6, pypy-3.7, pypy-3.8] - run-tests-ext: [sh] - include: - # atleast one of each CPython/PyPy tests must be in windows - - os: windows-latest - python-version: '3.8' - run-tests-ext: bat - - os: windows-latest - python-version: pypy-3.9 - run-tests-ext: bat - steps: - - uses: actions/checkout@v2 - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v2 - with: - python-version: ${{ matrix.python-version }} - - name: Install pytest - run: pip install pytest - - name: Run tests - continue-on-error: False - run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} core - # Linter is in quick-test diff --git a/.github/workflows/download.yml b/.github/workflows/download.yml deleted file mode 100644 index ac48e5805..000000000 --- a/.github/workflows/download.yml +++ /dev/null @@ -1,32 +0,0 @@ -name: Download Tests -on: [push, pull_request] -jobs: - tests: - name: Download Tests - if: "contains(github.event.head_commit.message, 'ci run dl')" - runs-on: ${{ matrix.os }} - strategy: - fail-fast: true - matrix: - os: [ubuntu-latest] - python-version: ['3.6', '3.7', '3.9', '3.10', 3.11-dev, pypy-3.6, pypy-3.7, pypy-3.8] - run-tests-ext: [sh] - include: - # atleast one of each CPython/PyPy tests must be in windows - - os: windows-latest - python-version: '3.8' - run-tests-ext: bat - - os: windows-latest - python-version: pypy-3.9 - run-tests-ext: bat - steps: - - uses: actions/checkout@v2 - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v2 - with: - python-version: ${{ matrix.python-version }} - - name: Install pytest - run: pip install pytest - - name: Run tests - continue-on-error: true - run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} download diff --git a/.github/workflows/quick-test.yml b/.github/workflows/quick-test.yml deleted file mode 100644 index d8e14f470..000000000 --- a/.github/workflows/quick-test.yml +++ /dev/null @@ -1,33 +0,0 @@ -name: Quick Test -on: [push, pull_request] -jobs: - tests: - name: Core Test - if: "!contains(github.event.head_commit.message, 'ci skip all')" - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Set up Python - uses: actions/setup-python@v2 - with: - python-version: 3.9 - - name: Install test requirements - run: pip install pytest pycryptodomex - - name: Run tests - run: ./devscripts/run_tests.sh core - flake8: - name: Linter - if: "!contains(github.event.head_commit.message, 'ci skip all')" - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Set up Python - uses: actions/setup-python@v2 - with: - python-version: 3.9 - - name: Install flake8 - run: pip install flake8 - - name: Make lazy extractors - run: python devscripts/make_lazy_extractors.py - - name: Run flake8 - run: flake8 . diff --git a/.gitignore b/.gitignore index 92f9029e3..48cef694f 100644 --- a/.gitignore +++ b/.gitignore @@ -14,10 +14,7 @@ cookies *.frag.urls *.info.json *.live_chat.json -*.meta *.part* -*.tmp -*.temp *.unknown_video *.ytdl .cache/ @@ -68,7 +65,6 @@ zip/ tmp/ venv/ completions/ - # Misc *~ *.DS_Store @@ -87,36 +83,124 @@ updates_key.pem # Generated AUTHORS README.txt +README.md +CONTRIBUTING.md .mailmap *.1 *.bash-completion *.fish *.tar.gz *.zsh -*.spec -test/testdata/sigs/player-*.js - +test/testdata/player-*.js # Binary -/youtube-dl -/youtube-dlc -/yt-dlp +/ytdlp yt-dlp.zip -*.exe - # Text Editor / IDE .idea *.iml .vscode *.sublime-* *.code-workspace - # Lazy extractors */extractor/lazy_extractors.py - # Plugins ytdlp_plugins/extractor/* !ytdlp_plugins/extractor/__init__.py !ytdlp_plugins/extractor/sample.py -ytdlp_plugins/postprocessor/* -!ytdlp_plugins/postprocessor/__init__.py -!ytdlp_plugins/postprocessor/sample.py +# VS Code related files +# branding +.readthedocs.yml +hypervideo.1 +hypervideo.bash-completion +hypervideo.fish +hypervideo +hypervideo.exe +hypervideo.tar.gz +setup.py +tox.in +# Ignore nonfree JS or SWF bytecode +.github/ISSUE_TEMPLATE/1_broken_site.md +.github/ISSUE_TEMPLATE/2_site_support_request.md +.github/ISSUE_TEMPLATE/3_site_feature_request.md +.github/ISSUE_TEMPLATE/4_bug_report.md +.github/ISSUE_TEMPLATE/5_feature_request.md +.github/ISSUE_TEMPLATE/6_question.md +.github/ISSUE_TEMPLATE_tmpl/1_broken_site.md +.github/ISSUE_TEMPLATE_tmpl/2_site_support_request.md +.github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.md +.github/ISSUE_TEMPLATE_tmpl/4_bug_report.md +.github/ISSUE_TEMPLATE_tmpl/5_feature_request.md +.github/PULL_REQUEST_TEMPLATE.md +.github/ISSUE_TEMPLATE.md +.github/ISSUE_TEMPLATE_tmpl.md +.github/FUNDING.yml +.github/banner.svg +.github/workflows/build.yml +.github/workflows/core.yml +.github/workflows/download.yml +.github/workflows/quick-test.yml +devscripts/create-github-release.py +devscripts/gh-pages.unused/add-version.py +devscripts/gh-pages.unused/generate-download.py +devscripts/gh-pages.unused/sign-versions.py +devscripts/gh-pages.unused/update-copyright.py +devscripts/gh-pages.unused/update-feed.py +devscripts/gh-pages.unused/update-sites.py +devscripts/make_issue_template.py +devscripts/release.sh +devscripts/show-downloads-statistics.py +devscripts/wine-py2exe.sh +devscripts/make_readme.py +devscripts/update-formulae.py +devscripts/update-version.py +test/swftests/.gitignore +test/swftests/ArrayAccess.as +test/swftests/ClassCall.as +test/swftests/ClassConstruction.as +test/swftests/ConstArrayAccess.as +test/swftests/ConstantInt.as +test/swftests/DictCall.as +test/swftests/EqualsOperator.as +test/swftests/LocalVars.as +test/swftests/MemberAssignment.as +test/swftests/NeOperator.as +test/swftests/PrivateCall.as +test/swftests/PrivateVoidCall.as +test/swftests/StaticAssignment.as +test/swftests/StaticRetrieval.as +test/swftests/StringBasics.as +test/swftests/StringCharCodeAt.as +test/swftests/StringConversion.as +test/swftests.unused/.gitignore +test/swftests.unused/ArrayAccess.as +test/swftests.unused/ClassCall.as +test/swftests.unused/ClassConstruction.as +test/swftests.unused/ConstArrayAccess.as +test/swftests.unused/ConstantInt.as +test/swftests.unused/DictCall.as +test/swftests.unused/EqualsOperator.as +test/swftests.unused/LocalVars.as +test/swftests.unused/MemberAssignment.as +test/swftests.unused/NeOperator.as +test/swftests.unused/PrivateCall.as +test/swftests.unused/PrivateVoidCall.as +test/swftests.unused/StaticAssignment.as +test/swftests.unused/StaticRetrieval.as +test/swftests.unused/StringBasics.as +test/swftests.unused/StringCharCodeAt.as +test/swftests.unused/StringConversion.as +test/test_iqiyi_sdk_interpreter.py +test/test_swfinterp.py.disabled +test/test_unicode_literals.py.disabled +test/test_update.py.disabled +test/test_write_annotations.py.disabled +test/test_iqiyi_sdk_interpreter.py~upstream_master +test/test_swfinterp.py +test/test_update.py +test/versions.json +yt_dlp/swfinterp.py +yt_dlp/update.py +ytdlp_plugins/extractor/__init__.py +ytdlp_plugins/extractor/sample.py +ytdlp_plugins/postprocessor/__init__.py +ytdlp_plugins/postprocessor/sample.py diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 988a94264..0ed1eb457 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,63 +1,26 @@ -# CONTRIBUTING TO YT-DLP - -- [OPENING AN ISSUE](#opening-an-issue) - - [Is the description of the issue itself sufficient?](#is-the-description-of-the-issue-itself-sufficient) - - [Are you using the latest version?](#are-you-using-the-latest-version) - - [Is the issue already documented?](#is-the-issue-already-documented) - - [Why are existing options not enough?](#why-are-existing-options-not-enough) - - [Have you read and understood the changes, between youtube-dl and yt-dlp](#have-you-read-and-understood-the-changes-between-youtube-dl-and-yt-dlp) - - [Is there enough context in your bug report?](#is-there-enough-context-in-your-bug-report) - - [Does the issue involve one problem, and one problem only?](#does-the-issue-involve-one-problem-and-one-problem-only) - - [Is anyone going to need the feature?](#is-anyone-going-to-need-the-feature) - - [Is your question about yt-dlp?](#is-your-question-about-yt-dlp) - - [Are you willing to share account details if needed?](#are-you-willing-to-share-account-details-if-needed) - - [Is the website primarily used for piracy](#is-the-website-primarily-used-for-piracy) -- [DEVELOPER INSTRUCTIONS](#developer-instructions) - - [Adding new feature or making overarching changes](#adding-new-feature-or-making-overarching-changes) - - [Adding support for a new site](#adding-support-for-a-new-site) - - [yt-dlp coding conventions](#yt-dlp-coding-conventions) - - [Mandatory and optional metafields](#mandatory-and-optional-metafields) - - [Provide fallbacks](#provide-fallbacks) - - [Regular expressions](#regular-expressions) - - [Long lines policy](#long-lines-policy) - - [Quotes](#quotes) - - [Inline values](#inline-values) - - [Collapse fallbacks](#collapse-fallbacks) - - [Trailing parentheses](#trailing-parentheses) - - [Use convenience conversion and parsing functions](#use-convenience-conversion-and-parsing-functions) - - [My pull request is labeled pending-fixes](#my-pull-request-is-labeled-pending-fixes) -- [EMBEDDING YT-DLP](README.md#embedding-yt-dlp) - - - -# OPENING AN ISSUE - -Bugs and suggestions should be reported at: [yt-dlp/yt-dlp/issues](https://github.com/yt-dlp/yt-dlp/issues). Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in our [discord server](https://discord.gg/H5MNcFW63r). - -**Please include the full output of yt-dlp when run with `-vU`**, i.e. **add** `-vU` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this: -``` -$ yt-dlp -vU <your command line> -[debug] Command-line config: ['-v', 'demo.com'] -[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8 -[debug] yt-dlp version 2021.09.25 (zip) -[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-74-generic-x86_64-with-glibc2.29 -[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4 +**Please include the full output of hypervideo when run with `-v`**, i.e. **add** `-v` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this: +``` +$ hypervideo -v <your command line> +[debug] System config: [] +[debug] User config: [] +[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj'] +[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251 +[debug] hypervideo version 1.1.11 +[debug] Git HEAD: 135392e +[debug] Python version 2.6.6 - Windows-2003Server-5.2.3790-SP2 +[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4 [debug] Proxy map: {} -Current Build Hash 25cc412d1d3c0725a1f2f5b7e4682f6fb40e6d15f7024e96f7afd572e9919535 -yt-dlp is up to date (2021.09.25) ... ``` **Do not post screenshots of verbose logs; only plain text is acceptable.** -The output (including the first lines) contains important debugging information. Issues without the full output are often not reproducible and therefore will be closed as `incomplete`. - -The templates provided for the Issues, should be completed and **not removed**, this helps aide the resolution of the issue. +The output (including the first lines) contains important debugging information. Issues without the full output are often not reproducible and therefore do not get solved in short order, if ever. Please re-read your issue once again to avoid a couple of common mistakes (you can and should use this as a checklist): ### Is the description of the issue itself sufficient? -We often get issue reports that we cannot really decipher. While in most cases we eventually get the required information after asking back multiple times, this poses an unnecessary drain on our resources. +We often get issue reports that we cannot really decipher. While in most cases we eventually get the required information after asking back multiple times, this poses an unnecessary drain on our resources. Many contributors, including myself, are also not native speakers, so we may misread some parts. So please elaborate on what feature you are requesting, or what bug you want to be fixed. Make sure that it's obvious @@ -65,31 +28,25 @@ So please elaborate on what feature you are requesting, or what bug you want to - How it could be fixed - How your proposed solution would look like -If your report is shorter than two lines, it is almost certainly missing some of these, which makes it hard for us to respond to it. We're often too polite to close the issue outright, but the missing info makes misinterpretation likely. We often get frustrated by these issues, since the only possible way for us to move forward on them is to ask for clarification over and over. +If your report is shorter than two lines, it is almost certainly missing some of these, which makes it hard for us to respond to it. We're often too polite to close the issue outright, but the missing info makes misinterpretation likely. As a committer myself, I often get frustrated by these issues, since the only possible way for me to move forward on them is to ask for clarification over and over. -For bug reports, this means that your report should contain the **complete** output of yt-dlp when called with the `-vU` flag. The error message you get for (most) bugs even says so, but you would not believe how many of our bug reports do not contain this information. +For bug reports, this means that your report should contain the *complete* output of hypervideo when called with the `-v` flag. The error message you get for (most) bugs even says so, but you would not believe how many of our bug reports do not contain this information. -If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--write-pages` and upload the `.dump` files you get [somewhere](https://gist.github.com). +If your server has multiple IPs or you suspect censorship, adding `--call-home` may be a good idea to get more diagnostics. If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--dump-pages` (warning: this will yield a rather large output, redirect it to the file `log.txt` by adding `>log.txt 2>&1` to your command-line) or upload the `.dump` files you get when you add `--write-pages` [somewhere](https://gist.github.com/). **Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `https://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `https://www.youtube.com/`) is *not* an example URL. ### Are you using the latest version? -Before reporting any issue, type `yt-dlp -U`. This should report that you're up-to-date. This goes for feature requests as well. +Before reporting any issue, type `doas pacman -Sy hypervideo`. This should report that you're up-to-date. About 20% of the reports we receive are already fixed, but people are using outdated versions. This goes for feature requests as well. ### Is the issue already documented? -Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/yt-dlp/yt-dlp/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2021.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity. - -Additionally, it is also helpful to see if the issue has already been documented in the [youtube-dl issue tracker](https://github.com/ytdl-org/youtube-dl/issues). If similar issues have already been reported in youtube-dl (but not in our issue tracker), links to them can be included in your issue report here. +Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity. ### Why are existing options not enough? -Before requesting a new feature, please have a quick peek at [the list of supported options](README.md#usage-and-options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem. - -### Have you read and understood the changes, between youtube-dl and yt-dlp - -There are many changes between youtube-dl and yt-dlp [(changes to default behavior)](README.md#differences-in-default-behavior), and some of the options available have a different behaviour in yt-dlp, or have been removed all together [(list of changes to options)](README.md#deprecated-options). Make sure you have read and understand the differences in the options and how this may impact your downloads before opening an issue. +Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem. ### Is there enough context in your bug report? @@ -101,89 +58,68 @@ We are then presented with a very complicated request when the original problem Some of our users seem to think there is a limit of issues they can or should open. There is no limit of issues they can or should open. While it may seem appealing to be able to dump all your issues into one ticket, that means that someone who solves one of your issues cannot mark the issue as closed. Typically, reporting a bunch of issues leads to the ticket lingering since nobody wants to attack that behemoth, until someone mercifully splits the issue into multiple ones. -In particular, every site support request issue should only pertain to services at one site (generally under a common domain, but always using the same backend technology). Do not request support for vimeo user videos, White house podcasts, and Google Plus pages in the same issue. Also, make sure that you don't post bug reports alongside feature requests. As a rule of thumb, a feature request does not include outputs of yt-dlp that are not immediately related to the feature at hand. Do not post reports of a network error alongside the request for a new video service. +In particular, every site support request issue should only pertain to services at one site (generally under a common domain, but always using the same backend technology). Do not request support for vimeo user videos, White house podcasts, and Google Plus pages in the same issue. Also, make sure that you don't post bug reports alongside feature requests. As a rule of thumb, a feature request does not include outputs of hypervideo that are not immediately related to the feature at hand. Do not post reports of a network error alongside the request for a new video service. ### Is anyone going to need the feature? Only post features that you (or an incapacitated friend you can personally talk to) require. Do not post features because they seem like a good idea. If they are really useful, they will be requested by someone who requires them. -### Is your question about yt-dlp? - -Some bug reports are completely unrelated to yt-dlp and relate to a different, or even the reporter's own, application. Please make sure that you are actually using yt-dlp. If you are using a UI for yt-dlp, report the bug to the maintainer of the actual application providing the UI. In general, if you are unable to provide the verbose log, you should not be opening the issue here. - -If the issue is with `youtube-dl` (the upstream fork of yt-dlp) and not with yt-dlp, the issue should be raised in the youtube-dl project. - -### Are you willing to share account details if needed? - -The maintainers and potential contributors of the project often do not have an account for the website you are asking support for. So any developer interested in solving your issue may ask you for account details. It is your personal discretion whether you are willing to share the account in order for the developer to try and solve your issue. However, if you are unwilling or unable to provide details, they obviously cannot work on the issue and it cannot be solved unless some developer who both has an account and is willing/able to contribute decides to solve it. - -By sharing an account with anyone, you agree to bear all risks associated with it. The maintainers and yt-dlp can't be held responsible for any misuse of the credentials. - -While these steps won't necessarily ensure that no misuse of the account takes place, these are still some good practices to follow. - -- Look for people with `Member` (maintainers of the project) or `Contributor` (people who have previously contributed code) tag on their messages. -- Change the password before sharing the account to something random (use [this](https://passwordsgenerator.net/) if you don't have a random password generator). -- Change the password after receiving the account back. - -### Is the website primarily used for piracy? - -We follow [youtube-dl's policy](https://github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free) to not support services that is primarily used for infringing copyright. Additionally, it has been decided to not to support porn sites that specialize in deep fake. We also cannot support any service that serves only [DRM protected content](https://en.wikipedia.org/wiki/Digital_rights_management). - - +### Is your question about hypervideo? +It may sound strange, but some bug reports we receive are completely unrelated to hypervideo and relate to a different, or even the reporter's own, application. Please make sure that you are actually using hypervideo. If you are using a UI for hypervideo, report the bug to the maintainer of the actual application providing the UI. On the other hand, if your UI for hypervideo fails in some way you believe is related to hypervideo, by all means, go ahead and report the bug. # DEVELOPER INSTRUCTIONS -Most users do not need to build yt-dlp and can [download the builds](https://github.com/yt-dlp/yt-dlp/releases) or get them via [the other installation methods](README.md#installation). +Most users do not need to build hypervideo and can [download the builds](https://ytdl-org.github.io/youtube-dl/download.html) or get them from their distribution. -To run yt-dlp as a developer, you don't need to build anything either. Simply execute +To run hypervideo as a developer, you don't need to build anything either. Simply execute - python -m yt_dlp + python -m youtube_dl To run the test, simply invoke your favorite test runner, or execute a test file directly; any of the following work: python -m unittest discover python test/test_download.py nosetests - pytest See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases. -If you want to create a build of yt-dlp yourself, you can follow the instructions [here](README.md#compile). +If you want to create a build of hypervideo yourself, you'll need +* python +* make (only GNU make is supported) +* pandoc +* zip +* nosetests -## Adding new feature or making overarching changes +### Adding support for a new site -Before you start writing code for implementing a new feature, open an issue explaining your feature request and atleast one use case. This allows the maintainers to decide whether such a feature is desired for the project in the first place, and will provide an avenue to discuss some implementation details. If you open a pull request for a new feature without discussing with us first, do not be surprised when we ask for large changes to the code, or even reject it outright. - -The same applies for changes to the documentation, code style, or overarching changes to the architecture - - -## Adding support for a new site - -If you want to add support for a new site, first of all **make sure** this site is **not dedicated to [copyright infringement](https://www.github.com/ytdl-org/youtube-dl#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. yt-dlp does **not support** such sites thus pull requests adding support for them **will be rejected**. +If you want to add support for a new site, first of all **make sure** this site is **not dedicated to [copyright infringement](README.md#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. hypervideo does **not support** such sites thus pull requests adding support for them **will be rejected**. After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`): -1. [Fork this repository](https://github.com/yt-dlp/yt-dlp/fork) -1. Check out the source code with: +1. [Fork this repository](https://c.hgit.ga/software/hypervideo) +2. Check out the source code with: - git clone git@github.com:YOUR_GITHUB_USERNAME/yt-dlp.git + git clone https://c.hgit.ga/software/hypervideo -1. Start a new git branch with +3. Start a new git branch with - cd yt-dlp + cd hypervideo git checkout -b yourextractor -1. Start with this simple template and save it to `yt_dlp/extractor/yourextractor.py`: +4. Start with this simple template and save it to `youtube_dl/extractor/yourextractor.py`: ```python + # coding: utf-8 + from __future__ import unicode_literals + from .common import InfoExtractor - - + + class YourExtractorIE(InfoExtractor): _VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)' - _TESTS = [{ + _TEST = { 'url': 'https://yourextractor.com/watch/42', 'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)', 'info_dict': { @@ -197,12 +133,12 @@ After you have ensured this site is distributing its content legally, you can fo # * A regular expression; start the string with re: # * Any Python type (for example int or float) } - }] + } def _real_extract(self, url): video_id = self._match_id(url) webpage = self._download_webpage(url, video_id) - + # TODO more code goes here, for example ... title = self._html_search_regex(r'<h1>(.+?)</h1>', webpage, 'title') @@ -211,59 +147,45 @@ After you have ensured this site is distributing its content legally, you can fo 'title': title, 'description': self._og_search_description(webpage), 'uploader': self._search_regex(r'<div[^>]+id="uploader"[^>]*>([^<]+)<', webpage, 'uploader', fatal=False), - # TODO more properties (see yt_dlp/extractor/common.py) + # TODO more properties (see youtube_dl/extractor/common.py) } ``` -1. Add an import in [`yt_dlp/extractor/_extractors.py`](yt_dlp/extractor/_extractors.py). Note that the class name must end with `IE`. -1. Run `python test/test_download.py TestDownload.test_YourExtractor` (note that `YourExtractor` doesn't end with `IE`). This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, the tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in. You can also run all the tests in one go with `TestDownload.test_YourExtractor_all` -1. Make sure you have atleast one test for your extractor. Even if all videos covered by the extractor are expected to be inaccessible for automated testing, tests should still be added with a `skip` parameter indicating why the particular test is disabled from running. -1. Have a look at [`yt_dlp/extractor/common.py`](yt_dlp/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](yt_dlp/extractor/common.py#L91-L426). Add tests and code for as many as you want. -1. Make sure your code follows [yt-dlp coding conventions](#yt-dlp-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart): +5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py). +6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in. +7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want. +8. Make sure your code follows [hypervideo coding conventions](#hypervideo-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart): - $ flake8 yt_dlp/extractor/yourextractor.py + $ flake8 youtube_dl/extractor/yourextractor.py -1. Make sure your code works under all [Python](https://www.python.org/) versions supported by yt-dlp, namely CPython and PyPy for Python 3.6 and above. Backward compatibility is not required for even older versions of Python. -1. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files, [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this: +9. Make sure your code works under all [Python](https://www.python.org/) versions claimed supported by hypervideo, namely 2.6, 2.7, and 3.2+. +10. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files and [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this: - $ git add yt_dlp/extractor/_extractors.py - $ git add yt_dlp/extractor/yourextractor.py - $ git commit -m '[yourextractor] Add extractor' + $ git add youtube_dl/extractor/extractors.py + $ git add youtube_dl/extractor/yourextractor.py + $ git commit -m '[yourextractor] Add new extractor' $ git push origin yourextractor -1. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it. +11. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it. In any case, thank you very much for your contributions! -**Tip:** To test extractors that require login information, create a file `test/local_parameters.json` and add `"usenetrc": true` or your username and password in it: -```json -{ - "username": "your user name", - "password": "your password" -} -``` - -## yt-dlp coding conventions +## hypervideo coding conventions This section introduces a guide lines for writing idiomatic, robust and future-proof extractor code. -Extractors are very fragile by nature since they depend on the layout of the source data provided by 3rd party media hosters out of your control and this layout tends to change. As an extractor implementer your task is not only to write code that will extract media links and metadata correctly but also to minimize dependency on the source's layout and even to make the code foresee potential future changes and be ready for that. This is important because it will allow the extractor not to break on minor layout changes thus keeping old yt-dlp versions working. Even though this breakage issue may be easily fixed by a new version of yt-dlp, this could take some time, during which the the extractor will remain broken. - +Extractors are very fragile by nature since they depend on the layout of the source data provided by 3rd party media hosters out of your control and this layout tends to change. As an extractor implementer your task is not only to write code that will extract media links and metadata correctly but also to minimize dependency on the source's layout and even to make the code foresee potential future changes and be ready for that. This is important because it will allow the extractor not to break on minor layout changes thus keeping old hypervideo versions working. Even though this breakage issue is easily fixed by emitting a new version of hypervideo with a fix incorporated, all the previous versions become broken in all repositories and distros' packages that may not be so prompt in fetching the update from us. Needless to say, some non rolling release distros may never receive an update at all. ### Mandatory and optional metafields -For extraction to work yt-dlp relies on metadata your extractor extracts and provides to yt-dlp expressed by an [information dictionary](yt_dlp/extractor/common.py#L91-L426) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by yt-dlp: +For extraction to work hypervideo relies on metadata your extractor extracts and provides to hypervideo expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl: - `id` (media identifier) - `title` (media title) - `url` (media download URL) or `formats` -The aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken. While all extractors must return a `title`, they must also allow it's extraction to be non-fatal. - -For pornographic sites, appropriate `age_limit` must also be returned. +In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention hypervideo also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken. -The extractor is allowed to return the info dict without url or formats in some special cases if it allows the user to extract usefull information with `--ignore-no-formats-error` - Eg: when the video is a live stream that has not started yet. - -[Any field](yt_dlp/extractor/common.py#219-L426) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields. +[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields. #### Example @@ -272,15 +194,13 @@ Say you have some source dictionary `meta` that you've fetched as JSON with HTTP ```python meta = self._download_json(url, video_id) ``` - + Assume at this point `meta`'s layout is: ```python { + ... "summary": "some fancy summary text", - "user": { - "name": "uploader name" - }, ... } ``` @@ -299,30 +219,6 @@ description = meta['summary'] # incorrect The latter will break extraction process with `KeyError` if `summary` disappears from `meta` at some later time but with the former approach extraction will just go ahead with `description` set to `None` which is perfectly fine (remember `None` is equivalent to the absence of data). - -If the data is nested, do not use `.get` chains, but instead make use of `traverse_obj`. - -Considering the above `meta` again, assume you want to extract `["user"]["name"]` and put it in the resulting info dict as `uploader` - -```python -uploader = traverse_obj(meta, ('user', 'name')) # correct -``` - -and not like: - -```python -uploader = meta['user']['name'] # incorrect -``` -or -```python -uploader = meta.get('user', {}).get('name') # incorrect -``` -or -```python -uploader = try_get(meta, lambda x: x['user']['name']) # old utility -``` - - Similarly, you should pass `fatal=False` when extracting optional data from a webpage with `_search_regex`, `_html_search_regex` or similar methods, for instance: ```python @@ -343,53 +239,27 @@ description = self._search_regex( On failure this code will silently continue the extraction with `description` set to `None`. That is useful for metafields that may or may not be present. - -Another thing to remember is not to try to iterate over `None` - -Say you extracted a list of thumbnails into `thumbnail_data` and want to iterate over them - -```python -thumbnail_data = data.get('thumbnails') or [] -thumbnails = [{ - 'url': item['url'] -} for item in thumbnail_data] # correct -``` - -and not like: - -```python -thumbnail_data = data.get('thumbnails') -thumbnails = [{ - 'url': item['url'] -} for item in thumbnail_data] # incorrect -``` - -In this case, `thumbnail_data` will be `None` if the field was not found and this will cause the loop `for item in thumbnail_data` to raise a fatal error. Using `or []` avoids this error and results in setting an empty list in `thumbnails` instead. - - ### Provide fallbacks When extracting metadata try to do so from multiple sources. For example if `title` is present in several places, try extracting from at least some of them. This makes it more future-proof in case some of the sources become unavailable. - #### Example -Say `meta` from the previous example has a `title` and you are about to extract it like: +Say `meta` from the previous example has a `title` and you are about to extract it. Since `title` is a mandatory meta field you should end up with something like: ```python -title = meta.get('title') +title = meta['title'] ``` -If `title` disappears from `meta` in future due to some changes on the hoster's side the title extraction would fail. +If `title` disappears from `meta` in future due to some changes on the hoster's side the extraction would fail since `title` is mandatory. That's expected. -Assume that you have some another source you can extract `title` from, for example `og:title` HTML meta of a `webpage`. In this case you can provide a fallback like: +Assume that you have some another source you can extract `title` from, for example `og:title` HTML meta of a `webpage`. In this case you can provide a fallback scenario: ```python title = meta.get('title') or self._og_search_title(webpage) ``` -This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`, making the extractor more robust. - +This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`. ### Regular expressions @@ -412,6 +282,7 @@ Incorrect: r'(id|ID)=(?P<id>\d+)' ``` + #### Make regular expressions relaxed and flexible When using regular expressions try to write them fuzzy, relaxed and flexible, skipping insignificant parts that are more likely to change, allowing both single and double quotes for quoted values and so on. @@ -427,84 +298,34 @@ Say you need to extract `title` from the following HTML code: The code for that task should look similar to: ```python -title = self._search_regex( # correct +title = self._search_regex( r'<span[^>]+class="title"[^>]*>([^<]+)', webpage, 'title') ``` -which tolerates potential changes in the `style` attribute's value. Or even better: +Or even better: ```python -title = self._search_regex( # correct +title = self._search_regex( r'<span[^>]+class=(["\'])title\1[^>]*>(?P<title>[^<]+)', webpage, 'title', group='title') ``` -which also handles both single quotes in addition to double quotes. +Note how you tolerate potential changes in the `style` attribute's value or switch from using double quotes to single for `class` attribute: The code definitely should not look like: ```python -title = self._search_regex( # incorrect +title = self._search_regex( r'<span style="position: absolute; left: 910px; width: 90px; float: right; z-index: 9999;" class="title">(.*?)</span>', webpage, 'title', group='title') ``` -or even - -```python -title = self._search_regex( # incorrect - r'<span style=".*?" class="title">(.*?)</span>', - webpage, 'title', group='title') -``` - -Here the presence or absence of other attributes including `style` is irrelevent for the data we need, and so the regex must not depend on it - - -#### Keep the regular expressions as simple as possible, but no simpler - -Since many extractors deal with unstructured data provided by websites, we will often need to use very complex regular expressions. You should try to use the *simplest* regex that can accomplish what you want. In other words, each part of the regex must have a reason for existing. If you can take out a symbol and the functionality does not change, the symbol should not be there. - -##### Example - -Correct: - -```python -_VALID_URL = r'https?://(?:www\.)?website\.com/(?:[^/]+/){3,4}(?P<display_id>[^/]+)_(?P<id>\d+)' -``` - -Incorrect: - -```python -_VALID_URL = r'https?:\/\/(?:www\.)?website\.com\/[^\/]+/[^\/]+/[^\/]+(?:\/[^\/]+)?\/(?P<display_id>[^\/]+)_(?P<id>\d+)' -``` - -#### Do not misuse `.` and use the correct quantifiers (`+*?`) - -Avoid creating regexes that over-match because of wrong use of quantifiers. Also try to avoid non-greedy matching (`?`) where possible since they could easily result in [catastrophic backtracking](https://www.regular-expressions.info/catastrophic.html) - -Correct: - -```python -title = self._search_regex(r'<span\b[^>]+class="title"[^>]*>([^<]+)', webpage, 'title') -``` - -Incorrect: - -```python -title = self._search_regex(r'<span\b.*class="title".*>(.+?)<', webpage, 'title') -``` - - ### Long lines policy -There is a soft limit to keep lines of code under 100 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse. Sometimes, it may be reasonable to go upto 120 characters and sometimes even 80 can be unreadable. Keep in mind that this is not a hard limit and is just one of many tools to make the code more readable. +There is a soft limit to keep lines of code under 80 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse. For example, you should **never** split long string literals like URLs or some other often copied entities over multiple lines to fit this limit: -Conversely, don't unecessarily split small lines further. As a rule of thumb, if removing the line split keeps the code under 80 characters, it should be a single line. - -##### Examples - Correct: ```python @@ -518,76 +339,26 @@ Incorrect: 'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4' ``` -Correct: - -```python -uploader = traverse_obj(info, ('uploader', 'name'), ('author', 'fullname')) -``` - -Incorrect: - -```python -uploader = traverse_obj( - info, - ('uploader', 'name'), - ('author', 'fullname')) -``` - -Correct: - -```python -formats = self._extract_m3u8_formats( - m3u8_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls', - note='Downloading HD m3u8 information', errnote='Unable to download HD m3u8 information') -``` - -Incorrect: - -```python -formats = self._extract_m3u8_formats(m3u8_url, - video_id, - 'mp4', - 'm3u8_native', - m3u8_id='hls', - note='Downloading HD m3u8 information', - errnote='Unable to download HD m3u8 information') -``` - - -### Quotes - -Always use single quotes for strings (even if the string has `'`) and double quotes for docstrings. Use `'''` only for multi-line strings. An exception can be made if a string has multiple single quotes in it and escaping makes it *significantly* harder to read. For f-strings, use you can use double quotes on the inside. But avoid f-strings that have too many quotes inside. - - ### Inline values Extracting variables is acceptable for reducing code duplication and improving readability of complex expressions. However, you should avoid extracting variables used only once and moving them to opposite parts of the extractor file, which makes reading the linear flow difficult. -#### Examples +#### Example Correct: ```python -return { - 'title': self._html_search_regex(r'<h1>([^<]+)</h1>', webpage, 'title'), - # ...some lines of code... -} +title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, 'title') ``` Incorrect: ```python -TITLE_RE = r'<h1>([^<]+)</h1>' +TITLE_RE = r'<title>([^<]+)</title>' # ...some lines of code... title = self._html_search_regex(TITLE_RE, webpage, 'title') -# ...some lines of code... -return { - 'title': title, - # ...some lines of code... -} ``` - ### Collapse fallbacks Multiple fallback values can quickly become unwieldy. Collapse multiple fallback values into a single expression via a list of patterns. @@ -613,100 +384,52 @@ description = ( Methods supporting list of patterns are: `_search_regex`, `_html_search_regex`, `_og_search_property`, `_html_search_meta`. - ### Trailing parentheses -Always move trailing parentheses used for grouping/functions after the last argument. On the other hand, multi-line literal list/tuple/dict/set should closed be in a new line. Generators and list/dict comprehensions may use either style +Always move trailing parentheses after the last argument. -#### Examples - -Correct: +#### Example -```python -url = traverse_obj(info, ( - 'context', 'dispatcher', 'stores', 'VideoTitlePageStore', 'data', 'video', 0, 'VideoUrlSet', 'VideoUrl'), list) -``` Correct: ```python -url = traverse_obj( - info, - ('context', 'dispatcher', 'stores', 'VideoTitlePageStore', 'data', 'video', 0, 'VideoUrlSet', 'VideoUrl'), + lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'], list) ``` Incorrect: ```python -url = traverse_obj( - info, - ('context', 'dispatcher', 'stores', 'VideoTitlePageStore', 'data', 'video', 0, 'VideoUrlSet', 'VideoUrl'), - list + lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'], + list, ) ``` -Correct: +### Use convenience conversion and parsing functions -```python -f = { - 'url': url, - 'format_id': format_id, -} -``` +Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well. -Incorrect: +Use `url_or_none` for safe URL processing. -```python -f = {'url': url, - 'format_id': format_id} -``` +Use `try_get` for safe metadata extraction from parsed JSON. -Correct: +Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction. -```python -formats = [process_formats(f) for f in format_data - if f.get('type') in ('hls', 'dash', 'direct') and f.get('downloadable')] -``` +Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions. -Correct: +#### More examples + +##### Safely extract optional description from parsed JSON ```python -formats = [ - process_formats(f) for f in format_data - if f.get('type') in ('hls', 'dash', 'direct') and f.get('downloadable') -] +description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str) ``` - -### Use convenience conversion and parsing functions - -Wrap all extracted numeric data into safe functions from [`yt_dlp/utils.py`](yt_dlp/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well. - -Use `url_or_none` for safe URL processing. - -Use `traverse_obj` and `try_call` (superseeds `dict_get` and `try_get`) for safe metadata extraction from parsed JSON. - -Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction. - -Explore [`yt_dlp/utils.py`](yt_dlp/utils.py) for more useful convenience functions. - -#### Examples +##### Safely extract more optional metadata ```python -description = traverse_obj(response, ('result', 'video', 'summary'), expected_type=str) -thumbnails = traverse_obj(response, ('result', 'thumbnails', ..., 'url'), expected_type=url_or_none) -video = traverse_obj(response, ('result', 'video', 0), default={}, expected_type=dict) +video = try_get(response, lambda x: x['result']['video'][0], dict) or {} +description = video.get('summary') duration = float_or_none(video.get('durationMs'), scale=1000) view_count = int_or_none(video.get('views')) ``` - - -# My pull request is labeled pending-fixes - -The `pending-fixes` label is added when there are changes requested to a PR. When the necessary changes are made, the label should be removed. However, despite our best efforts, it may sometimes happen that the maintainer did not see the changes or forgot to remove the label. If your PR is still marked as `pending-fixes` a few days after all requested changes have been made, feel free to ping the maintainer who labeled your issue and ask them to re-review and remove the label. - - - - -# EMBEDDING YT-DLP -See [README.md#embedding-yt-dlp](README.md#embedding-yt-dlp) for instructions on how to embed yt-dlp in another Python program diff --git a/Changelog.md b/Changelog.md index d1e103234..a6b898bd8 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1175,7 +1175,7 @@ * [update] Check for new version even if not updateable * [build] Add more files to the tarball * [build] Allow building with py2exe (and misc fixes) -* [build] Use pycryptodomex by [shirt](https://github.com/shirt-dev), [pukkandan](https://github.com/pukkandan) +* [build] Use pycryptodome by [shirt](https://github.com/shirt-dev), [pukkandan](https://github.com/pukkandan) * [cleanup] Some minor refactoring, improve docs and misc cleanup diff --git a/Collaborators.md b/Collaborators.md deleted file mode 100644 index 52e3b9cae..000000000 --- a/Collaborators.md +++ /dev/null @@ -1,51 +0,0 @@ -# Collaborators - -This is a list of the collaborators of the project and their major contributions. See the [Changelog](Changelog.md) for more details. - -You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [authors of youtube-dl](https://github.com/ytdl-org/youtube-dl/blob/master/AUTHORS) - - -## [pukkandan](https://github.com/pukkandan) - -[](https://ko-fi.com/pukkandan) - -* Owner of the fork - - - -## [shirt](https://github.com/shirt-dev) - -[](https://ko-fi.com/shirt) - -* Multithreading (`-N`) and aria2c support for fragment downloads -* Support for media initialization and discontinuity in HLS -* The self-updater (`-U`) - - - -## [coletdjnz](https://github.com/coletdjnz) - -[](https://github.com/sponsors/coletdjnz) - -* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements -* Added support for downloading YoutubeWebArchive videos -* Added support for new websites MainStreaming, PRX, nzherald, etc - - - -## [Ashish0804](https://github.com/Ashish0804) - -[](https://ko-fi.com/ashish0804) - -* Added support for new websites BiliIntl, DiscoveryPlusIndia, OlympicsReplay, PlanetMarathi, ShemarooMe, Utreon, Zee5 etc -* Added playlist/series downloads for Hotstar, ParamountPlus, Rumble, SonyLIV, Trovo, TubiTv, Voot etc -* Improved/fixed support for HiDive, HotStar, Hungama, LBRY, LinkedInLearning, Mxplayer, SonyLiv, TV2, Vimeo, VLive etc - - -## [Lesmiscore](https://github.com/Lesmiscore) (nao20010128nao) - -**Bitcoin**: bc1qfd02r007cutfdjwjmyy9w23rjvtls6ncve7r3s -**Monacoin**: mona1q3tf7dzvshrhfe3md379xtvt2n22duhglv5dskr - -* Download live from start to end for YouTube -* Added support for new websites mildom, PixivSketch, skeb, radiko, voicy, mirrativ, openrec, whowatch, damtomo, 17.live, mixch etc @@ -1,24 +1,121 @@ -This is free and unencumbered software released into the public domain. - -Anyone is free to copy, modify, publish, use, compile, sell, or -distribute this software, either in source code form or as a compiled -binary, for any purpose, commercial or non-commercial, and by any -means. - -In jurisdictions that recognize copyright laws, the author or authors -of this software dedicate any and all copyright interest in the -software to the public domain. We make this dedication for the benefit -of the public at large and to the detriment of our heirs and -successors. We intend this dedication to be an overt act of -relinquishment in perpetuity of all present and future rights to this -software under copyright law. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, -EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR -OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, -ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR -OTHER DEALINGS IN THE SOFTWARE. - -For more information, please refer to <http://unlicense.org/> +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. @@ -1,31 +1,25 @@ -all: lazy-extractors yt-dlp doc pypi-files -clean: clean-test clean-dist -clean-all: clean clean-cache +all: yt-dlp doc pypi-files +clean: clean-test clean-dist clean-cache completions: completion-bash completion-fish completion-zsh -doc: README.md CONTRIBUTING.md issuetemplates supportedsites +doc: README.md CONTRIBUTING.md ot: offlinetest tar: yt-dlp.tar.gz # Keep this list in sync with MANIFEST.in # intended use: when building a source distribution, # make pypi-files && python setup.py sdist -pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites \ - completions yt-dlp.1 requirements.txt setup.cfg devscripts/* test/* +pypi-files: AUTHORS Changelog.md LICENSE README.md completions devscripts/* test/* -.PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites +.PHONY: all clean install test tar pypi-files completions ot offlinetest codetest clean-test: - rm -rf test/testdata/sigs/player-*.js tmp/ *.annotations.xml *.aria2 *.description *.dump *.frag \ - *.frag.aria2 *.frag.urls *.info.json *.live_chat.json *.meta *.part* *.tmp *.temp *.unknown_video *.ytdl \ - *.3gp *.ape *.ass *.avi *.desktop *.flac *.flv *.jpeg *.jpg *.m4a *.m4v *.mhtml *.mkv *.mov *.mp3 \ - *.mp4 *.ogg *.opus *.png *.sbv *.srt *.swf *.swp *.ttml *.url *.vtt *.wav *.webloc *.webm *.webp + rm -rf *.3gp *.annotations.xml *.ape *.avi *.description *.dump *.flac *.flv *.frag *.frag.aria2 *.frag.urls \ + *.info.json *.jpeg *.jpg *.live_chat.json *.m4a *.m4v *.mkv *.mp3 *.mp4 *.ogg *.opus *.part* *.png *.sbv *.srt \ + *.swf *.swp *.ttml *.vtt *.wav *.webm *.webp *.ytdl test/testdata/player-*.js clean-dist: - rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ \ - yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap + rm -rf MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap clean-cache: - find . \( \ - -type d -name .pytest_cache -o -type d -name __pycache__ -o -name "*.pyc" -o -name "*.class" \ - \) -prune -exec rm -rf {} \; + find . -name "*.pyc" -o -name "*.class" -delete completion-bash: completions/bash/yt-dlp completion-fish: completions/fish/yt-dlp.fish @@ -42,27 +36,11 @@ PYTHON ?= /usr/bin/env python3 # set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi) -# set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2 -MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi) - -install: lazy-extractors yt-dlp yt-dlp.1 completions - mkdir -p $(DESTDIR)$(BINDIR) - install -m755 yt-dlp $(DESTDIR)$(BINDIR)/yt-dlp - mkdir -p $(DESTDIR)$(MANDIR)/man1 - install -m644 yt-dlp.1 $(DESTDIR)$(MANDIR)/man1/yt-dlp.1 - mkdir -p $(DESTDIR)$(SHAREDIR)/bash-completion/completions - install -m644 completions/bash/yt-dlp $(DESTDIR)$(SHAREDIR)/bash-completion/completions/yt-dlp - mkdir -p $(DESTDIR)$(SHAREDIR)/zsh/site-functions - install -m644 completions/zsh/_yt-dlp $(DESTDIR)$(SHAREDIR)/zsh/site-functions/_yt-dlp - mkdir -p $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d - install -m644 completions/fish/yt-dlp.fish $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish - -uninstall: - rm -f $(DESTDIR)$(BINDIR)/yt-dlp - rm -f $(DESTDIR)$(MANDIR)/man1/yt-dlp.1 - rm -f $(DESTDIR)$(SHAREDIR)/bash-completion/completions/yt-dlp - rm -f $(DESTDIR)$(SHAREDIR)/zsh/site-functions/_yt-dlp - rm -f $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish +install: yt-dlp completions + install -Dm755 yt-dlp $(DESTDIR)$(BINDIR) + install -Dm644 completions/bash/yt-dlp $(DESTDIR)$(SHAREDIR)/bash-completion/completions/yt-dlp + install -Dm644 completions/zsh/_yt-dlp $(DESTDIR)$(SHAREDIR)/zsh/site-functions/_yt-dlp + install -Dm644 completions/fish/yt-dlp.fish $(DESTDIR)$(SHAREDIR)/fish/vendor_completions.d/yt-dlp.fish codetest: flake8 . @@ -74,49 +52,27 @@ test: offlinetest: codetest $(PYTHON) -m pytest -k "not download" -# XXX: This is hard to maintain -CODE_FOLDERS = yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor yt_dlp/compat \ - yt_dlp/extractor/anvato_token_generator yt-dlp: yt_dlp/*.py yt_dlp/*/*.py mkdir -p zip - for d in $(CODE_FOLDERS) ; do \ + for d in yt_dlp yt_dlp/downloader yt_dlp/extractor yt_dlp/postprocessor ; do \ mkdir -p zip/$$d ;\ cp -pPR $$d/*.py zip/$$d/ ;\ done - touch -t 200001010101 zip/yt_dlp/*.py zip/yt_dlp/*/*.py zip/yt_dlp/*/*/*.py + touch -t 200001010101 zip/yt_dlp/*.py zip/yt_dlp/*/*.py mv zip/yt_dlp/__main__.py zip/ - cd zip ; zip -q ../yt-dlp yt_dlp/*.py yt_dlp/*/*.py yt_dlp/*/*/*.py __main__.py + cd zip ; zip -q ../yt-dlp yt_dlp/*.py yt_dlp/*/*.py __main__.py rm -rf zip echo '#!$(PYTHON)' > yt-dlp cat yt-dlp.zip >> yt-dlp rm yt-dlp.zip chmod a+x yt-dlp -README.md: yt_dlp/*.py yt_dlp/*/*.py devscripts/make_readme.py - COLUMNS=80 $(PYTHON) yt_dlp/__main__.py --ignore-config --help | $(PYTHON) devscripts/make_readme.py +README.md: yt_dlp/*.py yt_dlp/*/*.py + COLUMNS=80 $(PYTHON) yt_dlp/__main__.py --help | $(PYTHON) devscripts/make_readme.py -CONTRIBUTING.md: README.md devscripts/make_contributing.py +CONTRIBUTING.md: README.md $(PYTHON) devscripts/make_contributing.py README.md CONTRIBUTING.md -issuetemplates: devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.yml .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.yml .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.yml .github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml .github/ISSUE_TEMPLATE_tmpl/5_feature_request.yml yt_dlp/version.py - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/1_broken_site.yml .github/ISSUE_TEMPLATE/1_broken_site.yml - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/2_site_support_request.yml .github/ISSUE_TEMPLATE/2_site_support_request.yml - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/3_site_feature_request.yml .github/ISSUE_TEMPLATE/3_site_feature_request.yml - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/4_bug_report.yml .github/ISSUE_TEMPLATE/4_bug_report.yml - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/5_feature_request.yml .github/ISSUE_TEMPLATE/5_feature_request.yml - $(PYTHON) devscripts/make_issue_template.py .github/ISSUE_TEMPLATE_tmpl/6_question.yml .github/ISSUE_TEMPLATE/6_question.yml - -supportedsites: - $(PYTHON) devscripts/make_supportedsites.py supportedsites.md - -README.txt: README.md - pandoc -f $(MARKDOWN) -t plain README.md -o README.txt - -yt-dlp.1: README.md devscripts/prepare_manpage.py - $(PYTHON) devscripts/prepare_manpage.py yt-dlp.1.temp.md - pandoc -s -f $(MARKDOWN) -t man yt-dlp.1.temp.md -o yt-dlp.1 - rm -f yt-dlp.1.temp.md - completions/bash/yt-dlp: yt_dlp/*.py yt_dlp/*/*.py devscripts/bash-completion.in mkdir -p completions/bash $(PYTHON) devscripts/bash-completion.py @@ -129,7 +85,7 @@ completions/fish/yt-dlp.fish: yt_dlp/*.py yt_dlp/*/*.py devscripts/fish-completi mkdir -p completions/fish $(PYTHON) devscripts/fish-completion.py -_EXTRACTOR_FILES = $(shell find yt_dlp/extractor -name '*.py' -and -not -name 'lazy_extractors.py') +_EXTRACTOR_FILES = $(shell find yt_dlp/extractor -iname '*.py' -and -not -iname 'lazy_extractors.py') yt_dlp/extractor/lazy_extractors.py: devscripts/make_lazy_extractors.py devscripts/lazy_load_template.py $(_EXTRACTOR_FILES) $(PYTHON) devscripts/make_lazy_extractors.py $@ @@ -141,14 +97,13 @@ yt-dlp.tar.gz: all --exclude '*.pyo' \ --exclude '*~' \ --exclude '__pycache__' \ - --exclude '.pytest_cache' \ --exclude '.git' \ -- \ - README.md supportedsites.md Changelog.md LICENSE \ - CONTRIBUTING.md Collaborators.md CONTRIBUTORS AUTHORS \ - Makefile MANIFEST.in yt-dlp.1 README.txt completions \ + bin README.md Changelog.md LICENSE \ + CONTRIBUTING.md CONTRIBUTORS AUTHORS \ + Makefile MANIFEST.in README.md completions \ setup.py setup.cfg yt-dlp yt_dlp requirements.txt \ - devscripts test + devscripts test tox.ini pytest.ini AUTHORS: .mailmap git shortlog -s -n | cut -f2 | sort > AUTHORS @@ -1,1147 +1,465 @@ -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> -<div align="center"> - -[](#readme) - -[](#release-files "Release") -[](https://pypi.org/project/yt-dlp "PyPi") -[](Collaborators.md#collaborators "Donate") -[](https://matrix.to/#/#yt-dlp:matrix.org "Matrix") -[](https://discord.gg/H5MNcFW63r "Discord") -[](supportedsites.md "Supported Sites") -[](LICENSE "License") -[](https://github.com/yt-dlp/yt-dlp/actions "CI Status") -[](https://github.com/yt-dlp/yt-dlp/commits "Commit History") -[](https://github.com/yt-dlp/yt-dlp/commits "Commit History") - -</div> -<!-- MANPAGE: END EXCLUDED SECTION --> - -yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on the now inactive [youtube-dlc](https://github.com/blackjack4494/yt-dlc). The main focus of this project is adding new features and patches while also keeping up to date with the original project - -<!-- MANPAGE: MOVE "USAGE AND OPTIONS" SECTION HERE --> - -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> -* [NEW FEATURES](#new-features) - * [Differences in default behavior](#differences-in-default-behavior) -* [INSTALLATION](#installation) - * [Update](#update) - * [Release Files](#release-files) - * [Dependencies](#dependencies) - * [Compile](#compile) -* [USAGE AND OPTIONS](#usage-and-options) - * [General Options](#general-options) - * [Network Options](#network-options) - * [Geo-restriction](#geo-restriction) - * [Video Selection](#video-selection) - * [Download Options](#download-options) - * [Filesystem Options](#filesystem-options) - * [Thumbnail Options](#thumbnail-options) - * [Internet Shortcut Options](#internet-shortcut-options) - * [Verbosity and Simulation Options](#verbosity-and-simulation-options) - * [Workarounds](#workarounds) - * [Video Format Options](#video-format-options) - * [Subtitle Options](#subtitle-options) - * [Authentication Options](#authentication-options) - * [Post-processing Options](#post-processing-options) - * [SponsorBlock Options](#sponsorblock-options) - * [Extractor Options](#extractor-options) -* [CONFIGURATION](#configuration) - * [Authentication with .netrc file](#authentication-with-netrc-file) -* [OUTPUT TEMPLATE](#output-template) - * [Output template and Windows batch files](#output-template-and-windows-batch-files) - * [Output template examples](#output-template-examples) -* [FORMAT SELECTION](#format-selection) - * [Filtering Formats](#filtering-formats) - * [Sorting Formats](#sorting-formats) - * [Format Selection examples](#format-selection-examples) -* [MODIFYING METADATA](#modifying-metadata) - * [Modifying metadata examples](#modifying-metadata-examples) -* [EXTRACTOR ARGUMENTS](#extractor-arguments) -* [PLUGINS](#plugins) -* [EMBEDDING YT-DLP](#embedding-yt-dlp) - * [Embedding examples](#embedding-examples) -* [DEPRECATED OPTIONS](#deprecated-options) -* [CONTRIBUTING](CONTRIBUTING.md#contributing-to-yt-dlp) - * [Opening an Issue](CONTRIBUTING.md#opening-an-issue) - * [Developer Instructions](CONTRIBUTING.md#developer-instructions) -* [MORE](#more) -<!-- MANPAGE: END EXCLUDED SECTION --> - - -# NEW FEATURES - -* Based on **youtube-dl 2021.12.17 [commit/8a158a9](https://github.com/ytdl-org/youtube-dl/commit/8a158a936c8b002ef536e9e2b778ded02c09c0fa)**<!--([exceptions](https://github.com/yt-dlp/yt-dlp/issues/21))--> and **youtube-dlc 2020.11.11-3 [commit/f9401f2](https://github.com/blackjack4494/yt-dlc/commit/f9401f2a91987068139c5f757b12fc711d4c0cee)**: You get all the features and patches of [youtube-dlc](https://github.com/blackjack4494/yt-dlc) in addition to the latest [youtube-dl](https://github.com/ytdl-org/youtube-dl) - -* **[SponsorBlock Integration](#sponsorblock-options)**: You can mark/remove sponsor sections in youtube videos by utilizing the [SponsorBlock](https://sponsor.ajay.app) API - -* **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection than what is possible by simply using `--format` ([examples](#format-selection-examples)) - -* **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico livestreams are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details. - -* **Youtube improvements**: - * All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`, `:ytnotif`) and private playlists supports downloading multiple pages of content - * Search (`ytsearch:`, `ytsearchdate:`), search URLs and in-channel search works - * Mixes supports downloading multiple pages of content - * Some (but not all) age-gated content can be downloaded without cookies - * Fix for [n-sig based throttling](https://github.com/ytdl-org/youtube-dl/issues/29326) - * Redirect channel's home URL automatically to `/video` to preserve the old behaviour - * `255kbps` audio is extracted (if available) from youtube music when premium cookies are given - * Youtube music Albums, channels etc can be downloaded ([except self-uploaded music](https://github.com/yt-dlp/yt-dlp/issues/723)) - * Download livestreams from the start using `--live-from-start` (experimental) - * Support for downloading stories (`ytstories:<channel UCID>`) - * Support for downloading clips - -* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[+KEYRING][:PROFILE]` - -* **Download time range**: Videos can be downloaded partially based on either timestamps or chapters using `--download-sections` - -* **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters` - -* **Multi-threaded fragment downloads**: Download multiple fragments of m3u8/mpd videos in parallel. Use `--concurrent-fragments` (`-N`) option to set the number of threads used - -* **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats - -* **New and fixed extractors**: Many new extractors have been added and a lot of existing ones have been fixed. See the [changelog](Changelog.md) or the [list of supported sites](supportedsites.md) - -* **New MSOs**: Philo, Spectrum, SlingTV, Cablevision, RCN etc. - -* **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details - -* **Multiple paths and output templates**: You can give different [output templates](#output-template) and download paths for different types of files. You can also set a temporary path where intermediary files are downloaded to using `--paths` (`-P`) - -* **Portable Configuration**: Configuration files are automatically loaded from the home and root directories. See [configuration](#configuration) for details - -* **Output template improvements**: Output templates can now have date-time formatting, numeric offsets, object traversal etc. See [output template](#output-template) for details. Even more advanced operations can also be done with the help of `--parse-metadata` and `--replace-in-metadata` - -* **Other new options**: Many new options have been added such as `--alias`, `--print`, `--concat-playlist`, `--wait-for-video`, `--retry-sleep`, `--sleep-requests`, `--convert-thumbnails`, `--force-download-archive`, `--force-overwrites`, `--break-on-reject` etc - -* **Improvements**: Regex and other operators in `--format`/`--match-filter`, multiple `--postprocessor-args` and `--downloader-args`, faster archive checking, more [format selection options](#format-selection), merge multi-video/audio, multiple `--config-locations`, `--exec` at different stages, etc - -* **Plugins**: Extractors and PostProcessors can be loaded from an external file. See [plugins](#plugins) for details - -* **Self-updater**: The releases can be updated using `yt-dlp -U` - -See [changelog](Changelog.md) or [commits](https://github.com/yt-dlp/yt-dlp/commits) for the full list of changes - -### Differences in default behavior - -Some of yt-dlp's default options are different from that of youtube-dl and youtube-dlc: - -* The options `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details -* `avconv` is not supported as an alternative to `ffmpeg` -* yt-dlp stores config files in slightly different locations to youtube-dl. See [configuration](#configuration) for a list of correct locations -* The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s-%(id)s.%(ext)s`. Instead, you may use `--compat-options filename` -* The default [format sorting](#sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order -* The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be preferred. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this -* Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both -* `--no-abort-on-error` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead -* When writing metadata files such as thumbnails, description or infojson, the same information (if available) is also written for playlists. Use `--no-write-playlist-metafiles` or `--compat-options no-playlist-metafiles` to not write these files -* `--add-metadata` attaches the `infojson` to `mkv` files in addition to writing the metadata when used with `--write-info-json`. Use `--no-embed-info-json` or `--compat-options no-attach-info-json` to revert this -* Some metadata are embedded into different fields when using `--add-metadata` as compared to youtube-dl. Most notably, `comment` field contains the `webpage_url` and `synopsis` contains the `description`. You can [use `--parse-metadata`](#modifying-metadata) to modify this to your liking or use `--compat-options embed-metadata` to revert this -* `playlist_index` behaves differently when used with options like `--playlist-reverse` and `--playlist-items`. See [#302](https://github.com/yt-dlp/yt-dlp/issues/302) for details. You can use `--compat-options playlist-index` if you want to keep the earlier behavior -* The output of `-F` is listed in a new format. Use `--compat-options list-formats` to revert this -* All *experiences* of a funimation episode are considered as a single video. This behavior breaks existing archives. Use `--compat-options seperate-video-versions` to extract information from only the default player -* Youtube live chat (if available) is considered as a subtitle. Use `--sub-langs all,-live_chat` to download all subtitles except live chat. You can also use `--compat-options no-live-chat` to prevent live chat from downloading -* Youtube channel URLs are automatically redirected to `/video`. Append a `/featured` to the URL to download only the videos in the home page. If the channel does not have a videos tab, we try to download the equivalent `UU` playlist instead. For all other tabs, if the channel does not show the requested tab, an error will be raised. Also, `/live` URLs raise an error if there are no live videos instead of silently downloading the entire channel. You may use `--compat-options no-youtube-channel-redirect` to revert all these redirections -* Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this -* If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this -* Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead -* Some private fields such as filenames are removed by default from the infojson. Use `--no-clean-infojson` or `--compat-options no-clean-infojson` to revert this -* When `--embed-subs` and `--write-subs` are used together, the subtitles are written to disk and also embedded in the media file. You can use just `--embed-subs` to embed the subs and automatically delete the separate file. See [#630 (comment)](https://github.com/yt-dlp/yt-dlp/issues/630#issuecomment-893659460) for more info. `--compat-options no-keep-subs` can be used to revert this -* `certifi` will be used for SSL root certificates, if installed. If you want to use only system certificates, use `--compat-options no-certifi` -* youtube-dl tries to remove some superfluous punctuations from filenames. While this can sometimes be helpfull, it is often undesirable. So yt-dlp tries to keep the fields in the filenames as close to their original values as possible. You can use `--compat-options filename-sanitization` to revert to youtube-dl's behavior - -For ease of use, a few more compat options are available: - -* `--compat-options all`: Use all compat options (Do NOT use) -* `--compat-options youtube-dl`: Same as `--compat-options all,-multistreams` -* `--compat-options youtube-dlc`: Same as `--compat-options all,-no-live-chat,-no-youtube-channel-redirect` - +hypervideo - A fork of youtube-dl without nonfree parts + +- [INSTALLATION](#installation) +- [DESCRIPTION](#description) +- [OPTIONS](#options) +- [CONFIGURATION](#configuration) +- [OUTPUT TEMPLATE](#output-template) +- [FORMAT SELECTION](#format-selection) +- [VIDEO SELECTION](#video-selection) +- [FAQ](#faq) +- [DEVELOPER INSTRUCTIONS](#developer-instructions) +- [EMBEDDING YOUTUBE-DL](#embedding-hypervideo) +- [BUGS](#bugs) +- [COPYRIGHT](#copyright) # INSTALLATION -You can install yt-dlp using one of the following methods: - -### Using the release binary - -You can simply download the [correct binary file](#release-files) for your OS +A fork of [youtube-dl](https://ytdl-org.github.io/youtube-dl/) without nonfree parts -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> -[](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe) -[](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp) -[](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos) -[](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz) -[](#release-files) -[](https://github.com/yt-dlp/yt-dlp/releases) -<!-- MANPAGE: END EXCLUDED SECTION --> - -Note: The manpages, shell completion files etc. are available in the [source tarball](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz) - -<!-- TODO: Move to Wiki --> -In UNIX-like OSes (MacOS, Linux, BSD), you can also install the same in one of the following ways: - -``` -sudo curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp -sudo chmod a+rx /usr/local/bin/yt-dlp -``` +Hypervideo is distributed for [Hyperbola GNU/Linux-libre](https://www.hyperbola.info/). -``` -sudo wget https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -O /usr/local/bin/yt-dlp -sudo chmod a+rx /usr/local/bin/yt-dlp -``` +To install on Hyperbola run: +```console +# pacman -S hypervideo ``` -sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp --dir /usr/local/bin -o yt-dlp -sudo chmod a+rx /usr/local/bin/yt-dlp -``` - -### With [PIP](https://pypi.org/project/pip) +# DESCRIPTION +**hypervideo** A fork of youtube-dl without nonfree parts -You can install the [PyPI package](https://pypi.org/project/yt-dlp) with: -``` -python3 -m pip install -U yt-dlp -``` - -You can install without any of the optional dependencies using: -``` -python3 -m pip install --no-deps -U yt-dlp +```console +$ hypervideo [OPTIONS] URL [URL...] ``` -If you want to be on the cutting edge, you can also install the master branch with: -``` -python3 -m pip install --force-reinstall https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz -``` - -On some systems, you may need to use `py` or `python` instead of `python3` - -<!-- TODO: Add to Wiki, Remove Taps --> -### With [Homebrew](https://brew.sh) - -macOS or Linux users that are using Homebrew can also install it by: - -``` -brew install yt-dlp/taps/yt-dlp -``` - -## UPDATE -You can use `yt-dlp -U` to update if you are [using the provided release](#using-the-release-binary) - -If you [installed with pip](#with-pip), simply re-run the same command that was used to install the program - -If you [installed using Homebrew](#with-homebrew), run `brew upgrade yt-dlp/taps/yt-dlp` - -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> -## RELEASE FILES - -#### Recommended - -File|Description -:---|:--- -[yt-dlp](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp)|Platform-independant [zipimport](https://docs.python.org/3/library/zipimport.html) binary. Needs Python (recommended for **Linux/BSD**) -[yt-dlp.exe](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.exe)|Windows (Win7 SP1+) standalone x64 binary (recommended for **Windows**) -[yt-dlp_macos](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos)|MacOS (10.15+) standalone executable (recommended for **MacOS**) - -#### Alternatives - -File|Description -:---|:--- -[yt-dlp_x86.exe](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_x86.exe)|Windows (Vista SP2+) standalone x86 (32-bit) binary -[yt-dlp_min.exe](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_min.exe)|Windows (Win7 SP1+) standalone x64 binary built with `py2exe`<br/> ([Not recommended](#standalone-py2exe-builds-windows)) -[yt-dlp_linux](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux)|UNIX standalone x64 binary -[yt-dlp_linux.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux.zip)|Unpackaged Unix executable (no auto-update) -[yt-dlp_win.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_win.zip)|Unpackaged Windows executable (no auto-update) -[yt-dlp_macos.zip](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_macos.zip)|Unpackaged MacOS (10.15+) executable (no auto-update) - -#### Misc - -File|Description -:---|:--- -[yt-dlp.tar.gz](https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp.tar.gz)|Source tarball. Also contains manpages, completions, etc -[SHA2-512SUMS](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-512SUMS)|GNU-style SHA512 sums -[SHA2-256SUMS](https://github.com/yt-dlp/yt-dlp/releases/latest/download/SHA2-256SUMS)|GNU-style SHA256 sums -<!-- MANPAGE: END EXCLUDED SECTION --> - -## DEPENDENCIES -Python versions 3.7+ (CPython and PyPy) are supported. Other versions and implementations may or may not work correctly. - -<!-- Python 3.5+ uses VC++14 and it is already embedded in the binary created -<!x-- https://www.microsoft.com/en-us/download/details.aspx?id=26999 --x> -On windows, [Microsoft Visual C++ 2010 SP1 Redistributable Package (x86)](https://download.microsoft.com/download/1/6/5/165255E7-1014-4D0A-B094-B6A430A6BFFC/vcredist_x86.exe) is also necessary to run yt-dlp. You probably already have this, but if the executable throws an error due to missing `MSVCR100.dll` you need to install it manually. ---> - -While all the other dependencies are optional, `ffmpeg` and `ffprobe` are highly recommended - -### Strongly recommended - -* [**ffmpeg** and **ffprobe**](https://www.ffmpeg.org) - Required for [merging separate video and audio files](#format-selection) as well as for various [post-processing](#post-processing-options) tasks. License [depends on the build](https://www.ffmpeg.org/legal.html) - - <!-- TODO: ffmpeg has merged this patch. Remove this note once there is new release --> - **Note**: There are some regressions in newer ffmpeg versions that causes various issues when used alongside yt-dlp. Since ffmpeg is such an important dependency, we provide [custom builds](https://github.com/yt-dlp/FFmpeg-Builds#ffmpeg-static-auto-builds) with patches for these issues at [yt-dlp/FFmpeg-Builds](https://github.com/yt-dlp/FFmpeg-Builds). See [the readme](https://github.com/yt-dlp/FFmpeg-Builds#patches-applied) for details on the specific issues solved by these builds - -### Networking -* [**certifi**](https://github.com/certifi/python-certifi)\* - Provides Mozilla's root certificate bundle. Licensed under [MPLv2](https://github.com/certifi/python-certifi/blob/master/LICENSE) -* [**brotli**](https://github.com/google/brotli)\* or [**brotlicffi**](https://github.com/python-hyper/brotlicffi) - [Brotli](https://en.wikipedia.org/wiki/Brotli) content encoding support. Both licensed under MIT <sup>[1](https://github.com/google/brotli/blob/master/LICENSE) [2](https://github.com/python-hyper/brotlicffi/blob/master/LICENSE) </sup> -* [**websockets**](https://github.com/aaugustin/websockets)\* - For downloading over websocket. Licensed under [BSD-3-Clause](https://github.com/aaugustin/websockets/blob/main/LICENSE) - -### Metadata - -* [**mutagen**](https://github.com/quodlibet/mutagen)\* - For `--embed-thumbnail` in certain formats. Licensed under [GPLv2+](https://github.com/quodlibet/mutagen/blob/master/COPYING) -* [**AtomicParsley**](https://github.com/wez/atomicparsley) - For `--embed-thumbnail` in `mp4`/`m4a` files when `mutagen`/`ffmpeg` cannot. Licensed under [GPLv2+](https://github.com/wez/atomicparsley/blob/master/COPYING) -* [**xattr**](https://github.com/xattr/xattr), [**pyxattr**](https://github.com/iustin/pyxattr) or [**setfattr**](http://savannah.nongnu.org/projects/attr) - For writing xattr metadata (`--xattr`) on **Linux**. Licensed under [MIT](https://github.com/xattr/xattr/blob/master/LICENSE.txt), [LGPL2.1](https://github.com/iustin/pyxattr/blob/master/COPYING) and [GPLv2+](http://git.savannah.nongnu.org/cgit/attr.git/tree/doc/COPYING) respectively - -### Misc - -* [**pycryptodomex**](https://github.com/Legrandin/pycryptodome)\* - For decrypting AES-128 HLS streams and various other data. Licensed under [BSD-2-Clause](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst) -* [**phantomjs**](https://github.com/ariya/phantomjs) - Used in extractors where javascript needs to be run. Licensed under [BSD-3-Clause](https://github.com/ariya/phantomjs/blob/master/LICENSE.BSD) -* [**secretstorage**](https://github.com/mitya57/secretstorage) - For `--cookies-from-browser` to access the **Gnome** keyring while decrypting cookies of **Chromium**-based browsers on **Linux**. Licensed under [BSD-3-Clause](https://github.com/mitya57/secretstorage/blob/master/LICENSE) -* Any external downloader that you want to use with `--downloader` - -#### Deprecated - -* [**avconv** and **avprobe**](https://www.libav.org) - Now **deprecated** alternative to ffmpeg. License [depends on the build](https://libav.org/legal) -* [**sponskrub**](https://github.com/faissaloo/SponSkrub) - For using the now **deprecated** [sponskrub options](#sponskrub-options). Licensed under [GPLv3+](https://github.com/faissaloo/SponSkrub/blob/master/LICENCE.md) -* [**rtmpdump**](http://rtmpdump.mplayerhq.hu) - For downloading `rtmp` streams. ffmpeg can be used instead with `--downloader ffmpeg`. Licensed under [GPLv2+](http://rtmpdump.mplayerhq.hu) -* [**mplayer**](http://mplayerhq.hu/design7/info.html) or [**mpv**](https://mpv.io) - For downloading `rstp`/`mms` streams. ffmpeg can be used instead with `--downloader ffmpeg`. Licensed under [GPLv2+](https://github.com/mpv-player/mpv/blob/master/Copyright) - -To use or redistribute the dependencies, you must agree to their respective licensing terms. - -The Windows and MacOS standalone release binaries are built with the Python interpreter and the packages marked with **\*** included. - -If you do not have the necessary dependencies for a task you are attempting, yt-dlp will warn you. All the currently available dependencies are visible at the top of the `--verbose` output - - -## COMPILE - -### Standalone PyInstaller Builds -To build the Windows/MacOS executable, you must have Python and `pyinstaller` (plus any of yt-dlp's [optional dependencies](#dependencies) if needed). Once you have all the necessary dependencies installed, simply run `pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the Python used. - - python3 -m pip install -U pyinstaller -r requirements.txt - python3 devscripts/make_lazy_extractors.py - python3 pyinst.py - -On some systems, you may need to use `py` or `python` instead of `python3`. - -Note that pyinstaller [does not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment. - -**Important**: Running `pyinstaller` directly **without** using `pyinst.py` is **not** officially supported. This may or may not work correctly. - -### Platform-independent Binary (UNIX) -You will need the build tools `python` (3.6+), `zip`, `make` (GNU), `pandoc`\* and `pytest`\*. - -After installing these, simply run `make`. - -You can also run `make yt-dlp` instead to compile only the binary without updating any of the additional files. (The dependencies marked with **\*** are not needed for this) - -### Standalone Py2Exe Builds (Windows) - -While we provide the option to build with [py2exe](https://www.py2exe.org), it is recommended to build [using PyInstaller](#standalone-pyinstaller-builds) instead since the py2exe builds **cannot contain `pycryptodomex`/`certifi` and needs VC++14** on the target computer to run. - -If you wish to build it anyway, install Python and py2exe, and then simply run `setup.py py2exe` - - py -m pip install -U py2exe -r requirements.txt - py devscripts/make_lazy_extractors.py - py setup.py py2exe - -### Related scripts - -* **`devscripts/update-version.py`** - Update the version number based on current timestamp -* **`devscripts/make_lazy_extractors.py`** - Create lazy extractors. Running this before building the binaries (any variant) will improve their startup performance. Set the environment variable `YTDLP_NO_LAZY_EXTRACTORS=1` if you wish to forcefully disable lazy extractor loading. - -You can also fork the project on github and run your fork's [build workflow](.github/workflows/build.yml) to automatically build a full release - -# USAGE AND OPTIONS - -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> - yt-dlp [OPTIONS] [--] URL [URL...] - -`Ctrl+F` is your friend :D -<!-- MANPAGE: END EXCLUDED SECTION --> - -<!-- Auto generated --> -## General Options: - -h, --help Print this help text and exit - --version Print program version and exit - -U, --update Update this program to latest version - --no-update Do not update (default) - -i, --ignore-errors Ignore download and postprocessing errors. - The download will be considered successful - even if the postprocessing fails - --no-abort-on-error Continue with next video on download errors; - e.g. to skip unavailable videos in a - playlist (default) - --abort-on-error Abort downloading of further videos if an - error occurs (Alias: --no-ignore-errors) - --dump-user-agent Display the current user-agent and exit - --list-extractors List all supported extractors and exit - --extractor-descriptions Output descriptions of all supported - extractors and exit - --force-generic-extractor Force extraction to use the generic extractor - --default-search PREFIX Use this prefix for unqualified URLs. Eg: - "gvsearch2:python" downloads two videos from - google videos for the search term "python". - Use the value "auto" to let yt-dlp guess - ("auto_warning" to emit a warning when - guessing). "error" just throws an error. The - default value "fixup_error" repairs broken - URLs, but emits an error if this is not - possible instead of searching - --ignore-config Don't load any more configuration files - except those given by --config-locations. - For backward compatibility, if this option - is found inside the system configuration - file, the user configuration is not loaded. - (Alias: --no-config) - --no-config-locations Do not load any custom configuration files - (default). When given inside a configuration - file, ignore all previous --config-locations - defined in the current file - --config-locations PATH Location of the main configuration file; - either the path to the config or its - containing directory ("-" for stdin). Can be - used multiple times and inside other - configuration files - --flat-playlist Do not extract the videos of a playlist, - only list them - --no-flat-playlist Extract the videos of a playlist - --live-from-start Download livestreams from the start. - Currently only supported for YouTube - (Experimental) - --no-live-from-start Download livestreams from the current time - (default) - --wait-for-video MIN[-MAX] Wait for scheduled streams to become - available. Pass the minimum number of - seconds (or range) to wait between retries - --no-wait-for-video Do not wait for scheduled streams (default) - --mark-watched Mark videos watched (even with --simulate) - --no-mark-watched Do not mark videos watched (default) - --no-colors Do not emit color codes in output - --compat-options OPTS Options that can help keep compatibility - with youtube-dl or youtube-dlc - configurations by reverting some of the - changes made in yt-dlp. See "Differences in - default behavior" for details - --alias ALIASES OPTIONS Create aliases for an option string. Unless - an alias starts with a dash "-", it is - prefixed with "--". Arguments are parsed - according to the Python string formatting - mini-language. Eg: --alias get-audio,-X - "-S=aext:{0},abr -x --audio-format {0}" - creates options "--get-audio" and "-X" that - takes an argument (ARG0) and expands to - "-S=aext:ARG0,abr -x --audio-format ARG0". - All defined aliases are listed in the --help - output. Alias options can trigger more - aliases; so be carefull to avoid defining - recursive options. As a safety measure, each - alias may be triggered a maximum of 100 - times. This option can be used multiple times +# OPTIONS + -h, --help Print this help text and exit + --version Print program version and exit + -i, --ignore-errors Continue on download errors, for + example to skip unavailable videos in a + playlist + --abort-on-error Abort downloading of further videos (in + the playlist or the command line) if an + error occurs + --dump-user-agent Display the current browser + identification + --list-extractors List all supported extractors + --extractor-descriptions Output descriptions of all supported + extractors + --force-generic-extractor Force extraction to use the generic + extractor + --default-search PREFIX Use this prefix for unqualified URLs. + For example "gvsearch2:" downloads two + videos from google videos for youtube- + dl "large apple". Use the value "auto" + to let hypervideo guess ("auto_warning" + to emit a warning when guessing). + "error" just throws an error. The + default value "fixup_error" repairs + broken URLs, but emits an error if this + is not possible instead of searching. + --ignore-config Do not read configuration files. When + given in the global configuration file + /etc/hypervideo.conf: Do not read the + user configuration in + ~/.config/hypervideo/config + --config-location PATH Location of the configuration file; + either the path to the config or its + containing directory. + --flat-playlist Do not extract the videos of a + playlist, only list them. + --mark-watched Mark videos watched (YouTube only) + --no-mark-watched Do not mark videos watched (YouTube + only) + --no-color Do not emit color codes in output ## Network Options: - --proxy URL Use the specified HTTP/HTTPS/SOCKS proxy. To - enable SOCKS proxy, specify a proper scheme. - Eg: socks5://user:pass@127.0.0.1:1080/. Pass - in an empty string (--proxy "") for direct - connection - --socket-timeout SECONDS Time to wait before giving up, in seconds - --source-address IP Client-side IP address to bind to - -4, --force-ipv4 Make all connections via IPv4 - -6, --force-ipv6 Make all connections via IPv6 - -## Geo-restriction: - --geo-verification-proxy URL Use this proxy to verify the IP address for - some geo-restricted sites. The default proxy - specified by --proxy (or none, if the option - is not present) is used for the actual - downloading - --geo-bypass Bypass geographic restriction via faking - X-Forwarded-For HTTP header (default) - --no-geo-bypass Do not bypass geographic restriction via - faking X-Forwarded-For HTTP header - --geo-bypass-country CODE Force bypass geographic restriction with - explicitly provided two-letter ISO 3166-2 - country code - --geo-bypass-ip-block IP_BLOCK Force bypass geographic restriction with - explicitly provided IP block in CIDR notation + --proxy URL Use the specified HTTP/HTTPS/SOCKS + proxy. To enable SOCKS proxy, specify a + proper scheme. For example + socks5://127.0.0.1:1080/. Pass in an + empty string (--proxy "") for direct + connection + --socket-timeout SECONDS Time to wait before giving up, in + seconds + --source-address IP Client-side IP address to bind to + -4, --force-ipv4 Make all connections via IPv4 + -6, --force-ipv6 Make all connections via IPv6 + +## Geo Restriction: + --geo-verification-proxy URL Use this proxy to verify the IP address + for some geo-restricted sites. The + default proxy specified by --proxy (or + none, if the option is not present) is + used for the actual downloading. + --geo-bypass Bypass geographic restriction via + faking X-Forwarded-For HTTP header + --no-geo-bypass Do not bypass geographic restriction + via faking X-Forwarded-For HTTP header + --geo-bypass-country CODE Force bypass geographic restriction + with explicitly provided two-letter ISO + 3166-2 country code + --geo-bypass-ip-block IP_BLOCK Force bypass geographic restriction + with explicitly provided IP block in + CIDR notation ## Video Selection: - -I, --playlist-items ITEM_SPEC Comma seperated playlist_index of the videos - to download. You can specify a range using - "[START]:[STOP][:STEP]". For backward - compatibility, START-STOP is also supported. - Use negative indices to count from the right - and negative STEP to download in reverse - order. Eg: "-I 1:3,7,-5::2" used on a - playlist of size 15 will download the videos - at index 1,2,3,7,11,13,15 - --min-filesize SIZE Do not download any videos smaller than SIZE - (e.g. 50k or 44.6m) - --max-filesize SIZE Do not download any videos larger than SIZE - (e.g. 50k or 44.6m) - --date DATE Download only videos uploaded on this date. - The date can be "YYYYMMDD" or in the format - [now|today|yesterday][-N[day|week|month|year]]. - Eg: --date today-2weeks - --datebefore DATE Download only videos uploaded on or before - this date. The date formats accepted is the - same as --date - --dateafter DATE Download only videos uploaded on or after - this date. The date formats accepted is the - same as --date - --match-filters FILTER Generic video filter. Any "OUTPUT TEMPLATE" - field can be compared with a number or a - string using the operators defined in - "Filtering formats". You can also simply - specify a field to match if the field is - present, use "!field" to check if the field - is not present, and "&" to check multiple - conditions. Use a "\" to escape "&" or - quotes if needed. If used multiple times, - the filter matches if atleast one of the - conditions are met. Eg: --match-filter - !is_live --match-filter "like_count>?100 & - description~='(?i)\bcats \& dogs\b'" matches - only videos that are not live OR those that - have a like count more than 100 (or the like - field is not available) and also has a - description that contains the phrase "cats & - dogs" (caseless). Use "--match-filter -" to - interactively ask whether to download each - video - --no-match-filter Do not use generic video filter (default) - --no-playlist Download only the video, if the URL refers - to a video and a playlist - --yes-playlist Download the playlist, if the URL refers to - a video and a playlist - --age-limit YEARS Download only videos suitable for the given - age - --download-archive FILE Download only videos not listed in the - archive file. Record the IDs of all - downloaded videos in it - --no-download-archive Do not use archive file (default) - --max-downloads NUMBER Abort after downloading NUMBER files - --break-on-existing Stop the download process when encountering - a file that is in the archive - --break-on-reject Stop the download process when encountering - a file that has been filtered out - --break-per-input Make --break-on-existing, --break-on-reject - and --max-downloads act only on the current - input URL - --no-break-per-input --break-on-existing and similar options - terminates the entire download queue - --skip-playlist-after-errors N Number of allowed failures until the rest of - the playlist is skipped + --playlist-start NUMBER Playlist video to start at (default is + 1) + --playlist-end NUMBER Playlist video to end at (default is + last) + --playlist-items ITEM_SPEC Playlist video items to download. + Specify indices of the videos in the + playlist separated by commas like: "-- + playlist-items 1,2,5,8" if you want to + download videos indexed 1, 2, 5, 8 in + the playlist. You can specify range: " + --playlist-items 1-3,7,10-13", it will + download the videos at index 1, 2, 3, + 7, 10, 11, 12 and 13. + --match-title REGEX Download only matching titles (regex or + caseless sub-string) + --reject-title REGEX Skip download for matching titles + (regex or caseless sub-string) + --max-downloads NUMBER Abort after downloading NUMBER files + --min-filesize SIZE Do not download any videos smaller than + SIZE (e.g. 50k or 44.6m) + --max-filesize SIZE Do not download any videos larger than + SIZE (e.g. 50k or 44.6m) + --date DATE Download only videos uploaded in this + date + --datebefore DATE Download only videos uploaded on or + before this date (i.e. inclusive) + --dateafter DATE Download only videos uploaded on or + after this date (i.e. inclusive) + --min-views COUNT Do not download any videos with less + than COUNT views + --max-views COUNT Do not download any videos with more + than COUNT views + --match-filter FILTER Generic video filter. Specify any key + (see the "OUTPUT TEMPLATE" for a list + of available keys) to match if the key + is present, !key to check if the key is + not present, key > NUMBER (like + "comment_count > 12", also works with + >=, <, <=, !=, =) to compare against a + number, key = 'LITERAL' (like "uploader + = 'Mike Smith'", also works with !=) to + match against a string literal and & to + require multiple matches. Values which + are not known are excluded unless you + put a question mark (?) after the + operator. For example, to only match + videos that have been liked more than + 100 times and disliked less than 50 + times (or the dislike functionality is + not available at the given service), + but who also have a description, use + --match-filter "like_count > 100 & + dislike_count <? 50 & description" . + --no-playlist Download only the video, if the URL + refers to a video and a playlist. + --yes-playlist Download the playlist, if the URL + refers to a video and a playlist. + --age-limit YEARS Download only videos suitable for the + given age + --download-archive FILE Download only videos not listed in the + archive file. Record the IDs of all + downloaded videos in it. + --include-ads Download advertisements as well + (experimental) ## Download Options: - -N, --concurrent-fragments N Number of fragments of a dash/hlsnative - video that should be downloaded concurrently - (default is 1) - -r, --limit-rate RATE Maximum download rate in bytes per second - (e.g. 50K or 4.2M) - --throttled-rate RATE Minimum download rate in bytes per second - below which throttling is assumed and the - video data is re-extracted (e.g. 100K) - -R, --retries RETRIES Number of retries (default is 10), or - "infinite" - --file-access-retries RETRIES Number of times to retry on file access - error (default is 3), or "infinite" - --fragment-retries RETRIES Number of retries for a fragment (default is - 10), or "infinite" (DASH, hlsnative and ISM) - --retry-sleep [TYPE:]EXPR An expression for the time to sleep between - retries in seconds (optionally) prefixed by - the type of retry (file_access, fragment, - http (default)) to apply the sleep to. EXPR - can be a number, linear=START[:END[:STEP=1]] - or exp=START[:END[:BASE=2]]. This option can - be used multiple times to set the sleep for - the different retry types. Eg: --retry-sleep - linear=1::2 --retry-sleep fragment:exp=1:20 - --skip-unavailable-fragments Skip unavailable fragments for DASH, - hlsnative and ISM downloads (default) - (Alias: --no-abort-on-unavailable-fragment) - --abort-on-unavailable-fragment - Abort download if a fragment is unavailable - (Alias: --no-skip-unavailable-fragments) - --keep-fragments Keep downloaded fragments on disk after - downloading is finished - --no-keep-fragments Delete downloaded fragments after - downloading is finished (default) - --buffer-size SIZE Size of download buffer (e.g. 1024 or 16K) - (default is 1024) - --resize-buffer The buffer size is automatically resized - from an initial value of --buffer-size - (default) - --no-resize-buffer Do not automatically adjust the buffer size - --http-chunk-size SIZE Size of a chunk for chunk-based HTTP - downloading (e.g. 10485760 or 10M) (default - is disabled). May be useful for bypassing - bandwidth throttling imposed by a webserver - (experimental) - --playlist-random Download playlist videos in random order - --lazy-playlist Process entries in the playlist as they are - received. This disables n_entries, - --playlist-random and --playlist-reverse - --no-lazy-playlist Process videos in the playlist only after - the entire playlist is parsed (default) - --xattr-set-filesize Set file xattribute ytdl.filesize with - expected file size - --hls-use-mpegts Use the mpegts container for HLS videos; - allowing some players to play the video - while downloading, and reducing the chance - of file corruption if download is - interrupted. This is enabled by default for - live streams - --no-hls-use-mpegts Do not use the mpegts container for HLS - videos. This is default when not downloading - live streams - --download-sections REGEX Download only chapters whose title matches - the given regular expression. Time ranges - prefixed by a "*" can also be used in place - of chapters to download the specified range. - Eg: --download-sections "*10:15-15:00" - --download-sections "intro". Needs ffmpeg. - This option can be used multiple times to - download multiple sections - --downloader [PROTO:]NAME Name or path of the external downloader to - use (optionally) prefixed by the protocols - (http, ftp, m3u8, dash, rstp, rtmp, mms) to - use it for. Currently supports native, - aria2c, avconv, axel, curl, ffmpeg, httpie, - wget. You can use this option multiple times - to set different downloaders for different - protocols. For example, --downloader aria2c - --downloader "dash,m3u8:native" will use - aria2c for http/ftp downloads, and the - native downloader for dash/m3u8 downloads - (Alias: --external-downloader) - --downloader-args NAME:ARGS Give these arguments to the external - downloader. Specify the downloader name and - the arguments separated by a colon ":". For - ffmpeg, arguments can be passed to different - positions using the same syntax as - --postprocessor-args. You can use this - option multiple times to give different - arguments to different downloaders (Alias: - --external-downloader-args) + -r, --limit-rate RATE Maximum download rate in bytes per + second (e.g. 50K or 4.2M) + -R, --retries RETRIES Number of retries (default is 10), or + "infinite". + --fragment-retries RETRIES Number of retries for a fragment + (default is 10), or "infinite" (DASH, + hlsnative and ISM) + --skip-unavailable-fragments Skip unavailable fragments (DASH, + hlsnative and ISM) + --abort-on-unavailable-fragment Abort downloading when some fragment is + not available + --keep-fragments Keep downloaded fragments on disk after + downloading is finished; fragments are + erased by default + --buffer-size SIZE Size of download buffer (e.g. 1024 or + 16K) (default is 1024) + --no-resize-buffer Do not automatically adjust the buffer + size. By default, the buffer size is + automatically resized from an initial + value of SIZE. + --http-chunk-size SIZE Size of a chunk for chunk-based HTTP + downloading (e.g. 10485760 or 10M) + (default is disabled). May be useful + for bypassing bandwidth throttling + imposed by a webserver (experimental) + --playlist-reverse Download playlist videos in reverse + order + --playlist-random Download playlist videos in random + order + --xattr-set-filesize Set file xattribute ytdl.filesize with + expected file size + --hls-prefer-native Use the native HLS downloader instead + of ffmpeg + --hls-prefer-ffmpeg Use ffmpeg instead of the native HLS + downloader + --hls-use-mpegts Use the mpegts container for HLS + videos, allowing to play the video + while downloading (some players may not + be able to play it) + --external-downloader COMMAND Use the specified external downloader. + Currently supports aria2c,avconv,axel,c + url,ffmpeg,httpie,wget + --external-downloader-args ARGS Give these arguments to the external + downloader ## Filesystem Options: - -a, --batch-file FILE File containing URLs to download ("-" for - stdin), one URL per line. Lines starting - with "#", ";" or "]" are considered as - comments and ignored - --no-batch-file Do not read URLs from batch file (default) - -P, --paths [TYPES:]PATH The paths where the files should be - downloaded. Specify the type of file and the - path separated by a colon ":". All the same - TYPES as --output are supported. - Additionally, you can also provide "home" - (default) and "temp" paths. All intermediary - files are first downloaded to the temp path - and then the final files are moved over to - the home path after download is finished. - This option is ignored if --output is an - absolute path - -o, --output [TYPES:]TEMPLATE Output filename template; see "OUTPUT - TEMPLATE" for details - --output-na-placeholder TEXT Placeholder for unavailable fields in - "OUTPUT TEMPLATE" (default: "NA") - --restrict-filenames Restrict filenames to only ASCII characters, - and avoid "&" and spaces in filenames - --no-restrict-filenames Allow Unicode characters, "&" and spaces in - filenames (default) - --windows-filenames Force filenames to be Windows-compatible - --no-windows-filenames Make filenames Windows-compatible only if - using Windows (default) - --trim-filenames LENGTH Limit the filename length (excluding - extension) to the specified number of - characters - -w, --no-overwrites Do not overwrite any files - --force-overwrites Overwrite all video and metadata files. This - option includes --no-continue - --no-force-overwrites Do not overwrite the video, but overwrite - related files (default) - -c, --continue Resume partially downloaded files/fragments - (default) - --no-continue Do not resume partially downloaded - fragments. If the file is not fragmented, - restart download of the entire file - --part Use .part files instead of writing directly - into output file (default) - --no-part Do not use .part files - write directly into - output file - --mtime Use the Last-modified header to set the file - modification time (default) - --no-mtime Do not use the Last-modified header to set - the file modification time - --write-description Write video description to a .description file - --no-write-description Do not write video description (default) - --write-info-json Write video metadata to a .info.json file - (this may contain personal information) - --no-write-info-json Do not write video metadata (default) - --write-playlist-metafiles Write playlist metadata in addition to the - video metadata when using --write-info-json, - --write-description etc. (default) - --no-write-playlist-metafiles Do not write playlist metadata when using - --write-info-json, --write-description etc. - --clean-info-json Remove some private fields such as filenames - from the infojson. Note that it could still - contain some personal information (default) - --no-clean-info-json Write all fields to the infojson - --write-comments Retrieve video comments to be placed in the - infojson. The comments are fetched even - without this option if the extraction is - known to be quick (Alias: --get-comments) - --no-write-comments Do not retrieve video comments unless the - extraction is known to be quick (Alias: - --no-get-comments) - --load-info-json FILE JSON file containing the video information - (created with the "--write-info-json" option) - --cookies FILE Netscape formatted file to read cookies from - and dump cookie jar in - --no-cookies Do not read/dump cookies from/to file - (default) - --cookies-from-browser BROWSER[+KEYRING][:PROFILE] - The name of the browser and (optionally) the - name/path of the profile to load cookies - from, separated by a ":". Currently - supported browsers are: brave, chrome, - chromium, edge, firefox, opera, safari, - vivaldi. By default, the most recently - accessed profile is used. The keyring used - for decrypting Chromium cookies on Linux can - be (optionally) specified after the browser - name separated by a "+". Currently supported - keyrings are: basictext, gnomekeyring, kwallet - --no-cookies-from-browser Do not load cookies from browser (default) - --cache-dir DIR Location in the filesystem where youtube-dl - can store some downloaded information (such - as client ids and signatures) permanently. - By default $XDG_CACHE_HOME/yt-dlp or - ~/.cache/yt-dlp - --no-cache-dir Disable filesystem caching - --rm-cache-dir Delete all filesystem cache files + -a, --batch-file FILE File containing URLs to download ('-' + for stdin), one URL per line. Lines + starting with '#', ';' or ']' are + considered as comments and ignored. + --id Use only video ID in file name + -o, --output TEMPLATE Output filename template, see the + "OUTPUT TEMPLATE" for all the info + --output-na-placeholder PLACEHOLDER Placeholder value for unavailable meta + fields in output filename template + (default is "NA") + --autonumber-start NUMBER Specify the start value for + %(autonumber)s (default is 1) + --restrict-filenames Restrict filenames to only ASCII + characters, and avoid "&" and spaces in + filenames + -w, --no-overwrites Do not overwrite files + -c, --continue Force resume of partially downloaded + files. By default, hypervideo will + resume downloads if possible. + --no-continue Do not resume partially downloaded + files (restart from beginning) + --no-part Do not use .part files - write directly + into output file + --no-mtime Do not use the Last-modified header to + set the file modification time + --write-description Write video description to a + .description file + --write-info-json Write video metadata to a .info.json + file + --write-annotations Write video annotations to a + .annotations.xml file + --load-info-json FILE JSON file containing the video + information (created with the "--write- + info-json" option) + --cookies FILE File to read cookies from and dump + cookie jar in + --cache-dir DIR Location in the filesystem where + hypervideo can store some downloaded + information permanently. By default + $XDG_CACHE_HOME/hypervideo or + ~/.cache/hypervideo . At the moment, + only YouTube player files (for videos + with obfuscated signatures) are cached, + but that may change. + --no-cache-dir Disable filesystem caching + --rm-cache-dir Delete all filesystem cache files ## Thumbnail Options: - --write-thumbnail Write thumbnail image to disk - --no-write-thumbnail Do not write thumbnail image to disk (default) - --write-all-thumbnails Write all thumbnail image formats to disk - --list-thumbnails List available thumbnails of each video. - Simulate unless --no-simulate is used - -## Internet Shortcut Options: - --write-link Write an internet shortcut file, depending - on the current platform (.url, .webloc or - .desktop). The URL may be cached by the OS - --write-url-link Write a .url Windows internet shortcut. The - OS caches the URL based on the file path - --write-webloc-link Write a .webloc macOS internet shortcut - --write-desktop-link Write a .desktop Linux internet shortcut - -## Verbosity and Simulation Options: - -q, --quiet Activate quiet mode. If used with --verbose, - print the log to stderr - --no-warnings Ignore warnings - -s, --simulate Do not download the video and do not write - anything to disk - --no-simulate Download the video even if printing/listing - options are used - --ignore-no-formats-error Ignore "No video formats" error. Useful for - extracting metadata even if the videos are - not actually available for download - (experimental) - --no-ignore-no-formats-error Throw error when no downloadable video - formats are found (default) - --skip-download Do not download the video but write all - related files (Alias: --no-download) - -O, --print [WHEN:]TEMPLATE Field name or output template to print to - screen, optionally prefixed with when to - print it, separated by a ":". Supported - values of "WHEN" are the same as that of - --use-postprocessor, and "video" (default). - Implies --quiet. Implies --simulate unless - --no-simulate or later stages of WHEN are - used. This option can be used multiple times - --print-to-file [WHEN:]TEMPLATE FILE - Append given template to the file. The - values of WHEN and TEMPLATE are same as that - of --print. FILE uses the same syntax as the - output template. This option can be used - multiple times - -j, --dump-json Quiet, but print JSON information for each - video. Simulate unless --no-simulate is - used. See "OUTPUT TEMPLATE" for a - description of available keys - -J, --dump-single-json Quiet, but print JSON information for each - url or infojson passed. Simulate unless - --no-simulate is used. If the URL refers to - a playlist, the whole playlist information - is dumped in a single line - --force-write-archive Force download archive entries to be written - as far as no errors occur, even if -s or - another simulation option is used (Alias: - --force-download-archive) - --newline Output progress bar as new lines - --no-progress Do not print progress bar - --progress Show progress bar, even if in quiet mode - --console-title Display progress in console titlebar - --progress-template [TYPES:]TEMPLATE - Template for progress outputs, optionally - prefixed with one of "download:" (default), - "download-title:" (the console title), - "postprocess:", or "postprocess-title:". - The video's fields are accessible under the - "info" key and the progress attributes are - accessible under "progress" key. E.g.: - --console-title --progress-template - "download-title:%(info.id)s-%(progress.eta)s" - -v, --verbose Print various debugging information - --dump-pages Print downloaded pages encoded using base64 - to debug problems (very verbose) - --write-pages Write downloaded intermediary pages to files - in the current directory to debug problems - --print-traffic Display sent and read HTTP traffic + --write-thumbnail Write thumbnail image to disk + --write-all-thumbnails Write all thumbnail image formats to + disk + --list-thumbnails Simulate and list all available + thumbnail formats + +## Verbosity / Simulation Options: + -q, --quiet Activate quiet mode + --no-warnings Ignore warnings + -s, --simulate Do not download the video and do not + write anything to disk + --skip-download Do not download the video + -g, --get-url Simulate, quiet but print URL + -e, --get-title Simulate, quiet but print title + --get-id Simulate, quiet but print id + --get-thumbnail Simulate, quiet but print thumbnail URL + --get-description Simulate, quiet but print video + description + --get-duration Simulate, quiet but print video length + --get-filename Simulate, quiet but print output + filename + --get-format Simulate, quiet but print output format + -j, --dump-json Simulate, quiet but print JSON + information. See the "OUTPUT TEMPLATE" + for a description of available keys. + -J, --dump-single-json Simulate, quiet but print JSON + information for each command-line + argument. If the URL refers to a + playlist, dump the whole playlist + information in a single line. + --print-json Be quiet and print the video + information as JSON (video is still + being downloaded). + --newline Output progress bar as new lines + --no-progress Do not print progress bar + --console-title Display progress in console titlebar + -v, --verbose Print various debugging information + --dump-pages Print downloaded pages encoded using + base64 to debug problems (very verbose) + --write-pages Write downloaded intermediary pages to + files in the current directory to debug + problems + --print-traffic Display sent and read HTTP traffic + -C, --call-home Contact the hypervideo server for + debugging + --no-call-home Do NOT contact the hypervideo server + for debugging ## Workarounds: - --encoding ENCODING Force the specified encoding (experimental) - --legacy-server-connect Explicitly allow HTTPS connection to servers - that do not support RFC 5746 secure - renegotiation - --no-check-certificates Suppress HTTPS certificate validation - --prefer-insecure Use an unencrypted connection to retrieve - information about the video (Currently - supported only for YouTube) - --add-header FIELD:VALUE Specify a custom HTTP header and its value, - separated by a colon ":". You can use this - option multiple times - --bidi-workaround Work around terminals that lack - bidirectional text support. Requires bidiv - or fribidi executable in PATH - --sleep-requests SECONDS Number of seconds to sleep between requests - during data extraction - --sleep-interval SECONDS Number of seconds to sleep before each - download. This is the minimum time to sleep - when used along with --max-sleep-interval - (Alias: --min-sleep-interval) - --max-sleep-interval SECONDS Maximum number of seconds to sleep. Can only - be used along with --min-sleep-interval - --sleep-subtitles SECONDS Number of seconds to sleep before each - subtitle download + --encoding ENCODING Force the specified encoding + (experimental) + --no-check-certificate Suppress HTTPS certificate validation + --prefer-insecure Use an unencrypted connection to + retrieve information about the video. + (Currently supported only for YouTube) + --user-agent UA Specify a custom user agent + --referer URL Specify a custom referer, use if the + video access is restricted to one + domain + --add-header FIELD:VALUE Specify a custom HTTP header and its + value, separated by a colon ':'. You + can use this option multiple times + --bidi-workaround Work around terminals that lack + bidirectional text support. Requires + bidiv or fribidi executable in PATH + --sleep-interval SECONDS Number of seconds to sleep before each + download when used alone or a lower + bound of a range for randomized sleep + before each download (minimum possible + number of seconds to sleep) when used + along with --max-sleep-interval. + --max-sleep-interval SECONDS Upper bound of a range for randomized + sleep before each download (maximum + possible number of seconds to sleep). + Must only be used along with --min- + sleep-interval. ## Video Format Options: - -f, --format FORMAT Video format code, see "FORMAT SELECTION" - for more details - -S, --format-sort SORTORDER Sort the formats by the fields given, see - "Sorting Formats" for more details - --format-sort-force Force user specified sort order to have - precedence over all fields, see "Sorting - Formats" for more details (Alias: --S-force) - --no-format-sort-force Some fields have precedence over the user - specified sort order (default) - --video-multistreams Allow multiple video streams to be merged - into a single file - --no-video-multistreams Only one video stream is downloaded for each - output file (default) - --audio-multistreams Allow multiple audio streams to be merged - into a single file - --no-audio-multistreams Only one audio stream is downloaded for each - output file (default) - --prefer-free-formats Prefer video formats with free containers - over non-free ones of same quality. Use with - "-S ext" to strictly prefer free containers - irrespective of quality - --no-prefer-free-formats Don't give any special preference to free - containers (default) - --check-formats Make sure formats are selected only from - those that are actually downloadable - --check-all-formats Check all formats for whether they are - actually downloadable - --no-check-formats Do not check that the formats are actually - downloadable - -F, --list-formats List available formats of each video. - Simulate unless --no-simulate is used - --merge-output-format FORMAT If a merge is required (e.g. - bestvideo+bestaudio), output to given - container format. One of mkv, mp4, ogg, - webm, flv. Ignored if no merge is required + -f, --format FORMAT Video format code, see the "FORMAT + SELECTION" for all the info + --all-formats Download all available video formats + --prefer-free-formats Prefer free video formats unless a + specific one is requested + -F, --list-formats List all available formats of requested + videos + --youtube-skip-dash-manifest Do not download the DASH manifests and + related data on YouTube videos + --merge-output-format FORMAT If a merge is required (e.g. + bestvideo+bestaudio), output to given + container format. One of mkv, mp4, ogg, + webm, flv. Ignored if no merge is + required ## Subtitle Options: - --write-subs Write subtitle file - --no-write-subs Do not write subtitle file (default) - --write-auto-subs Write automatically generated subtitle file - (Alias: --write-automatic-subs) - --no-write-auto-subs Do not write auto-generated subtitles - (default) (Alias: --no-write-automatic-subs) - --list-subs List available subtitles of each video. - Simulate unless --no-simulate is used - --sub-format FORMAT Subtitle format; accepts formats preference, - Eg: "srt" or "ass/srt/best" - --sub-langs LANGS Languages of the subtitles to download (can - be regex) or "all" separated by commas. (Eg: - --sub-langs "en.*,ja") You can prefix the - language code with a "-" to exclude it from - the requested languages. (Eg: --sub-langs - all,-live_chat) Use --list-subs for a list - of available language tags + --write-sub Write subtitle file + --write-auto-sub Write automatically generated subtitle + file (YouTube only) + --all-subs Download all the available subtitles of + the video + --list-subs List all available subtitles for the + video + --sub-format FORMAT Subtitle format, accepts formats + preference, for example: "srt" or + "ass/srt/best" + --sub-lang LANGS Languages of the subtitles to download + (optional) separated by commas, use + --list-subs for available language tags ## Authentication Options: - -u, --username USERNAME Login with this account ID - -p, --password PASSWORD Account password. If this option is left - out, yt-dlp will ask interactively - -2, --twofactor TWOFACTOR Two-factor authentication code - -n, --netrc Use .netrc authentication data - --netrc-location PATH Location of .netrc authentication data; - either the path or its containing directory. - Defaults to ~/.netrc - --video-password PASSWORD Video password (vimeo, youku) - --ap-mso MSO Adobe Pass multiple-system operator (TV - provider) identifier, use --ap-list-mso for - a list of available MSOs - --ap-username USERNAME Multiple-system operator account login - --ap-password PASSWORD Multiple-system operator account password. - If this option is left out, yt-dlp will ask - interactively - --ap-list-mso List all supported multiple-system operators - --client-certificate CERTFILE Path to client certificate file in PEM - format. May include the private key - --client-certificate-key KEYFILE - Path to private key file for client - certificate - --client-certificate-password PASSWORD - Password for client certificate private key, - if encrypted. If not provided, and the key - is encrypted, yt-dlp will ask interactively - -## Post-Processing Options: - -x, --extract-audio Convert video files to audio-only files - (requires ffmpeg and ffprobe) - --audio-format FORMAT Format to convert the audio to when -x is - used. (currently supported: best (default), - mp3, aac, m4a, opus, vorbis, flac, alac, - wav). You can specify multiple rules using - similar syntax as --remux-video - --audio-quality QUALITY Specify ffmpeg audio quality to use when - converting the audio with -x. Insert a value - between 0 (best) and 10 (worst) for VBR or a - specific bitrate like 128K (default 5) - --remux-video FORMAT Remux the video into another container if - necessary (currently supported: mp4, mkv, - flv, webm, mov, avi, mka, ogg, mp3, aac, - m4a, opus, vorbis, flac, alac, wav). If - target container does not support the - video/audio codec, remuxing will fail. You - can specify multiple rules; Eg. - "aac>m4a/mov>mp4/mkv" will remux aac to m4a, - mov to mp4 and anything else to mkv - --recode-video FORMAT Re-encode the video into another format if - necessary. The syntax and supported formats - are the same as --remux-video - --postprocessor-args NAME:ARGS Give these arguments to the postprocessors. - Specify the postprocessor/executable name - and the arguments separated by a colon ":" - to give the argument to the specified - postprocessor/executable. Supported PP are: - Merger, ModifyChapters, SplitChapters, - ExtractAudio, VideoRemuxer, VideoConvertor, - Metadata, EmbedSubtitle, EmbedThumbnail, - SubtitlesConvertor, ThumbnailsConvertor, - FixupStretched, FixupM4a, FixupM3u8, - FixupTimestamp and FixupDuration. The - supported executables are: AtomicParsley, - FFmpeg and FFprobe. You can also specify - "PP+EXE:ARGS" to give the arguments to the - specified executable only when being used by - the specified postprocessor. Additionally, - for ffmpeg/ffprobe, "_i"/"_o" can be - appended to the prefix optionally followed - by a number to pass the argument before the - specified input/output file. Eg: --ppa - "Merger+ffmpeg_i1:-v quiet". You can use - this option multiple times to give different - arguments to different postprocessors. - (Alias: --ppa) - -k, --keep-video Keep the intermediate video file on disk - after post-processing - --no-keep-video Delete the intermediate video file after - post-processing (default) - --post-overwrites Overwrite post-processed files (default) - --no-post-overwrites Do not overwrite post-processed files - --embed-subs Embed subtitles in the video (only for mp4, - webm and mkv videos) - --no-embed-subs Do not embed subtitles (default) - --embed-thumbnail Embed thumbnail in the video as cover art - --no-embed-thumbnail Do not embed thumbnail (default) - --embed-metadata Embed metadata to the video file. Also - embeds chapters/infojson if present unless - --no-embed-chapters/--no-embed-info-json are - used (Alias: --add-metadata) - --no-embed-metadata Do not add metadata to file (default) - (Alias: --no-add-metadata) - --embed-chapters Add chapter markers to the video file - (Alias: --add-chapters) - --no-embed-chapters Do not add chapter markers (default) (Alias: - --no-add-chapters) - --embed-info-json Embed the infojson as an attachment to - mkv/mka video files - --no-embed-info-json Do not embed the infojson as an attachment - to the video file - --parse-metadata FROM:TO Parse additional metadata like title/artist - from other fields; see "MODIFYING METADATA" - for details - --replace-in-metadata FIELDS REGEX REPLACE - Replace text in a metadata field using the - given regex. This option can be used - multiple times - --xattrs Write metadata to the video file's xattrs - (using dublin core and xdg standards) - --concat-playlist POLICY Concatenate videos in a playlist. One of - "never", "always", or "multi_video" - (default; only when the videos form a single - show). All the video files must have same - codecs and number of streams to be - concatable. The "pl_video:" prefix can be - used with "--paths" and "--output" to set - the output filename for the concatenated - files. See "OUTPUT TEMPLATE" for details - --fixup POLICY Automatically correct known faults of the - file. One of never (do nothing), warn (only - emit a warning), detect_or_warn (the - default; fix file if we can, warn - otherwise), force (try fixing even if file - already exists) - --ffmpeg-location PATH Location of the ffmpeg binary; either the - path to the binary or its containing directory - --exec [WHEN:]CMD Execute a command, optionally prefixed with - when to execute it (after_move if - unspecified), separated by a ":". Supported - values of "WHEN" are the same as that of - --use-postprocessor. Same syntax as the - output template can be used to pass any - field as arguments to the command. After - download, an additional field "filepath" - that contains the final path of the - downloaded file is also available, and if no - fields are passed, %(filepath)q is appended - to the end of the command. This option can - be used multiple times - --no-exec Remove any previously defined --exec - --convert-subs FORMAT Convert the subtitles to another format - (currently supported: srt, vtt, ass, lrc) - (Alias: --convert-subtitles) - --convert-thumbnails FORMAT Convert the thumbnails to another format - (currently supported: jpg, png, webp). You - can specify multiple rules using similar - syntax as --remux-video - --split-chapters Split video into multiple files based on - internal chapters. The "chapter:" prefix can - be used with "--paths" and "--output" to set - the output filename for the split files. See - "OUTPUT TEMPLATE" for details - --no-split-chapters Do not split video based on chapters (default) - --remove-chapters REGEX Remove chapters whose title matches the - given regular expression. The syntax is the - same as --download-sections. This option can - be used multiple times - --no-remove-chapters Do not remove any chapters from the file - (default) - --force-keyframes-at-cuts Force keyframes at cuts when - downloading/splitting/removing sections. - This is slow due to needing a re-encode, but - the resulting video may have fewer artifacts - around the cuts - --no-force-keyframes-at-cuts Do not force keyframes around the chapters - when cutting/splitting (default) - --use-postprocessor NAME[:ARGS] - The (case sensitive) name of plugin - postprocessors to be enabled, and - (optionally) arguments to be passed to it, - separated by a colon ":". ARGS are a - semicolon ";" delimited list of NAME=VALUE. - The "when" argument determines when the - postprocessor is invoked. It can be one of - "pre_process" (after video extraction), - "after_filter" (after video passes filter), - "before_dl" (before each video download), - "post_process" (after each video download; - default), "after_move" (after moving video - file to it's final locations), "after_video" - (after downloading and processing all - formats of a video), or "playlist" (at end - of playlist). This option can be used - multiple times to add different postprocessors - -## SponsorBlock Options: -Make chapter entries for, or remove various segments (sponsor, - introductions, etc.) from downloaded YouTube videos using the - [SponsorBlock API](https://sponsor.ajay.app) - - --sponsorblock-mark CATS SponsorBlock categories to create chapters - for, separated by commas. Available - categories are sponsor, intro, outro, - selfpromo, preview, filler, interaction, - music_offtopic, poi_highlight, all and - default (=all). You can prefix the category - with a "-" to exclude it. See [1] for - description of the categories. Eg: - --sponsorblock-mark all,-preview - [1] https://wiki.sponsor.ajay.app/w/Segment_Categories - --sponsorblock-remove CATS SponsorBlock categories to be removed from - the video file, separated by commas. If a - category is present in both mark and remove, - remove takes precedence. The syntax and - available categories are the same as for - --sponsorblock-mark except that "default" - refers to "all,-filler" and poi_highlight is - not available - --sponsorblock-chapter-title TEMPLATE - An output template for the title of the - SponsorBlock chapters created by - --sponsorblock-mark. The only available - fields are start_time, end_time, category, - categories, name, category_names. Defaults - to "[SponsorBlock]: %(category_names)l" - --no-sponsorblock Disable both --sponsorblock-mark and - --sponsorblock-remove - --sponsorblock-api URL SponsorBlock API location, defaults to - https://sponsor.ajay.app - -## Extractor Options: - --extractor-retries RETRIES Number of retries for known extractor errors - (default is 3), or "infinite" - --allow-dynamic-mpd Process dynamic DASH manifests (default) - (Alias: --no-ignore-dynamic-mpd) - --ignore-dynamic-mpd Do not process dynamic DASH manifests - (Alias: --no-allow-dynamic-mpd) - --hls-split-discontinuity Split HLS playlists to different formats at - discontinuities such as ad breaks - --no-hls-split-discontinuity Do not split HLS playlists to different - formats at discontinuities such as ad breaks - (default) - --extractor-args KEY:ARGS Pass these arguments to the extractor. See - "EXTRACTOR ARGUMENTS" for details. You can - use this option multiple times to give - arguments for different extractors + -u, --username USERNAME Login with this account ID + -p, --password PASSWORD Account password. If this option is + left out, hypervideo will ask + interactively. + -2, --twofactor TWOFACTOR Two-factor authentication code + -n, --netrc Use .netrc authentication data + --video-password PASSWORD Video password (vimeo, youku) + +## Adobe Pass Options: + --ap-mso MSO Adobe Pass multiple-system operator (TV + provider) identifier, use --ap-list-mso + for a list of available MSOs + --ap-username USERNAME Multiple-system operator account login + --ap-password PASSWORD Multiple-system operator account + password. If this option is left out, + hypervideo will ask interactively. + --ap-list-mso List all supported multiple-system + operators + +## Post-processing Options: + -x, --extract-audio Convert video files to audio-only files + (requires ffmpeg/avconv and + ffprobe/avprobe) + --audio-format FORMAT Specify audio format: "best", "aac", + "flac", "mp3", "m4a", "opus", "vorbis", + or "wav"; "best" by default; No effect + without -x + --audio-quality QUALITY Specify ffmpeg/avconv audio quality, + insert a value between 0 (better) and 9 + (worse) for VBR or a specific bitrate + like 128K (default 5) + --recode-video FORMAT Encode the video to another format if + necessary (currently supported: + mp4|flv|ogg|webm|mkv|avi) + --postprocessor-args ARGS Give these arguments to the + postprocessor + -k, --keep-video Keep the video file on disk after the + post-processing; the video is erased by + default + --no-post-overwrites Do not overwrite post-processed files; + the post-processed files are + overwritten by default + --embed-subs Embed subtitles in the video (only for + mp4, webm and mkv videos) + --embed-thumbnail Embed thumbnail in the audio as cover + art + --add-metadata Write metadata to the video file + --metadata-from-title FORMAT Parse additional metadata like song + title / artist from the video title. + The format syntax is the same as + --output. Regular expression with named + capture groups may also be used. The + parsed parameters replace existing + values. Example: --metadata-from-title + "%(artist)s - %(title)s" matches a + title like "Coldplay - Paradise". + Example (regex): --metadata-from-title + "(?P<artist>.+?) - (?P<title>.+)" + --xattrs Write metadata to the video file's + xattrs (using dublin core and xdg + standards) + --fixup POLICY Automatically correct known faults of + the file. One of never (do nothing), + warn (only emit a warning), + detect_or_warn (the default; fix file + if we can, warn otherwise) + --prefer-avconv Prefer avconv over ffmpeg for running + the postprocessors + --prefer-ffmpeg Prefer ffmpeg over avconv for running + the postprocessors (default) + --ffmpeg-location PATH Location of the ffmpeg/avconv binary; + either the path to the binary or its + containing directory. + --exec CMD Execute a command on the file after + downloading and post-processing, + similar to find's -exec syntax. + Example: --exec 'adb push {} + /sdcard/Music/ && rm {}' + --convert-subs FORMAT Convert the subtitles to other format + (currently supported: srt|ass|vtt|lrc) # CONFIGURATION -You can configure yt-dlp by placing any supported command line option to a configuration file. The configuration is loaded from the following locations: - -1. **Main Configuration**: The file given by `--config-location` -1. **Portable Configuration**: `yt-dlp.conf` in the same directory as the bundled binary. If you are running from source-code (`<root dir>/yt_dlp/__main__.py`), the root directory is used instead. -1. **Home Configuration**: `yt-dlp.conf` in the home path given by `-P`, or in the current directory if no such path is given -1. **User Configuration**: - * `%XDG_CONFIG_HOME%/yt-dlp/config` (recommended on Linux/macOS) - * `%XDG_CONFIG_HOME%/yt-dlp.conf` - * `%APPDATA%/yt-dlp/config` (recommended on Windows) - * `%APPDATA%/yt-dlp/config.txt` - * `~/yt-dlp.conf` - * `~/yt-dlp.conf.txt` - - `%XDG_CONFIG_HOME%` defaults to `~/.config` if undefined. On windows, `%APPDATA%` generally points to `C:\Users\<user name>\AppData\Roaming` and `~` points to `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`), or `%HOMEDRIVE%%HOMEPATH%` +You can configure hypervideo by placing any supported command line option to a configuration file. On GNU+Linux and macOS, the system wide configuration file is located at `/etc/hypervideo.conf` and the user wide configuration file at `~/.config/hypervideo/config`. Note that by default configuration file may not exist so you may need to create it yourself. -1. **System Configuration**: `/etc/yt-dlp.conf` +For example, with the following configuration file hypervideo will always extract the audio, not copy the mtime, use a proxy and save all videos under `Movies` directory in your home directory: -For example, with the following configuration file yt-dlp will always extract the audio, not copy the mtime, use a proxy and save all videos under `YouTube` directory in your home directory: ``` # Lines starting with # are comments @@ -1154,109 +472,72 @@ For example, with the following configuration file yt-dlp will always extract th # Use this proxy --proxy 127.0.0.1:3128 -# Save all videos under YouTube directory in your home directory --o ~/YouTube/%(title)s.%(ext)s +# Save all videos under Movies directory in your home directory +-o ~/Movies/%(title)s.%(ext)s ``` -Note that options in configuration file are just the same options aka switches used in regular command line calls; thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. +Note that options in configuration file are just the same options aka switches used in regular command line calls thus there **must be no whitespace** after `-` or `--`, e.g. `-o` or `--proxy` but not `- o` or `-- proxy`. -You can use `--ignore-config` if you want to disable all configuration files for a particular yt-dlp run. If `--ignore-config` is found inside any configuration file, no further configuration will be loaded. For example, having the option in the portable configuration file prevents loading of home, user, and system configurations. Additionally, (for backward compatibility) if `--ignore-config` is found inside the system configuration file, the user configuration is not loaded. +You can use `--ignore-config` if you want to disable the configuration file for a particular hypervideo run. + +You can also use `--config-location` if you want to use custom configuration file for a particular hypervideo run. ### Authentication with `.netrc` file -You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every yt-dlp execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in `--netrc-location` and restrict permissions to read/write by only you: +You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with `--username` and `--password`) in order not to pass credentials as command line arguments on every hypervideo execution and prevent tracking plain text passwords in the shell command history. You can achieve this using a [`.netrc` file](https://stackoverflow.com/tags/.netrc/info) on a per extractor basis. For that you will need to create a `.netrc` file in your `$HOME` and restrict permissions to read/write by only you: ``` touch $HOME/.netrc chmod a-rwx,u+rw $HOME/.netrc ``` After that you can add credentials for an extractor in the following format, where *extractor* is the name of the extractor in lowercase: ``` -machine <extractor> login <username> password <password> +machine <extractor> login <login> password <password> ``` For example: ``` machine youtube login myaccount@gmail.com password my_youtube_password machine twitch login my_twitch_account_name password my_twitch_password ``` -To activate authentication with the `.netrc` file you should pass `--netrc` to yt-dlp or place it in the [configuration file](#configuration). +To activate authentication with the `.netrc` file you should pass `--netrc` to hypervideo or place it in the [configuration file](#configuration). -The default location of the .netrc file is `$HOME` (`~`) in UNIX. On Windows, it is `%HOME%` if present, `%USERPROFILE%` (generally `C:\Users\<user name>`) or `%HOMEDRIVE%%HOMEPATH%` +On Windows you may also need to setup the `%HOME%` environment variable manually. For example: +``` +set HOME=%USERPROFILE% +``` # OUTPUT TEMPLATE -The `-o` option is used to indicate a template for the output file names while `-P` option is used to specify the path each type of file should be saved to. +The `-o` option allows users to indicate a template for the output file names. -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> **tl;dr:** [navigate me to examples](#output-template-examples). -<!-- MANPAGE: END EXCLUDED SECTION --> - -The simplest usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is _not_ recommended and could break some post-processing). - -It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [Python string formatting operations](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. - -The field names themselves (the part inside the parenthesis) can also have some special formatting: - -1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)s`, `%(id.3:7:-1)s`, `%(formats.:.format_id)s`. `%()s` refers to the entire infodict. Note that all the fields that become available using this method are not listed below. Use `-j` to see such fields - -1. **Addition**: Addition and subtraction of numeric fields can be done using `+` and `-` respectively. Eg: `%(playlist_index+10)03d`, `%(n_entries+1-playlist_index)d` - -1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s`, `%(upload_date>%Y-%m-%d)s`, `%(epoch-3600>%H-%M-%S)s` - -1. **Alternatives**: Alternate fields can be specified separated with a `,`. Eg: `%(release_date>%Y,upload_date>%Y|Unknown)s` - -1. **Replacement**: A replacement value can specified using a `&` separator. If the field is *not* empty, this replacement value will be used instead of the actual field content. This is done after alternate fields are considered; thus the replacement is used if *any* of the alternative fields is *not* empty. - -1. **Default**: A literal default value can be specified for when the field is empty using a `|` separator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s` - -1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `B`, `j`, `l`, `q`, `D`, `S` can be used for converting to **B**ytes, **j**son (flag `#` for pretty-printing), a comma separated **l**ist (flag `#` for `\n` newline-separated), a string **q**uoted for the terminal (flag `#` to split a list into different arguments), to add **D**ecimal suffixes (Eg: 10M) (flag `#` to use 1024 as factor), and to **S**anitize as filename (flag `#` for restricted), respectively - -1. **Unicode normalization**: The format type `U` can be used for NFC [unicode normalization](https://docs.python.org/3/library/unicodedata.html#unicodedata.normalize). The alternate form flag (`#`) changes the normalization to NFD and the conversion flag `+` can be used for NFKC/NFKD compatibility equivalence normalization. Eg: `%(title)+.100U` is NFKC -To summarize, the general syntax for a field is: -``` -%(name[.keys][addition][>strf][,alternate][&replacement][|default])[flags][width][.precision][length]type -``` - -Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation` (deprecated), `infojson`, `link`, `pl_thumbnail`, `pl_description`, `pl_infojson`, `chapter`, `pl_video`. For example, `-o "%(title)s.%(ext)s" -o "thumbnail:%(title)s\%(title)s.%(ext)s"` will put the thumbnails in a folder with the same name as the video. If any of the templates is empty, that type of file will not be written. Eg: `--write-thumbnail -o "thumbnail:"` will write thumbnails only for playlists and not for video. - -The available fields are: +The basic usage is not to set any template arguments when downloading a single file, like in `hypervideo -o funny_video.flv "https://some/video"`. However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Allowed names along with sequence type are: - `id` (string): Video identifier - `title` (string): Video title - - `fulltitle` (string): Video title ignoring live timestamp and generic title - `url` (string): Video URL - `ext` (string): Video filename extension - `alt_title` (string): A secondary title of the video - - `description` (string): The description of the video - `display_id` (string): An alternative identifier for the video - `uploader` (string): Full name of the video uploader - `license` (string): License name the video is licensed under - `creator` (string): The creator of the video + - `release_date` (string): The date (YYYYMMDD) when the video was released - `timestamp` (numeric): UNIX timestamp of the moment the video became available - - `upload_date` (string): Video upload date in UTC (YYYYMMDD) - - `release_timestamp` (numeric): UNIX timestamp of the moment the video was released - - `release_date` (string): The date (YYYYMMDD) when the video was released in UTC - - `modified_timestamp` (numeric): UNIX timestamp of the moment the video was last modified - - `modified_date` (string): The date (YYYYMMDD) when the video was last modified in UTC + - `upload_date` (string): Video upload date (YYYYMMDD) - `uploader_id` (string): Nickname or id of the video uploader - `channel` (string): Full name of the channel the video is uploaded on - `channel_id` (string): Id of the channel - - `channel_follower_count` (numeric): Number of followers of the channel - `location` (string): Physical location where the video was filmed - `duration` (numeric): Length of the video in seconds - - `duration_string` (string): Length of the video (HH:mm:ss) - `view_count` (numeric): How many users have watched the video on the platform - `like_count` (numeric): Number of positive ratings of the video - `dislike_count` (numeric): Number of negative ratings of the video - `repost_count` (numeric): Number of reposts of the video - `average_rating` (numeric): Average rating give by users, the scale used depends on the webpage - - `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used) + - `comment_count` (numeric): Number of comments on the video - `age_limit` (numeric): Age restriction for the video (years) - - `live_status` (string): One of "is_live", "was_live", "is_upcoming", "not_live" - `is_live` (boolean): Whether this video is a live stream or a fixed-length video - - `was_live` (boolean): Whether this video was originally a live stream - - `playable_in_embed` (string): Whether this video is allowed to play in embedded players on other sites - - `availability` (string): Whether the video is "private", "premium_only", "subscriber_only", "needs_auth", "unlisted" or "public" - `start_time` (numeric): Time in seconds where the reproduction should start, as specified in the URL - `end_time` (numeric): Time in seconds where the reproduction should end, as specified in the URL - `format` (string): A human-readable description of the format @@ -1271,8 +552,6 @@ The available fields are: - `asr` (numeric): Audio sampling rate in Hertz - `vbr` (numeric): Average video bitrate in KBit/s - `fps` (numeric): Frame rate - - `dynamic_range` (string): The dynamic range of the video - - `stretched_ratio` (float): `width:height` of the video's pixels, if not square - `vcodec` (string): Name of the video codec in use - `container` (string): Name of the container format - `filesize` (numeric): The number of bytes, if known in advance @@ -1280,22 +559,14 @@ The available fields are: - `protocol` (string): The protocol that will be used for the actual download - `extractor` (string): Name of the extractor - `extractor_key` (string): Key name of the extractor - - `epoch` (numeric): Unix epoch of when the information extraction was completed + - `epoch` (numeric): Unix epoch when creating the file - `autonumber` (numeric): Number that will be increased with each download, starting at `--autonumber-start` - - `video_autonumber` (numeric): Number that will be increased with each video - - `n_entries` (numeric): Total number of extracted items in the playlist - - `playlist_id` (string): Identifier of the playlist that contains the video - - `playlist_title` (string): Name of the playlist that contains the video - - `playlist` (string): `playlist_id` or `playlist_title` - - `playlist_count` (numeric): Total number of items in the playlist. May not be known if entire playlist is not extracted - - `playlist_index` (numeric): Index of the video in the playlist padded with leading zeros according the final index - - `playlist_autonumber` (numeric): Position of the video in the playlist download queue padded with leading zeros according to the total length of the playlist + - `playlist` (string): Name or id of the playlist that contains the video + - `playlist_index` (numeric): Index of the video in the playlist padded with leading zeros according to the total length of the playlist + - `playlist_id` (string): Playlist identifier + - `playlist_title` (string): Playlist title - `playlist_uploader` (string): Full name of the playlist uploader - `playlist_uploader_id` (string): Nickname or id of the playlist uploader - - `webpage_url` (string): A URL to the video webpage which if given to yt-dlp should allow to get the same result again - - `webpage_url_basename` (string): The basename of the webpage URL - - `webpage_url_domain` (string): The domain of the webpage URL - - `original_url` (string): The URL given by the user (or same as `webpage_url` for playlist entries) Available for the video that belongs to some logical chapter or section: @@ -1326,107 +597,60 @@ Available for the media that is a track or a part of a music album: - `disc_number` (numeric): Number of the disc or other physical medium the track belongs to - `release_year` (numeric): Year (YYYY) when the album was released -Available only when using `--download-sections` and for `chapter:` prefix when using `--split-chapters` for videos with internal chapters: - - - `section_title` (string): Title of the chapter - - `section_number` (numeric): Number of the chapter within the file - - `section_start` (numeric): Start time of the chapter in seconds - - `section_end` (numeric): End time of the chapter in seconds - -Available only when used in `--print`: - - - `urls` (string): The URLs of all requested formats, one in each line - - `filename` (string): Name of the video file. Note that the actual filename may be different due to post-processing. Use `--exec echo` to get the name after all postprocessing is complete - - `formats_table` (table): The video format table as printed by `--list-formats` - - `thumbnails_table` (table): The thumbnail format table as printed by `--list-thumbnails` - - `subtitles_table` (table): The subtitle format table as printed by `--list-subs` - - `automatic_captions_table` (table): The automatic subtitle format table as printed by `--list-subs` - - -Available only in `--sponsorblock-chapter-title`: - - - `start_time` (numeric): Start time of the chapter in seconds - - `end_time` (numeric): End time of the chapter in seconds - - `categories` (list): The SponsorBlock categories the chapter belongs to - - `category` (string): The smallest SponsorBlock category the chapter belongs to - - `category_names` (list): Friendly names of the categories - - `name` (string): Friendly name of the smallest category - -Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `yt-dlp test video` and id `BaW_jenozKc`, this will result in a `yt-dlp test video-BaW_jenozKc.mp4` file created in the current directory. +Each aforementioned sequence when referenced in an output template will be replaced by the actual value corresponding to the sequence name. Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default). -Note that some of the sequences are not guaranteed to be present since they depend on the metadata obtained by a particular extractor. Such sequences will be replaced with placeholder value provided with `--output-na-placeholder` (`NA` by default). +For example for `-o %(title)s-%(id)s.%(ext)s` and an mp4 video with title `hypervideo test video` and id `BaW_jenozKcj`, this will result in a `hypervideo test video-BaW_jenozKcj.mp4` file created in the current directory. -**Tip**: Look at the `-j` output to identify which fields are available for the particular URL +For numeric sequences you can use numeric related formatting, for example, `%(view_count)05d` will result in a string with view count padded with zeros up to 5 characters, like in `00042`. -For numeric sequences you can use [numeric related formatting](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting), for example, `%(view_count)05d` will result in a string with view count padded with zeros up to 5 characters, like in `00042`. - -Output templates can also contain arbitrary hierarchical path, e.g. `-o "%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s"` which will result in downloading each video in a directory corresponding to this path template. Any missing directory will be automatically created for you. +Output templates can also contain arbitrary hierarchical path, e.g. `-o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s'` which will result in downloading each video in a directory corresponding to this path template. Any missing directory will be automatically created for you. To use percent literals in an output template use `%%`. To output to stdout use `-o -`. -The current default template is `%(title)s [%(id)s].%(ext)s`. +The current default template is `%(title)s-%(id)s.%(ext)s`. -In some cases, you don't want special characters such as 中, spaces, or &, such as when transferring the downloaded filename to a Windows system or the filename through an 8bit-unsafe channel. In these cases, add the `--restrict-filenames` flag to get a shorter title. +In some cases, you don't want special characters such as 中, spaces, or &, such as when transferring the downloaded filename to a Windows system or the filename through an 8bit-unsafe channel. In these cases, add the `--restrict-filenames` flag to get a shorter title: -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> #### Output template and Windows batch files If you are using an output template inside a Windows batch file then you must escape plain percent characters (`%`) by doubling, so that `-o "%(title)s-%(id)s.%(ext)s"` should become `-o "%%(title)s-%%(id)s.%%(ext)s"`. However you should not touch `%`'s that are not plain characters, e.g. environment variables for expansion should stay intact: `-o "C:\%HOMEPATH%\Desktop\%%(title)s.%%(ext)s"`. -<!-- MANPAGE: END EXCLUDED SECTION --> #### Output template examples -```bash -$ yt-dlp --get-filename -o "test video.%(ext)s" BaW_jenozKc -test video.webm # Literal name with correct extension +Note that on Windows you may need to use double quotes instead of single. -$ yt-dlp --get-filename -o "%(title)s.%(ext)s" BaW_jenozKc -youtube-dl test video ''_ä↭𝕐.webm # All kinds of weird characters +```bash +$ hypervideo --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc +hypervideo test video ''_ä↭𝕐.mp4 # All kinds of weird characters -$ yt-dlp --get-filename -o "%(title)s.%(ext)s" BaW_jenozKc --restrict-filenames -youtube-dl_test_video_.webm # Restricted file name +$ hypervideo --get-filename -o '%(title)s.%(ext)s' BaW_jenozKc --restrict-filenames +hypervideo_test_video_.mp4 # A simple file name # Download YouTube playlist videos in separate directory indexed by video order in a playlist -$ yt-dlp -o "%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s" "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re" - -# Download YouTube playlist videos in separate directories according to their uploaded year -$ yt-dlp -o "%(upload_date>%Y)s/%(title)s.%(ext)s" "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re" - -# Prefix playlist index with " - " separator, but only if it is available -$ yt-dlp -o '%(playlist_index|)s%(playlist_index& - |)s%(title)s.%(ext)s' BaW_jenozKc "https://www.youtube.com/user/TheLinuxFoundation/playlists" +$ hypervideo -o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re # Download all playlists of YouTube channel/user keeping each playlist in separate directory: -$ yt-dlp -o "%(uploader)s/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s" "https://www.youtube.com/user/TheLinuxFoundation/playlists" +$ hypervideo -o '%(uploader)s/%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' https://www.youtube.com/user/TheLinuxFoundation/playlists # Download Udemy course keeping each chapter in separate directory under MyVideos directory in your home -$ yt-dlp -u user -p password -P "~/MyVideos" -o "%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s" "https://www.udemy.com/java-tutorial" +$ hypervideo -u user -p password -o '~/MyVideos/%(playlist)s/%(chapter_number)s - %(chapter)s/%(title)s.%(ext)s' https://www.udemy.com/java-tutorial/ # Download entire series season keeping each series and each season in separate directory under C:/MyVideos -$ yt-dlp -P "C:/MyVideos" -o "%(series)s/%(season_number)s - %(season)s/%(episode_number)s - %(episode)s.%(ext)s" "https://videomore.ru/kino_v_detalayah/5_sezon/367617" - -# Download video as "C:\MyVideos\uploader\title.ext", subtitles as "C:\MyVideos\subs\uploader\title.ext" -# and put all temporary files in "C:\MyVideos\tmp" -$ yt-dlp -P "C:/MyVideos" -P "temp:tmp" -P "subtitle:subs" -o "%(uploader)s/%(title)s.%(ext)s" BaW_jenoz --write-subs - -# Download video as "C:\MyVideos\uploader\title.ext" and subtitles as "C:\MyVideos\uploader\subs\title.ext" -$ yt-dlp -P "C:/MyVideos" -o "%(uploader)s/%(title)s.%(ext)s" -o "subtitle:%(uploader)s/subs/%(title)s.%(ext)s" BaW_jenozKc --write-subs +$ hypervideo -o "C:/MyVideos/%(series)s/%(season_number)s - %(season)s/%(episode_number)s - %(episode)s.%(ext)s" https://videomore.ru/kino_v_detalayah/5_sezon/367617 # Stream the video being downloaded to stdout -$ yt-dlp -o - BaW_jenozKc +$ hypervideo -o - BaW_jenozKc ``` # FORMAT SELECTION -By default, yt-dlp tries to download the best available quality if you **don't** pass any options. -This is generally equivalent to using `-f bestvideo*+bestaudio/best`. However, if multiple audiostreams is enabled (`--audio-multistreams`), the default format changes to `-f bestvideo+bestaudio/best`. Similarly, if ffmpeg is unavailable, or if you use yt-dlp to stream to `stdout` (`-o -`), the default becomes `-f best/bestvideo+bestaudio`. +By default hypervideo tries to download the best available quality, i.e. if you want the best quality you **don't need** to pass any special options, hypervideo will guess it for you by **default**. -**Deprecation warning**: Latest versions of yt-dlp can stream multiple formats to the stdout simultaneously using ffmpeg. So, in future versions, the default for this will be set to `-f bv*+ba/b` similar to normal downloads. If you want to preserve the `-f b/bv+ba` setting, it is recommended to explicitly specify it in the configuration options. +But sometimes you may want to download in a different format, for example when you are on a slow or intermittent connection. The key mechanism for achieving this is so-called *format selection* based on which you can explicitly specify desired format, select formats based on some criterion or criteria, setup precedence and much more. -The general syntax for format selection is `-f FORMAT` (or `--format FORMAT`) where `FORMAT` is a *selector expression*, i.e. an expression that describes format or formats you would like to download. +The general syntax for format selection is `--format FORMAT` or shorter `-f FORMAT` where `FORMAT` is a *selector expression*, i.e. an expression that describes format or formats you would like to download. -<!-- MANPAGE: BEGIN EXCLUDED SECTION --> **tl;dr:** [navigate me to examples](#format-selection-examples). -<!-- MANPAGE: END EXCLUDED SECTION --> The simplest case is requesting a specific format, for example with `-f 22` you can download the format with format code equal to 22. You can get the list of available format codes for particular video using `--list-formats` or `-F`. Note that these format codes are extractor specific. @@ -1436,37 +660,19 @@ You can use `-f -` to interactively provide the format selector *for each video* You can also use special names to select particular edge case formats: - - `all`: Select **all formats** separately - - `mergeall`: Select and **merge all formats** (Must be used with `--audio-multistreams`, `--video-multistreams` or both) - - `b*`, `best*`: Select the best quality format that **contains either** a video or an audio - - `b`, `best`: Select the best quality format that **contains both** video and audio. Equivalent to `best*[vcodec!=none][acodec!=none]` - - `bv`, `bestvideo`: Select the best quality **video-only** format. Equivalent to `best*[acodec=none]` - - `bv*`, `bestvideo*`: Select the best quality format that **contains video**. It may also contain audio. Equivalent to `best*[vcodec!=none]` - - `ba`, `bestaudio`: Select the best quality **audio-only** format. Equivalent to `best*[vcodec=none]` - - `ba*`, `bestaudio*`: Select the best quality format that **contains audio**. It may also contain video. Equivalent to `best*[acodec!=none]` ([Do not use!](https://github.com/yt-dlp/yt-dlp/issues/979#issuecomment-919629354)) - - `w*`, `worst*`: Select the worst quality format that contains either a video or an audio - - `w`, `worst`: Select the worst quality format that contains both video and audio. Equivalent to `worst*[vcodec!=none][acodec!=none]` - - `wv`, `worstvideo`: Select the worst quality video-only format. Equivalent to `worst*[acodec=none]` - - `wv*`, `worstvideo*`: Select the worst quality format that contains video. It may also contain audio. Equivalent to `worst*[vcodec!=none]` - - `wa`, `worstaudio`: Select the worst quality audio-only format. Equivalent to `worst*[vcodec=none]` - - `wa*`, `worstaudio*`: Select the worst quality format that contains audio. It may also contain video. Equivalent to `worst*[acodec!=none]` + - `best`: Select the best quality format represented by a single file with video and audio. + - `worst`: Select the worst quality format represented by a single file with video and audio. + - `bestvideo`: Select the best quality video-only format (e.g. DASH video). May not be available. + - `worstvideo`: Select the worst quality video-only format. May not be available. + - `bestaudio`: Select the best quality audio only-format. May not be available. + - `worstaudio`: Select the worst quality audio only-format. May not be available. -For example, to download the worst quality video-only format you can use `-f worstvideo`. It is however recommended not to use `worst` and related options. When your format selector is `worst`, the format which is worst in all respects is selected. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-S +size` or more rigorously, `-S +size,+br,+res,+fps` instead of `-f worst`. See [sorting formats](#sorting-formats) for more details. +For example, to download the worst quality video-only format you can use `-f worstvideo`. -You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream. - -If you want to download multiple videos and they don't have the same formats available, you can specify the order of preference using slashes. Note that formats on the left hand side are preferred, for example `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download. +If you want to download multiple videos and they don't have the same formats available, you can specify the order of preference using slashes. Note that slash is left-associative, i.e. formats on the left hand side are preferred, for example `-f 22/17/18` will download format 22 if it's available, otherwise it will download format 17 if it's available, otherwise it will download format 18 if it's available, otherwise it will complain that no suitable formats are available for download. If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`. -You can merge the video and audio of multiple formats into a single file using `-f <format1>+<format2>+...` (requires ffmpeg installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg. - -**Deprecation warning**: Since the *below* described behavior is complex and counter-intuitive, this will be removed and multistreams will be enabled by default in the future. A new operator will be instead added to limit formats to single audio/video - -Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. For example, `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`. - -## Filtering Formats - You can also filter the video formats by putting a condition in brackets, as in `-f "best[height=720]"` (or `-f "[filesize>10M]"`). The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, `>=`, `=` (equals), `!=` (not equals): @@ -1480,7 +686,7 @@ The following numeric meta fields can be used with comparisons `<`, `<=`, `>`, ` - `asr`: Audio sampling rate in Hertz - `fps`: Frame rate -Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains), `~=` (matches regex) and following string meta fields: +Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends with), `*=` (contains) and following string meta fields: - `ext`: File extension - `acodec`: Name of the audio codec in use @@ -1490,443 +696,648 @@ Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends - `format_id`: A short description of the format - `language`: Language code -Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). The comparand of a string comparison needs to be quoted with either double or single quotes if it contains spaces or special characters other than `._-`. - -Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering. - -Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter. For example, `-f "all[vcodec=none]"` selects all audio-only formats. - -Format selectors can also be grouped using parentheses, for example if you want to download the best pre-merged mp4 and webm formats with a height lower than 480 you can use `-f "(mp4,webm)[height<480]"`. - -## Sorting Formats - -You can change the criteria for being considered the `best` by using `-S` (`--format-sort`). The general format for this is `--format-sort field1,field2...`. - -The available fields are: - - - `hasvid`: Gives priority to formats that has a video stream - - `hasaud`: Gives priority to formats that has a audio stream - - `ie_pref`: The format preference - - `lang`: The language preference - - `quality`: The quality of the format - - `source`: The preference of the source - - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native`/`m3u8` > `http_dash_segments`> `websocket_frag` > `mms`/`rtsp` > `f4f`/`f4m`) - - `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other) - - `acodec`: Audio Codec (`flac`/`alac` > `wav`/`aiff` > `opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `eac3` > `ac3` > `dts` > other) - - `codec`: Equivalent to `vcodec,acodec` - - `vext`: Video Extension (`mp4` > `webm` > `flv` > other). If `--prefer-free-formats` is used, `webm` is preferred. - - `aext`: Audio Extension (`m4a` > `aac` > `mp3` > `ogg` > `opus` > `webm` > other). If `--prefer-free-formats` is used, the order changes to `opus` > `ogg` > `webm` > `m4a` > `mp3` > `aac`. - - `ext`: Equivalent to `vext,aext` - - `filesize`: Exact filesize, if known in advance - - `fs_approx`: Approximate filesize calculated from the manifests - - `size`: Exact filesize if available, otherwise approximate filesize - - `height`: Height of video - - `width`: Width of video - - `res`: Video resolution, calculated as the smallest dimension. - - `fps`: Framerate of video - - `hdr`: The dynamic range of the video (`DV` > `HDR12` > `HDR10+` > `HDR10` > `HLG` > `SDR`) - - `tbr`: Total average bitrate in KBit/s - - `vbr`: Average video bitrate in KBit/s - - `abr`: Average audio bitrate in KBit/s - - `br`: Equivalent to using `tbr,vbr,abr` - - `asr`: Audio sample rate in Hz - -**Deprecation warning**: Many of these fields have (currently undocumented) aliases, that may be removed in a future version. It is recommended to use only the documented field names. +Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain). -All fields, unless specified otherwise, are sorted in descending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a preferred value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two preferred values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB. +Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the video hoster. -The fields `hasvid` and `ie_pref` are always given highest priority in sorting, irrespective of the user-defined order. This behaviour can be changed by using `--format-sort-force`. Apart from these, the default order used is: `lang,quality,res,fps,hdr:12,codec:vp9.2,size,br,asr,proto,ext,hasaud,source,id`. The extractors may override this default order, but they cannot override the user-provided order. +Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height <=? 720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. -Note that the default has `codec:vp9.2`; i.e. `av1` is not preferred. Similarly, the default for hdr is `hdr:12`; i.e. dolby vision is not preferred. These choices are made since DV and AV1 formats are not yet fully compatible with most devices. This may be changed in the future as more devices become capable of smoothly playing back these formats. +You can merge the video and audio of two formats into a single file using `-f <video-format>+<audio-format>` (requires ffmpeg or avconv installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg/avconv. -If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`. +Format selectors can also be grouped using parentheses, for example if you want to download the best mp4 and webm formats with a height lower than 480 you can use `-f '(mp4,webm)[height<480]'`. -**Tip**: You can use the `-v -F` to see how the formats have been sorted (worst to best). +Since the end of April 2015 and version 2015.04.26, hypervideo uses `-f bestvideo+bestaudio/best` as the default format selection (see [#5447](https://github.com/ytdl-org/youtube-dl/issues/5447), [#5456](https://github.com/ytdl-org/youtube-dl/issues/5456)). If ffmpeg or avconv are installed this results in downloading `bestvideo` and `bestaudio` separately and muxing them together into a single file giving the best overall quality available. Otherwise it falls back to `best` and results in downloading the best available quality served as a single file. `best` is also needed for videos that don't come from YouTube because they don't provide the audio and video in two different files. If you want to only download some DASH formats (for example if you are not interested in getting videos with a resolution higher than 1080p), you can add `-f bestvideo[height<=?1080]+bestaudio/best` to your configuration file. Note that if you use youtube-dl to stream to `stdout` (and most likely to pipe it to your media player then), i.e. you explicitly specify output template as `-o -`, youtube-dl still uses `-f best` format selection in order to start content delivery immediately to your player and not to wait until `bestvideo` and `bestaudio` are downloaded and muxed. -## Format Selection examples +If you want to preserve the old format selection behavior (prior to youtube-dl 2015.04.26), i.e. you want to download the best available quality media served as a single file, you should explicitly specify your choice with `-f best`. You may want to add it to the [configuration file](#configuration) in order not to type it every time you run hypervideo. + +#### Format selection examples + +Note that on Windows you may need to use double quotes instead of single. ```bash -# Download and merge the best video-only format and the best audio-only format, -# or download the best combined format if video-only format is not available -$ yt-dlp -f "bv+ba/b" +# Download best mp4 format available or any other best if no mp4 available +$ hypervideo -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' -# Download best format that contains video, -# and if it doesn't already have an audio stream, merge it with best audio-only format -$ yt-dlp -f "bv*+ba/b" +# Download best format available but no better than 480p +$ hypervideo -f 'bestvideo[height<=480]+bestaudio/best[height<=480]' -# Same as above -$ yt-dlp +# Download best video only format but no bigger than 50 MB +$ hypervideo -f 'best[filesize<50M]' -# Download the best video-only format and the best audio-only format without merging them -# For this case, an output template should be used since -# by default, bestvideo and bestaudio will have the same file name. -$ yt-dlp -f "bv,ba" -o "%(title)s.f%(format_id)s.%(ext)s" +# Download best format available via direct link over HTTP/HTTPS protocol +$ hypervideo -f '(bestvideo+bestaudio/best)[protocol^=http]' -# Download and merge the best format that has a video stream, -# and all audio-only formats into one file -$ yt-dlp -f "bv*+mergeall[vcodec=none]" --audio-multistreams +# Download the best video format and the best audio format without merging them +$ hypervideo -f 'bestvideo,bestaudio' -o '%(title)s.f%(format_id)s.%(ext)s' +``` +Note that in the last example, an output template is recommended as bestvideo and bestaudio may have the same file name. -# Download and merge the best format that has a video stream, -# and the best 2 audio-only formats into one file -$ yt-dlp -f "bv*+ba+ba.2" --audio-multistreams +# VIDEO SELECTION -# The following examples show the old method (without -S) of format selection -# and how to use -S to achieve a similar but (generally) better result +Videos can be filtered by their upload date using the options `--date`, `--datebefore` or `--dateafter`. They accept dates in two formats: -# Download the worst video available (old method) -$ yt-dlp -f "wv*+wa/w" + - Absolute dates: Dates in the format `YYYYMMDD`. + - Relative dates: Dates in the format `(now|today)[+-][0-9](day|week|month|year)(s)?` -# Download the best video available but with the smallest resolution -$ yt-dlp -S "+res" +Examples: -# Download the smallest video available -$ yt-dlp -S "+size,+br" +```bash +# Download only the videos uploaded in the last 6 months +$ hypervideo --dateafter now-6months +# Download only the videos uploaded on January 1, 1970 +$ hypervideo --date 19700101 +$ # Download only the videos uploaded in the 200x decade +$ hypervideo --dateafter 20000101 --datebefore 20091231 +``` + +# FAQ + +### How do I update hypervideo? + +If you have used pacman, a simple `doas pacman -Syu hypervideo` is sufficient to update. -# Download the best mp4 video available, or the best video if no mp4 available -$ yt-dlp -f "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4] / bv*+ba/b" +As a last resort, you can also uninstall the version installed by your package manager and follow our manual installation instructions. For that, remove the distribution's package, with a line like -# Download the best video with the best extension -# (For video, mp4 > webm > flv. For audio, m4a > aac > mp3 ...) -$ yt-dlp -S "ext" +```console +$ doas pacman -Rs hypervideo +``` + +### I'm getting an error `Unable to extract OpenGraph title` on YouTube playlists +YouTube changed their playlist format in March 2014 and later on, so you'll need at least hypervideo to download all YouTube videos. +### I'm getting an error when trying to use output template: `error: using output template conflicts with using title, video ID or auto number` -# Download the best video available but no better than 480p, -# or the worst video if there is no video under 480p -$ yt-dlp -f "bv*[height<=480]+ba/b[height<=480] / wv*+ba/w" +Make sure you are not using `-o` with any of these options `-t`, `--title`, `--id`, `-A` or `--auto-number` set in command line or in a configuration file. Remove the latter if any. -# Download the best video available with the largest height but no better than 480p, -# or the best video with the smallest resolution if there is no video under 480p -$ yt-dlp -S "height:480" +### Do I always have to pass `-citw`? -# Download the best video available with the largest resolution but no better than 480p, -# or the best video with the smallest resolution if there is no video under 480p -# Resolution is determined by using the smallest dimension. -# So this works correctly for vertical videos as well -$ yt-dlp -S "res:480" +By default, hypervideo intends to have the best options (incidentally, if you have a convincing case that these should be different, [please file an issue where you explain that](https://yt-dl.org/bug)). Therefore, it is unnecessary and sometimes harmful to copy long option strings from webpages. In particular, the only option out of `-citw` that is regularly useful is `-i`. +### Can you please put the `-b` option back? +Most people asking this question are not aware that hypervideo now defaults to downloading the highest available quality as reported by YouTube, which will be 1080p or 720p in some cases, so you no longer need the `-b` option. For some specific videos, maybe YouTube does not report them to be available in a specific high quality format you're interested in. In that case, simply request it with the `-f` option and hypervideo will try to download it. -# Download the best video (that also has audio) but no bigger than 50 MB, -# or the worst video (that also has audio) if there is no video under 50 MB -$ yt-dlp -f "b[filesize<50M] / w" +### I get HTTP error 402 when trying to download a video. What's this? -# Download largest video (that also has audio) but no bigger than 50 MB, -# or the smallest video (that also has audio) if there is no video under 50 MB -$ yt-dlp -f "b" -S "filesize:50M" +Apparently YouTube requires you to pass a CAPTCHA test if you download too much. We're [considering to provide a way to let you solve the CAPTCHA](https://github.com/ytdl-org/youtube-dl/issues/154), but at the moment, your best course of action is pointing a web browser to the youtube URL, solving the CAPTCHA, and restart hypervideo. -# Download best video (that also has audio) that is closest in size to 50 MB -$ yt-dlp -f "b" -S "filesize~50M" +### Do I need any other programs? +hypervideo works fine on its own on most sites. However, if you want to convert video/audio, you'll need [avconv](https://libav.org/) or [ffmpeg](https://www.ffmpeg.org/). On some sites - most notably YouTube - videos can be retrieved in a higher quality format without sound. hypervideo will detect whether avconv/ffmpeg is present and automatically pick the best option. +Videos or video formats streamed via RTMP protocol can only be downloaded when [rtmpdump](https://rtmpdump.mplayerhq.hu/) is installed. Downloading MMS and RTSP videos requires either [mplayer](https://mplayerhq.hu/) or [mpv](https://mpv.io/) to be installed. -# Download best video available via direct link over HTTP/HTTPS protocol, -# or the best video available via any protocol if there is no such video -$ yt-dlp -f "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)" +### I have downloaded a video but how can I play it? -# Download best video available via the best protocol -# (https/ftps > http/ftp > m3u8_native > m3u8 > http_dash_segments ...) -$ yt-dlp -S "proto" +Once the video is fully downloaded, use any video player, such as [mpv](https://mpv.io/), [vlc](https://www.videolan.org/) or [mplayer](https://www.mplayerhq.hu/). +### I extracted a video URL with `-g`, but it does not play on another machine / in my web browser. +It depends a lot on the service. In many cases, requests for the video (to download/play it) must come from the same IP address and with the same cookies and/or HTTP headers. Use the `--cookies` option to write the required cookies into a file, and advise your downloader to read cookies from that file. Some sites also require a common user agent to be used, use `--dump-user-agent` to see the one in use by hypervideo. You can also get necessary cookies and HTTP headers from JSON output obtained with `--dump-json`. -# Download the best video with either h264 or h265 codec, -# or the best video if there is no such video -$ yt-dlp -f "(bv*[vcodec~='^((he|a)vc|h26[45])']+ba) / (bv*+ba/b)" +It may be beneficial to use IPv6; in some cases, the restrictions are only applied to IPv4. Some services (sometimes only for a subset of videos) do not restrict the video URL by IP address, cookie, or user-agent, but these are the exception rather than the rule. -# Download the best video with best codec no better than h264, -# or the best video with worst codec if there is no such video -$ yt-dlp -S "codec:h264" +Please bear in mind that some URL protocols are **not** supported by browsers out of the box, including RTMP. If you are using `-g`, your own downloader must support these as well. -# Download the best video with worst codec no worse than h264, -# or the best video with best codec if there is no such video -$ yt-dlp -S "+codec:h264" +If you want to play the video on a machine that is not running hypervideo, you can relay the video content from the machine that runs hypervideo. You can use `-o -` to let hypervideo stream a video to stdout, or simply allow the player to download the files written by hypervideo in turn. +### ERROR: no fmt_url_map or conn information found in video info +YouTube has switched to a new video info format in July 2011 which is not supported by old versions of hypervideo. See [above](#how-do-i-update-hypervideo) for how to update hypervideo. -# More complex examples +### ERROR: unable to download video -# Download the best video no better than 720p preferring framerate greater than 30, -# or the worst video (still preferring framerate greater than 30) if there is no such video -$ yt-dlp -f "((bv*[fps>30]/bv*)[height<=720]/(wv*[fps>30]/wv*)) + ba / (b[fps>30]/b)[height<=720]/(w[fps>30]/w)" +YouTube requires an additional signature since September 2012 which is not supported by old versions of hypervideo. See [above](#how-do-i-update-hypervideo) for how to hypervideo. -# Download the video with the largest resolution no better than 720p, -# or the video with the smallest resolution available if there is no such video, -# preferring larger framerate for formats with the same resolution -$ yt-dlp -S "res:720,fps" +### Video URL contains an ampersand and I'm getting some strange output `[1] 2839` or `'v' is not recognized as an internal or external command` +That's actually the output from your shell. Since ampersand is one of the special shell characters it's interpreted by the shell preventing you from passing the whole URL to hypervideo. To disable your shell from interpreting the ampersands (or any other special characters) you have to either put the whole URL in quotes or escape them with a backslash (which approach will work depends on your shell). +For example if your URL is https://www.youtube.com/watch?t=4&v=BaW_jenozKc you should end up with following command: -# Download the video with smallest resolution no worse than 480p, -# or the video with the largest resolution available if there is no such video, -# preferring better codec and then larger total bitrate for the same resolution -$ yt-dlp -S "+res:480,codec,br" +```console +$ hypervideo 'https://www.youtube.com/watch?t=4&v=BaW_jenozKc' ``` -# MODIFYING METADATA +or -The metadata obtained by the extractors can be modified by using `--parse-metadata` and `--replace-in-metadata` +```console +$ hypervideo https://www.youtube.com/watch?t=4\&v=BaW_jenozKc +``` -`--replace-in-metadata FIELDS REGEX REPLACE` is used to replace text in any metadata field using [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax). [Backreferences](https://docs.python.org/3/library/re.html?highlight=backreferences#re.sub) can be used in the replace string for advanced use. +### ExtractorError: Could not find JS function u'OF' -The general syntax of `--parse-metadata FROM:TO` is to give the name of a field or an [output template](#output-template) to extract data from, and the format to interpret it as, separated by a colon `:`. Either a [python regular expression](https://docs.python.org/3/library/re.html#regular-expression-syntax) with named capture groups or a similar syntax to the [output template](#output-template) (only `%(field)s` formatting is supported) can be used for `TO`. The option can be used multiple times to parse and modify various fields. +In February 2015, the new YouTube player contained a character sequence in a string that was misinterpreted by old versions of hypervideo. See [above](#how-do-i-update-hypervideo) for how to update hypervideo. -Note that any field created by this can be used in the [output template](#output-template) and will also affect the media file's metadata added when using `--add-metadata`. +### HTTP Error 429: Too Many Requests or 402: Payment Required -This option also has a few special uses: +These two error codes indicate that the service is blocking your IP address because of overuse. Usually this is a soft block meaning that you can gain access again after solving CAPTCHA. Just open a browser and solve a CAPTCHA the service suggests you and after that [pass cookies](#how-do-i-pass-cookies-to-hypervideo) to hypervideo. Note that if your machine has multiple external IPs then you should also pass exactly the same IP you've used for solving CAPTCHA with [`--source-address`](#network-options). Also you may need to pass a `User-Agent` HTTP header of your browser with [`--user-agent`](#workarounds). -* You can download an additional URL based on the metadata of the currently downloaded video. To do this, set the field `additional_urls` to the URL that you want to download. Eg: `--parse-metadata "description:(?P<additional_urls>https?://www\.vimeo\.com/\d+)` will download the first vimeo video found in the description +If this is not the case (no CAPTCHA suggested to solve by the service) then you can contact the service and ask them to unblock your IP address, or - if you have acquired a whitelisted IP address already - use the [`--proxy` or `--source-address` options](#network-options) to select another IP address. -* You can use this to change the metadata that is embedded in the media file. To do this, set the value of the corresponding field with a `meta_` prefix. For example, any value you set to `meta_description` field will be added to the `description` field in the file. For example, you can use this to set a different "description" and "synopsis". To modify the metadata of individual streams, use the `meta<n>_` prefix (Eg: `meta1_language`). Any value set to the `meta_` field will overwrite all default values. +### SyntaxError: Non-ASCII character -**Note**: Metadata modification happens before format selection, post-extraction and other post-processing operations. Some fields may be added or changed during these steps, overriding your changes. +The error -For reference, these are the fields yt-dlp adds by default to the file metadata: + File "hypervideo", line 2 + SyntaxError: Non-ASCII character '\x93' ... -Metadata fields | From -:--------------------------|:------------------------------------------------ -`title` | `track` or `title` -`date` | `upload_date` -`description`, `synopsis` | `description` -`purl`, `comment` | `webpage_url` -`track` | `track_number` -`artist` | `artist`, `creator`, `uploader` or `uploader_id` -`genre` | `genre` -`album` | `album` -`album_artist` | `album_artist` -`disc` | `disc_number` -`show` | `series` -`season_number` | `season_number` -`episode_id` | `episode` or `episode_id` -`episode_sort` | `episode_number` -`language` of each stream | the format's `language` +means you're using an outdated version of Python. Please update to Python 2.6 or 2.7. -**Note**: The file format may not support some of these fields +### What is this binary file? Where has the code gone? +Since June 2012 ([#342](https://github.com/ytdl-org/youtube-dl/issues/342)) hypervideo is packed as an executable zipfile, simply unzip it (might need renaming to `hypervideo.zip` first on some systems) or clone the git repository, as laid out above. If you modify the code, you can run it by executing the `__main__.py` file. To recompile the executable, run `make hypervideo`. -## Modifying metadata examples +### The exe throws an error due to missing `MSVCR100.dll` -```bash -# Interpret the title as "Artist - Title" -$ yt-dlp --parse-metadata "title:%(artist)s - %(title)s" +To run the exe you need to install first the [Microsoft Visual C++ 2010 Service Pack 1 Redistributable Package (x86)](https://download.microsoft.com/download/1/6/5/165255E7-1014-4D0A-B094-B6A430A6BFFC/vcredist_x86.exe). + +### On Windows, how should I set up ffmpeg and hypervideo? Where should I put the exe files? + +If you put hypervideo and ffmpeg in the same directory that you're running the command from, it will work, but that's rather cumbersome. + +To make a different directory work - either for ffmpeg, or for hypervideo, or for both - simply create the directory (say, `C:\bin`, or `C:\Users\<User name>\bin`), put all the executables directly in there, and then [set your PATH environment variable](https://www.java.com/en/download/help/path.xml) to include that directory. + +From then on, after restarting your shell, you will be able to access both hypervideo and ffmpeg (and hypervideo will be able to find ffmpeg) by simply typing `hypervideo` or `ffmpeg`, no matter what directory you're in. + +### How do I put downloads into a specific folder? + +Use the `-o` to specify an [output template](#output-template), for example `-o "/home/user/videos/%(title)s-%(id)s.%(ext)s"`. If you want this for all of your downloads, put the option into your [configuration file](#configuration). + +### How do I download a video starting with a `-`? + +Either prepend `https://www.youtube.com/watch?v=` or separate the ID from the options with `--`: + + $ hypervideo -- -wNyEUrxzFU + $ hypervideo "https://www.youtube.com/watch?v=-wNyEUrxzFU" + +### How do I pass cookies to hypervideo? + +Use the `--cookies` option, for example `--cookies /path/to/cookies/file.txt`. + +In order to extract cookies from browser use any conforming browser extension for exporting cookies. For example, [Get cookies.txt](https://chrome.google.com/webstore/detail/get-cookiestxt/bgaddhkoddajcdgocldbbfleckgcbcid/) (for Chrome) or [cookies.txt](https://addons.mozilla.org/en-US/firefox/addon/cookies-txt/) (for Firefox). + +Note that the cookies file must be in Mozilla/Netscape format and the first line of the cookies file must be either `# HTTP Cookie File` or `# Netscape HTTP Cookie File`. Make sure you have correct [newline format](https://en.wikipedia.org/wiki/Newline) in the cookies file and convert newlines if necessary to correspond with your OS, namely `CRLF` (`\r\n`) for Windows and `LF` (`\n`) for Unix and Unix-like systems (Linux, macOS, etc.). `HTTP Error 400: Bad Request` when using `--cookies` is a good sign of invalid newline format. + +Passing cookies to hypervideo is a good way to workaround login when a particular extractor does not implement it explicitly. Another use case is working around [CAPTCHA](https://en.wikipedia.org/wiki/CAPTCHA) some websites require you to solve in particular cases in order to get access (e.g. YouTube, CloudFlare). + +### How do I stream directly to media player? + +You will first need to tell hypervideo to stream media to stdout with `-o -`, and also tell your media player to read from stdin (it must be capable of this for streaming) and then pipe former to latter. For example, streaming to [vlc](https://www.videolan.org/) can be achieved with: + + $ hypervideo -o - "https://www.youtube.com/watch?v=BaW_jenozKcj" | vlc - + +### How do I download only new videos from a playlist? + +Use download-archive feature. With this feature you should initially download the complete playlist with `--download-archive /path/to/download/archive/file.txt` that will record identifiers of all the videos in a special file. Each subsequent run with the same `--download-archive` will download only new videos and skip all videos that have been downloaded before. Note that only successful downloads are recorded in the file. + +For example, at first, + + $ hypervideo --download-archive archive.txt "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re" + +will download the complete `PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re` playlist and create a file `archive.txt`. Each subsequent run will only download new videos if any: + + $ hypervideo --download-archive archive.txt "https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re" + +### Should I add `--hls-prefer-native` into my config? + +When hypervideo detects an HLS video, it can download it either with the built-in downloader or ffmpeg. Since many HLS streams are slightly invalid and ffmpeg/hypervideo each handle some invalid cases better than the other, there is an option to switch the downloader if needed. + +When hypervideo knows that one particular downloader works better for a given website, that downloader will be picked. Otherwise, hypervideo will pick the best downloader for general compatibility, which at the moment happens to be ffmpeg. This choice may change in future versions of hypervideo, with improvements of the built-in downloader and/or ffmpeg. + +In particular, the generic extractor (used when your website is not in the [list of supported sites by youtube-dl](https://ytdl-org.github.io/youtube-dl/supportedsites.html) cannot mandate one specific downloader. + +If you put either `--hls-prefer-native` or `--hls-prefer-ffmpeg` into your configuration, a different subset of videos will fail to download correctly. Instead, it is much better to [file an issue](https://yt-dl.org/bug) or a pull request which details why the native or the ffmpeg HLS downloader is a better choice for your use case. + +### Can you add support for this anime video site, or site which shows current movies for free? + +As a matter of policy (as well as legality), hypervideo does not include support for services that specialize in infringing copyright. As a rule of thumb, if you cannot easily find a video that the service is quite obviously allowed to distribute (i.e. that has been uploaded by the creator, the creator's distributor, or is published under a free license), the service is probably unfit for inclusion to hypervideo. + +A note on the service that they don't host the infringing content, but just link to those who do, is evidence that the service should **not** be included into hypervideo. The same goes for any DMCA note when the whole front page of the service is filled with videos they are not allowed to distribute. A "fair use" note is equally unconvincing if the service shows copyright-protected videos in full without authorization. + +Support requests for services that **do** purchase the rights to distribute their content are perfectly fine though. If in doubt, you can simply include a source that mentions the legitimate purchase of content. + +### How can I speed up work on my issue? + +(Also known as: Help, my important issue not being solved!) The hypervideo core developer team is quite small. While we do our best to solve as many issues as possible, sometimes that can take quite a while. To speed up your issue, here's what you can do: + +First of all, please do report the issue [at our issue tracker](https://yt-dl.org/bugs). That allows us to coordinate all efforts by users and developers, and serves as a unified point. Unfortunately, the hypervideo project has grown too large to use personal email as an effective communication channel. -# Regex example -$ yt-dlp --parse-metadata "description:Artist - (?P<artist>.+)" +Please read the [bug reporting instructions](#bugs) below. A lot of bugs lack all the necessary information. If you can, offer proxy, VPN, or shell access to the hypervideo developers. If you are able to, test the issue from multiple computers in multiple countries to exclude local censorship or misconfiguration issues. -# Set title as "Series name S01E05" -$ yt-dlp --parse-metadata "%(series)s S%(season_number)02dE%(episode_number)02d:%(title)s" +If nobody is interested in solving your issue, you are welcome to take matters into your own hands and submit a pull request (or coerce/pay somebody else to do so). -# Prioritize uploader as the "artist" field in video metadata -$ yt-dlp --parse-metadata "%(uploader|)s:%(meta_artist)s" --add-metadata +Feel free to bump the issue from time to time by writing a small comment ("Issue is still present in hypervideo version ...from France, but fixed from Belgium"), but please not more than once a month. Please do not declare your issue as `important` or `urgent`. -# Set "comment" field in video metadata using description instead of webpage_url, -# handling multiple lines correctly -$ yt-dlp --parse-metadata "description:(?s)(?P<meta_comment>.+)" --add-metadata +### How can I detect whether a given URL is supported by hypervideo? -# Do not set any "synopsis" in the video metadata -$ yt-dlp --parse-metadata ":(?P<meta_synopsis>)" +For one, have a look at the [list of supported sites](docs/supportedsites.md). Note that it can sometimes happen that the site changes its URL scheme (say, from https://example.com/video/1234567 to https://example.com/v/1234567 ) and hypervideo reports an URL of a service in that list as unsupported. In that case, simply report a bug. -# Remove "formats" field from the infojson by setting it to an empty string -$ yt-dlp --parse-metadata ":(?P<formats>)" -j +It is *not* possible to detect whether a URL is supported or not. That's because hypervideo contains a generic extractor which matches **all** URLs. You may be tempted to disable, exclude, or remove the generic extractor, but the generic extractor not only allows users to extract videos from lots of websites that embed a video from another service, but may also be used to extract video from a service that it's hosting itself. Therefore, we neither recommend nor support disabling, excluding, or removing the generic extractor. -# Replace all spaces and "_" in title and uploader with a `-` -$ yt-dlp --replace-in-metadata "title,uploader" "[ _]" "-" +If you want to find out whether a given URL is supported, simply call hypervideo with it. If you get no videos back, chances are the URL is either not referring to a video or unsupported. You can find out which by examining the output (if you run hypervideo on the console) or catching an `UnsupportedError` exception if you run it from a Python program. +# Why do I need to go through that much red tape when filing bugs? + +Before we had the issue template, despite our extensive [bug reporting instructions](#bugs), about 80% of the issue reports we got were useless, for instance because people used ancient versions hundreds of releases old, because of simple syntactic errors (not in hypervideo but in general shell usage), because the problem was already reported multiple times before, because people did not actually read an error message, even if it said "please install ffmpeg", because people did not mention the URL they were trying to download and many more simple, easy-to-avoid problems, many of whom were totally unrelated to hypervideo. + +hypervideo is an free software project manned by too few volunteers, so we'd rather spend time fixing bugs where we are certain none of those simple problems apply, and where we can be reasonably confident to be able to reproduce the issue without asking the reporter repeatedly. As such, the output of `hypervideo -v YOUR_URL_HERE` is really all that's required to file an issue. The issue template also guides you through some basic steps you can do, such as checking that your version of hypervideo is current. + +# DEVELOPER INSTRUCTIONS + +Most users do not need to build hypervideo and can [download the builds](https://git.conocimientoslibres.ga/software/hypervideo.git) or get them from their distribution. + +To run hypervideo as a developer, you don't need to build anything either. Simply execute + + $ python -m youtube_dl + +To run the test, simply invoke your favorite test runner, or execute a test file directly; any of the following work: + + $ python -m unittest discover + $ python test/test_download.py + $ nosetests + +See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases. + +If you want to create a build of hypervideo yourself, you'll need + +* python +* make (only GNU make is supported) +* pandoc +* zip +* nosetests + +### Adding support for a new site + +If you want to add support for a new site, first of all **make sure** this site is **not dedicated to [copyright infringement](README.md#can-you-add-support-for-this-anime-video-site-or-site-which-shows-current-movies-for-free)**. hypervideo does **not support** such sites thus pull requests adding support for them **will be rejected**. + +After you have ensured this site is distributing its content legally, you can follow this quick list (assuming your service is called `yourextractor`): + +1. [Fork this repository](https://git.conocimientoslibres.ga/software/hypervideo.git) +2. Check out the source code with: + + $ git clone https://git.conocimientoslibres.ga/software/hypervideo.git + +3. Start a new git branch with + + $ cd hypervideo + $ git checkout -b yourextractor + +4. Start with this simple template and save it to `youtube_dl/extractor/yourextractor.py`: + +```python + # coding: utf-8 + from __future__ import unicode_literals + + from .common import InfoExtractor + + + class YourExtractorIE(InfoExtractor): + _VALID_URL = r'https?://(?:www\.)?yourextractor\.com/watch/(?P<id>[0-9]+)' + _TEST = { + 'url': 'https://yourextractor.com/watch/42', + 'md5': 'TODO: md5 sum of the first 10241 bytes of the video file (use --test)', + 'info_dict': { + 'id': '42', + 'ext': 'mp4', + 'title': 'Video title goes here', + 'thumbnail': r're:^https?://.*\.jpg$', + # TODO more properties, either as: + # * A value + # * MD5 checksum; start the string with md5: + # * A regular expression; start the string with re: + # * Any Python type (for example int or float) + } + } + + def _real_extract(self, url): + video_id = self._match_id(url) + webpage = self._download_webpage(url, video_id) + + # TODO more code goes here, for example ... + title = self._html_search_regex(r'<h1>(.+?)</h1>', webpage, 'title') + + return { + 'id': video_id, + 'title': title, + 'description': self._og_search_description(webpage), + 'uploader': self._search_regex(r'<div[^>]+id="uploader"[^>]*>([^<]+)<', webpage, 'uploader', fatal=False), + # TODO more properties (see youtube_dl/extractor/common.py) + } ``` -# EXTRACTOR ARGUMENTS +5. Add an import in [`youtube_dl/extractor/extractors.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/extractors.py). +6. Run `python test/test_download.py TestDownload.test_YourExtractor`. This *should fail* at first, but you can continually re-run it until you're done. If you decide to add more than one test, then rename ``_TEST`` to ``_TESTS`` and make it into a list of dictionaries. The tests will then be named `TestDownload.test_YourExtractor`, `TestDownload.test_YourExtractor_1`, `TestDownload.test_YourExtractor_2`, etc. Note that tests with `only_matching` key in test's dict are not counted in. +7. Have a look at [`youtube_dl/extractor/common.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/extractor/common.py) for possible helper methods and a [detailed description of what your extractor should and may return](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303). Add tests and code for as many as you want. +8. Make sure your code follows [hypervideo coding conventions](#hypervideo-coding-conventions) and check the code with [flake8](https://flake8.pycqa.org/en/latest/index.html#quickstart): -Some extractors accept additional arguments which can be passed using `--extractor-args KEY:ARGS`. `ARGS` is a `;` (semicolon) separated string of `ARG=VAL1,VAL2`. Eg: `--extractor-args "youtube:player-client=android_embedded,web;include_live_dash" --extractor-args "funimation:version=uncut"` + $ flake8 youtube_dl/extractor/yourextractor.py -The following extractors use this feature: +9. Make sure your code works under all [Python](https://www.python.org/) versions claimed supported by hypervideo, namely 2.6, 2.7, and 3.2+. +10. When the tests pass, [add](https://git-scm.com/docs/git-add) the new files and [commit](https://git-scm.com/docs/git-commit) them and [push](https://git-scm.com/docs/git-push) the result, like this: -#### youtube -* `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively -* `player_client`: Clients to extract video data from. The main clients are `web`, `android` and `ios` with variants `_music`, `_embedded`, `_embedscreen`, `_creator` (Eg: `web_embedded`); and `mweb` and `tv_embedded` (agegate bypass) with no variants. By default, `android,web` is used, but `tv_embedded` and `creator` variants are added as required for age-gated videos. Similarly the music variants are added for `music.youtube.com` urls. You can use `all` to use all the clients, and `default` for the default clients. -* `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause some issues. See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) for more details -* `include_live_dash`: Include live dash formats even without `--live-from-start` (These formats don't download properly) -* `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side) -* `max_comments`: Limit the amount of comments to gather. Comma-separated list of integers representing `max-comments,max-parents,max-replies,max-replies-per-thread`. Default is `all,all,all,all` - * E.g. `all,all,1000,10` will get a maximum of 1000 replies total, with up to 10 replies per thread. `1000,all,100` will get a maximum of 1000 comments, with a maximum of 100 replies total -* `innertube_host`: Innertube API host to use for all API requests - * e.g. `studio.youtube.com`, `youtubei.googleapis.com` - * Note: Cookies exported from `www.youtube.com` will not work with hosts other than `*.youtube.com` -* `innertube_key`: Innertube API key to use for all API requests + $ git add youtube_dl/extractor/extractors.py + $ git add youtube_dl/extractor/yourextractor.py + $ git commit -m '[yourextractor] Add new extractor' + $ git push origin yourextractor -#### youtubetab (YouTube playlists, channels, feeds, etc.) -* `skip`: One or more of `webpage` (skip initial webpage download), `authcheck` (allow the download of playlists requiring authentication when no initial webpage is downloaded. This may cause unwanted behavior, see [#1122](https://github.com/yt-dlp/yt-dlp/pull/1122) for more details) -* `approximate_date`: Extract approximate `upload_date` in flat-playlist. This may cause date-based filters to be slightly off +11. Finally, [create a pull request](https://help.github.com/articles/creating-a-pull-request). We'll then review and merge it. -#### funimation -* `language`: Languages to extract. Eg: `funimation:language=english,japanese` -* `version`: The video version to extract - `uncut` or `simulcast` +In any case, thank you very much for your contributions! -#### crunchyroll -* `language`: Languages to extract. Eg: `crunchyroll:language=jaJp` -* `hardsub`: Which hard-sub versions to extract. Eg: `crunchyroll:hardsub=None,enUS` +## hypervideo coding conventions -#### crunchyrollbeta -* `format`: Which stream type(s) to extract. Default is `adaptive_hls` Eg: `crunchyrollbeta:format=vo_adaptive_hls` - * Potentially useful values include `adaptive_hls`, `adaptive_dash`, `vo_adaptive_hls`, `vo_adaptive_dash`, `download_hls`, `trailer_hls`, `trailer_dash` -* `hardsub`: Preference order for which hardsub versions to extract. Default is `None` (no hardsubs). Eg: `crunchyrollbeta:hardsub=en-US,None` +This section introduces a guide lines for writing idiomatic, robust and future-proof extractor code. -#### vikichannel -* `video_types`: Types of videos to download - one or more of `episodes`, `movies`, `clips`, `trailers` +Extractors are very fragile by nature since they depend on the layout of the source data provided by 3rd party media hosters out of your control and this layout tends to change. As an extractor implementer your task is not only to write code that will extract media links and metadata correctly but also to minimize dependency on the source's layout and even to make the code foresee potential future changes and be ready for that. This is important because it will allow the extractor not to break on minor layout changes thus keeping old hypervideo versions working. Even though this breakage issue is easily fixed by emitting a new version of hypervideo with a fix incorporated, all the previous versions become broken in all repositories and distros' packages that may not be so prompt in fetching the update from us. Needless to say, some non rolling release distros may never receive an update at all. -#### niconico -* `segment_duration`: Segment duration in milliseconds for HLS-DMC formats. Use it at your own risk since this feature **may result in your account termination.** +### Mandatory and optional metafields -#### youtubewebarchive -* `check_all`: Try to check more at the cost of more requests. One or more of `thumbnails`, `captures` +For extraction to work hypervideo relies on metadata your extractor extracts and provides to hypervideo expressed by an [information dictionary](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L94-L303) or simply *info dict*. Only the following meta fields in the *info dict* are considered mandatory for a successful extraction process by youtube-dl: -#### gamejolt -* `comment_sort`: `hot` (default), `you` (cookies needed), `top`, `new` - choose comment sorting mode (on GameJolt's side) + - `id` (media identifier) + - `title` (media title) + - `url` (media download URL) or `formats` -#### hotstar -* `res`: resolution to ignore - one or more of `sd`, `hd`, `fhd` -* `vcodec`: vcodec to ignore - one or more of `h264`, `h265`, `dvh265` -* `dr`: dynamic range to ignore - one or more of `sdr`, `hdr10`, `dv` +In fact only the last option is technically mandatory (i.e. if you can't figure out the download location of the media the extraction does not make any sense). But by convention hypervideo also treats `id` and `title` as mandatory. Thus the aforementioned metafields are the critical data that the extraction does not make any sense without and if any of them fail to be extracted then the extractor is considered completely broken. -#### tiktok -* `app_version`: App version to call mobile APIs with - should be set along with `manifest_app_version`. (e.g. `20.2.1`) -* `manifest_app_version`: Numeric app version to call mobile APIs with. (e.g. `221`) +[Any field](https://github.com/ytdl-org/youtube-dl/blob/7f41a598b3fba1bcab2817de64a08941200aa3c8/youtube_dl/extractor/common.py#L188-L303) apart from the aforementioned ones are considered **optional**. That means that extraction should be **tolerant** to situations when sources for these fields can potentially be unavailable (even if they are always available at the moment) and **future-proof** in order not to break the extraction of general purpose mandatory fields. -#### rokfinchannel -* `tab`: Which tab to download. One of `new`, `top`, `videos`, `podcasts`, `streams`, `stacks`. (E.g. `rokfinchannel:tab=streams`) +#### Example +Say you have some source dictionary `meta` that you've fetched as JSON with HTTP request and it has a key `summary`: -NOTE: These options may be changed/removed in the future without concern for backward compatibility +```python +meta = self._download_json(url, video_id) +``` -<!-- MANPAGE: MOVE "INSTALLATION" SECTION HERE --> +Assume at this point `meta`'s layout is: +```python +{ + ... + "summary": "some fancy summary text", + ... +} +``` -# PLUGINS +Assume you want to extract `summary` and put it into the resulting info dict as `description`. Since `description` is an optional meta field you should be ready that this key may be missing from the `meta` dict, so that you should extract it like: -Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`; where `<root-dir>` is the directory of the binary (`<root-dir>/yt-dlp`), or the root directory of the module if you are running directly from source-code (`<root dir>/yt_dlp/__main__.py`). Plugins are currently not supported for the `pip` version +```python +description = meta.get('summary') # correct +``` + +and not like: + +```python +description = meta['summary'] # incorrect +``` + +The latter will break extraction process with `KeyError` if `summary` disappears from `meta` at some later time but with the former approach extraction will just go ahead with `description` set to `None` which is perfectly fine (remember `None` is equivalent to the absence of data). -Plugins can be of `<type>`s `extractor` or `postprocessor`. Extractor plugins do not need to be enabled from the CLI and are automatically invoked when the input URL is suitable for it. Postprocessor plugins can be invoked using `--use-postprocessor NAME`. +Similarly, you should pass `fatal=False` when extracting optional data from a webpage with `_search_regex`, `_html_search_regex` or similar methods, for instance: -See [ytdlp_plugins](ytdlp_plugins) for example plugins. +```python +description = self._search_regex( + r'<span[^>]+id="title"[^>]*>([^<]+)<', + webpage, 'description', fatal=False) +``` -Note that **all** plugins are imported even if not invoked, and that **there are no checks** performed on plugin code. Use plugins at your own risk and only if you trust the code +With `fatal` set to `False` if `_search_regex` fails to extract `description` it will emit a warning and continue extraction. -If you are a plugin author, add [ytdlp-plugins](https://github.com/topics/ytdlp-plugins) as a topic to your repository for discoverability +You can also pass `default=<some fallback value>`, for example: +```python +description = self._search_regex( + r'<span[^>]+id="title"[^>]*>([^<]+)<', + webpage, 'description', default=None) +``` +On failure this code will silently continue the extraction with `description` set to `None`. That is useful for metafields that may or may not be present. -# EMBEDDING YT-DLP +### Provide fallbacks -yt-dlp makes the best effort to be a good command-line program, and thus should be callable from any programming language. +When extracting metadata try to do so from multiple sources. For example if `title` is present in several places, try extracting from at least some of them. This makes it more future-proof in case some of the sources become unavailable. -Your program should avoid parsing the normal stdout since they may change in future versions. Instead they should use options such as `-J`, `--print`, `--progress-template`, `--exec` etc to create console output that you can reliably reproduce and parse. +#### Example -From a Python program, you can embed yt-dlp in a more powerful fashion, like this: +Say `meta` from the previous example has a `title` and you are about to extract it. Since `title` is a mandatory meta field you should end up with something like: ```python -from yt_dlp import YoutubeDL +title = meta['title'] +``` + +If `title` disappears from `meta` in future due to some changes on the hoster's side the extraction would fail since `title` is mandatory. That's expected. + +Assume that you have some another source you can extract `title` from, for example `og:title` HTML meta of a `webpage`. In this case you can provide a fallback scenario: -URLS = ['https://www.youtube.com/watch?v=BaW_jenozKc'] -with YoutubeDL() as ydl: - ydl.download(URLS) +```python +title = meta.get('title') or self._og_search_title(webpage) ``` -Most likely, you'll want to use various options. For a list of options available, have a look at [`yt_dlp/YoutubeDL.py`](yt_dlp/YoutubeDL.py#L180). +This code will try to extract from `meta` first and if it fails it will try extracting `og:title` from a `webpage`. + +### Regular expressions + +#### Don't capture groups you don't use + +Capturing group must be an indication that it's used somewhere in the code. Any group that is not used must be non capturing. -**Tip**: If you are porting your code from youtube-dl to yt-dlp, one important point to look out for is that we do not guarantee the return value of `YoutubeDL.extract_info` to be json serializable, or even be a dictionary. It will be dictionary-like, but if you want to ensure it is a serializable dictionary, pass it through `YoutubeDL.sanitize_info` as shown in the [example below](#extracting-information) +##### Example -## Embedding examples +Don't capture id attribute name here since you can't use it for anything anyway. -#### Extracting information +Correct: ```python -import json -import yt_dlp +r'(?:id|ID)=(?P<id>\d+)' +``` -URL = 'https://www.youtube.com/watch?v=BaW_jenozKc' +Incorrect: +```python +r'(id|ID)=(?P<id>\d+)' +``` -# ℹ️ See help(yt_dlp.YoutubeDL) for a list of available options and public functions -ydl_opts = {} -with yt_dlp.YoutubeDL(ydl_opts) as ydl: - info = ydl.extract_info(URL, download=False) - # ℹ️ ydl.sanitize_info makes the info json-serializable - print(json.dumps(ydl.sanitize_info(info))) +#### Make regular expressions relaxed and flexible + +When using regular expressions try to write them fuzzy, relaxed and flexible, skipping insignificant parts that are more likely to change, allowing both single and double quotes for quoted values and so on. + +##### Example + +Say you need to extract `title` from the following HTML code: + +```html +<span style="position: absolute; left: 910px; width: 90px; float: right; z-index: 9999;" class="title">some fancy title</span> ``` -#### Download using an info-json + +The code for that task should look similar to: ```python -import yt_dlp +title = self._search_regex( + r'<span[^>]+class="title"[^>]*>([^<]+)', webpage, 'title') +``` + +Or even better: + +```python +title = self._search_regex( + r'<span[^>]+class=(["\'])title\1[^>]*>(?P<title>[^<]+)', + webpage, 'title', group='title') +``` -INFO_FILE = 'path/to/video.info.json' +Note how you tolerate potential changes in the `style` attribute's value or switch from using double quotes to single for `class` attribute: -with yt_dlp.YoutubeDL() as ydl: - error_code = ydl.download_with_info_file(INFO_FILE) +The code definitely should not look like: -print('Some videos failed to download' if error_code - else 'All videos successfully downloaded') +```python +title = self._search_regex( + r'<span style="position: absolute; left: 910px; width: 90px; float: right; z-index: 9999;" class="title">(.*?)</span>', + webpage, 'title', group='title') ``` -#### Extract audio +### Long lines policy + +There is a soft limit to keep lines of code under 80 characters long. This means it should be respected if possible and if it does not make readability and code maintenance worse. + +For example, you should **never** split long string literals like URLs or some other often copied entities over multiple lines to fit this limit: + +Correct: ```python -import yt_dlp +'https://www.youtube.com/watch?v=FqZTN594JQw&list=PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4' +``` -URLS = ['https://www.youtube.com/watch?v=BaW_jenozKc'] +Incorrect: -ydl_opts = { - 'format': 'm4a/bestaudio/best', - # ℹ️ See help(yt_dlp.postprocessor) for a list of available Postprocessors and their arguments - 'postprocessors': [{ # Extract audio using ffmpeg - 'key': 'FFmpegExtractAudio', - 'preferredcodec': 'm4a', - }] -} +```python +'https://www.youtube.com/watch?v=FqZTN594JQw&list=' +'PLMYEtVRpaqY00V9W81Cwmzp6N6vZqfUKD4' +``` + +### Inline values + +Extracting variables is acceptable for reducing code duplication and improving readability of complex expressions. However, you should avoid extracting variables used only once and moving them to opposite parts of the extractor file, which makes reading the linear flow difficult. -with yt_dlp.YoutubeDL(ydl_opts) as ydl: - error_code = ydl.download(URLS) +#### Example + +Correct: + +```python +title = self._html_search_regex(r'<title>([^<]+)</title>', webpage, 'title') ``` -#### Filter videos +Incorrect: ```python -import yt_dlp +TITLE_RE = r'<title>([^<]+)</title>' +# ...some lines of code... +title = self._html_search_regex(TITLE_RE, webpage, 'title') +``` -URLS = ['https://www.youtube.com/watch?v=BaW_jenozKc'] +### Collapse fallbacks -def longer_than_a_minute(info, *, incomplete): - """Download only videos longer than a minute (or with unknown duration)""" - duration = info.get('duration') - if duration and duration < 60: - return 'The video is too short' +Multiple fallback values can quickly become unwieldy. Collapse multiple fallback values into a single expression via a list of patterns. -ydl_opts = { - 'match_filter': longer_than_a_minute, -} +#### Example + +Good: + +```python +description = self._html_search_meta( + ['og:description', 'description', 'twitter:description'], + webpage, 'description', default=None) +``` + +Unwieldy: -with yt_dlp.YoutubeDL(ydl_opts) as ydl: - error_code = ydl.download(URLS) +```python +description = ( + self._og_search_description(webpage, default=None) + or self._html_search_meta('description', webpage, default=None) + or self._html_search_meta('twitter:description', webpage, default=None)) ``` -#### Adding logger and progress hook +Methods supporting list of patterns are: `_search_regex`, `_html_search_regex`, `_og_search_property`, `_html_search_meta`. + +### Trailing parentheses + +Always move trailing parentheses after the last argument. + +#### Example + +Correct: ```python -import yt_dlp + lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'], + list) +``` + +Incorrect: + +```python + lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'], + list, +) +``` + +### Use convenience conversion and parsing functions + +Wrap all extracted numeric data into safe functions from [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py): `int_or_none`, `float_or_none`. Use them for string to number conversions as well. + +Use `url_or_none` for safe URL processing. + +Use `try_get` for safe metadata extraction from parsed JSON. + +Use `unified_strdate` for uniform `upload_date` or any `YYYYMMDD` meta field extraction, `unified_timestamp` for uniform `timestamp` extraction, `parse_filesize` for `filesize` extraction, `parse_count` for count meta fields extraction, `parse_resolution`, `parse_duration` for `duration` extraction, `parse_age_limit` for `age_limit` extraction. + +Explore [`youtube_dl/utils.py`](https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/utils.py) for more useful convenience functions. + +#### More examples + +##### Safely extract optional description from parsed JSON +```python +description = try_get(response, lambda x: x['result']['video'][0]['summary'], compat_str) +``` + +##### Safely extract more optional metadata +```python +video = try_get(response, lambda x: x['result']['video'][0], dict) or {} +description = video.get('summary') +duration = float_or_none(video.get('durationMs'), scale=1000) +view_count = int_or_none(video.get('views')) +``` + +# EMBEDDING YOUTUBE-DL -URLS = ['https://www.youtube.com/watch?v=BaW_jenozKc'] +hypervideo makes the best effort to be a good command-line program, and thus should be callable from any programming language. If you encounter any problems parsing its output, feel free to [create a report](https://github.com/ytdl-org/youtube-dl/issues/new). -class MyLogger: +From a Python program, you can embed hypervideo in a more powerful fashion, like this: + +```python +from __future__ import unicode_literals +import youtube_dl + +ydl_opts = {} +with youtube_dl.YoutubeDL(ydl_opts) as ydl: + ydl.download(['https://www.youtube.com/watch?v=BaW_jenozKc']) +``` + +Most likely, you'll want to use various options. For a list of options available, have a look at [`youtube_dl/YoutubeDL.py`](https://github.com/ytdl-org/youtube-dl/blob/3e4cedf9e8cd3157df2457df7274d0c842421945/youtube_dl/YoutubeDL.py#L137-L312). For a start, if you want to intercept youtube-dl's output, set a `logger` object. + +Here's a more complete example of a program that outputs only errors (and a short message after the download is finished), and downloads/converts the video to an mp3 file: + +```python +from __future__ import unicode_literals +import youtube_dl + + +class MyLogger(object): def debug(self, msg): - # For compatibility with youtube-dl, both debug and info are passed into debug - # You can distinguish them by the prefix '[debug] ' - if msg.startswith('[debug] '): - pass - else: - self.info(msg) - - def info(self, msg): pass def warning(self, msg): @@ -1936,206 +1347,103 @@ class MyLogger: print(msg) -# ℹ️ See "progress_hooks" in help(yt_dlp.YoutubeDL) def my_hook(d): if d['status'] == 'finished': - print('Done downloading, now post-processing ...') + print('Done downloading, now converting ...') ydl_opts = { + 'format': 'bestaudio/best', + 'postprocessors': [{ + 'key': 'FFmpegExtractAudio', + 'preferredcodec': 'mp3', + 'preferredquality': '192', + }], 'logger': MyLogger(), 'progress_hooks': [my_hook], } +with youtube_dl.YoutubeDL(ydl_opts) as ydl: + ydl.download(['https://www.youtube.com/watch?v=BaW_jenozKc']) +``` -with yt_dlp.YoutubeDL(ydl_opts) as ydl: - ydl.download(URLS) +# BUGS + +Bugs and suggestions should be reported at: <https://github.com/ytdl-org/youtube-dl/issues>. Unless you were prompted to or there is another pertinent reason (e.g. GitHub fails to accept the bug report), please do not send bug reports via personal email. For discussions, join us in the IRC channel [#youtube-dl](irc://chat.freenode.net/#youtube-dl) on freenode ([webchat](https://webchat.freenode.net/?randomnick=1&channels=youtube-dl)). + +**Please include the full output of hypervideo when run with `-v`**, i.e. **add** `-v` flag to **your command line**, copy the **whole** output and post it in the issue body wrapped in \`\`\` for better formatting. It should look similar to this: + +``` +$ hypervideo -v <your command line> +[debug] System config: [] +[debug] User config: [] +[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=BaW_jenozKcj'] +[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251 +[debug] hypervideo version 1.1.11 +[debug] Git HEAD: 135392e +[debug] Python version 2.6.6 - Windows-2003Server-5.2.3790-SP2 +[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4 +[debug] Proxy map: {} +... ``` -#### Add a custom PostProcessor +**Do not post screenshots of verbose logs; only plain text is acceptable.** -```python -import yt_dlp +The output (including the first lines) contains important debugging information. Issues without the full output are often not reproducible and therefore do not get solved in short order, if ever. -URLS = ['https://www.youtube.com/watch?v=BaW_jenozKc'] +Please re-read your issue once again to avoid a couple of common mistakes (you can and should use this as a checklist): -# ℹ️ See help(yt_dlp.postprocessor.PostProcessor) -class MyCustomPP(yt_dlp.postprocessor.PostProcessor): - def run(self, info): - self.to_screen('Doing stuff') - return [], info +### Is the description of the issue itself sufficient? +We often get issue reports that we cannot really decipher. While in most cases we eventually get the required information after asking back multiple times, this poses an unnecessary drain on our resources. Many contributors, including myself, are also not native speakers, so we may misread some parts. -with yt_dlp.YoutubeDL() as ydl: - # ℹ️ "when" can take any value in yt_dlp.utils.POSTPROCESS_WHEN - ydl.add_post_processor(MyCustomPP(), when='pre_process') - ydl.download(URLS) -``` +So please elaborate on what feature you are requesting, or what bug you want to be fixed. Make sure that it's obvious +- What the problem is +- How it could be fixed +- How your proposed solution would look like -#### Use a custom format selector +If your report is shorter than two lines, it is almost certainly missing some of these, which makes it hard for us to respond to it. We're often too polite to close the issue outright, but the missing info makes misinterpretation likely. As a committer myself, I often get frustrated by these issues, since the only possible way for me to move forward on them is to ask for clarification over and over. -```python -import yt_dlp +For bug reports, this means that your report should contain the *complete* output of hypervideo when called with the `-v` flag. The error message you get for (most) bugs even says so, but you would not believe how many of our bug reports do not contain this information. -URL = ['https://www.youtube.com/watch?v=BaW_jenozKc'] +If your server has multiple IPs or you suspect censorship, adding `--call-home` may be a good idea to get more diagnostics. If the error is `ERROR: Unable to extract ...` and you cannot reproduce it from multiple countries, add `--dump-pages` (warning: this will yield a rather large output, redirect it to the file `log.txt` by adding `>log.txt 2>&1` to your command-line) or upload the `.dump` files you get when you add `--write-pages` [somewhere](https://gist.github.com/). -def format_selector(ctx): - """ Select the best video and the best audio that won't result in an mkv. - NOTE: This is just an example and does not handle all cases """ +**Site support requests must contain an example URL**. An example URL is a URL you might want to download, like `https://www.youtube.com/watch?v=BaW_jenozKc`. There should be an obvious video present. Except under very special circumstances, the main page of a video service (e.g. `https://www.youtube.com/`) is *not* an example URL. - # formats are already sorted worst to best - formats = ctx.get('formats')[::-1] +### Are you using the latest version? - # acodec='none' means there is no audio - best_video = next(f for f in formats - if f['vcodec'] != 'none' and f['acodec'] == 'none') +Before reporting any issue, type `doas pacman -Sy hypervideo`. This should report that you're up-to-date. About 20% of the reports we receive are already fixed, but people are using outdated versions. This goes for feature requests as well. - # find compatible audio extension - audio_ext = {'mp4': 'm4a', 'webm': 'webm'}[best_video['ext']] - # vcodec='none' means there is no video - best_audio = next(f for f in formats if ( - f['acodec'] != 'none' and f['vcodec'] == 'none' and f['ext'] == audio_ext)) +### Is the issue already documented? - # These are the minimum required fields for a merged format - yield { - 'format_id': f'{best_video["format_id"]}+{best_audio["format_id"]}', - 'ext': best_video['ext'], - 'requested_formats': [best_video, best_audio], - # Must be + separated list of protocols - 'protocol': f'{best_video["protocol"]}+{best_audio["protocol"]}' - } +Make sure that someone has not already opened the issue you're trying to open. Search at the top of the window or browse the [GitHub Issues](https://github.com/ytdl-org/youtube-dl/search?type=Issues) of this repository. If there is an issue, feel free to write something along the lines of "This affects me as well, with version 2015.01.01. Here is some more information on the issue: ...". While some issues may be old, a new post into them often spurs rapid activity. +### Why are existing options not enough? -ydl_opts = { - 'format': format_selector, -} +Before requesting a new feature, please have a quick peek at [the list of supported options](https://github.com/ytdl-org/youtube-dl/blob/master/README.md#options). Many feature requests are for features that actually exist already! Please, absolutely do show off your work in the issue report and detail how the existing similar options do *not* solve your problem. -with yt_dlp.YoutubeDL(ydl_opts) as ydl: - ydl.download(URLS) -``` +### Is there enough context in your bug report? + +People want to solve problems, and often think they do us a favor by breaking down their larger problems (e.g. wanting to skip already downloaded files) to a specific request (e.g. requesting us to look whether the file exists before downloading the info page). However, what often happens is that they break down the problem into two steps: One simple, and one impossible (or extremely complicated one). + +We are then presented with a very complicated request when the original problem could be solved far easier, e.g. by recording the downloaded video IDs in a separate file. To avoid this, you must include the greater context where it is non-obvious. In particular, every feature request that does not consist of adding support for a new site should contain a use case scenario that explains in what situation the missing feature would be useful. + +### Does the issue involve one problem, and one problem only? + +Some of our users seem to think there is a limit of issues they can or should open. There is no limit of issues they can or should open. While it may seem appealing to be able to dump all your issues into one ticket, that means that someone who solves one of your issues cannot mark the issue as closed. Typically, reporting a bunch of issues leads to the ticket lingering since nobody wants to attack that behemoth, until someone mercifully splits the issue into multiple ones. + +In particular, every site support request issue should only pertain to services at one site (generally under a common domain, but always using the same backend technology). Do not request support for vimeo user videos, White house podcasts, and Google Plus pages in the same issue. Also, make sure that you don't post bug reports alongside feature requests. As a rule of thumb, a feature request does not include outputs of hypervideo that are not immediately related to the feature at hand. Do not post reports of a network error alongside the request for a new video service. + +### Is anyone going to need the feature? + +Only post features that you (or an incapacitated friend you can personally talk to) require. Do not post features because they seem like a good idea. If they are really useful, they will be requested by someone who requires them. + +### Is your question about hypervideo? + +It may sound strange, but some bug reports we receive are completely unrelated to hypervideo and relate to a different, or even the reporter's own, application. Please make sure that you are actually using hypervideo. If you are using a UI for hypervideo, report the bug to the maintainer of the actual application providing the UI. On the other hand, if your UI for hypervideo fails in some way you believe is related to hypervideo, by all means, go ahead and report the bug. + +# COPYRIGHT + +hypervideo is released into the [CC0 1.0 Universal](LICENSE) by the copyright holders. -<!-- MANPAGE: MOVE "NEW FEATURES" SECTION HERE --> - -# DEPRECATED OPTIONS - -These are all the deprecated options and the current alternative to achieve the same effect - -#### Almost redundant options -While these options are almost the same as their new counterparts, there are some differences that prevents them being redundant - - -j, --dump-json --print "%()j" - -F, --list-formats --print formats_table - --list-thumbnails --print thumbnails_table --print playlist:thumbnails_table - --list-subs --print automatic_captions_table --print subtitles_table - -#### Redundant options -While these options are redundant, they are still expected to be used due to their ease of use - - --get-description --print description - --get-duration --print duration_string - --get-filename --print filename - --get-format --print format - --get-id --print id - --get-thumbnail --print thumbnail - -e, --get-title --print title - -g, --get-url --print urls - --match-title REGEX --match-filter "title ~= (?i)REGEX" - --reject-title REGEX --match-filter "title !~= (?i)REGEX" - --min-views COUNT --match-filter "view_count >=? COUNT" - --max-views COUNT --match-filter "view_count <=? COUNT" - --user-agent UA --add-header "User-Agent:UA" - --referer URL --add-header "Referer:URL" - --playlist-start NUMBER -I NUMBER: - --playlist-end NUMBER -I :NUMBER - --playlist-reverse -I ::-1 - --no-playlist-reverse Default - - -#### Not recommended -While these options still work, their use is not recommended since there are other alternatives to achieve the same - - --exec-before-download CMD --exec "before_dl:CMD" - --no-exec-before-download --no-exec - --all-formats -f all - --all-subs --sub-langs all --write-subs - --print-json -j --no-simulate - --autonumber-size NUMBER Use string formatting. Eg: %(autonumber)03d - --autonumber-start NUMBER Use internal field formatting like %(autonumber+NUMBER)s - --id -o "%(id)s.%(ext)s" - --metadata-from-title FORMAT --parse-metadata "%(title)s:FORMAT" - --hls-prefer-native --downloader "m3u8:native" - --hls-prefer-ffmpeg --downloader "m3u8:ffmpeg" - --list-formats-old --compat-options list-formats (Alias: --no-list-formats-as-table) - --list-formats-as-table --compat-options -list-formats [Default] (Alias: --no-list-formats-old) - --youtube-skip-dash-manifest --extractor-args "youtube:skip=dash" (Alias: --no-youtube-include-dash-manifest) - --youtube-skip-hls-manifest --extractor-args "youtube:skip=hls" (Alias: --no-youtube-include-hls-manifest) - --youtube-include-dash-manifest Default (Alias: --no-youtube-skip-dash-manifest) - --youtube-include-hls-manifest Default (Alias: --no-youtube-skip-hls-manifest) - - -#### Developer options -These options are not intended to be used by the end-user - - --test Download only part of video for testing extractors - --load-pages Load pages dumped by --write-pages - --youtube-print-sig-code For testing youtube signatures - --allow-unplayable-formats List unplayable formats also - --no-allow-unplayable-formats Default - - -#### Old aliases -These are aliases that are no longer documented for various reasons - - --avconv-location --ffmpeg-location - --clean-infojson --clean-info-json - --cn-verification-proxy URL --geo-verification-proxy URL - --dump-headers --print-traffic - --dump-intermediate-pages --dump-pages - --force-write-download-archive --force-write-archive - --load-info --load-info-json - --no-clean-infojson --no-clean-info-json - --no-split-tracks --no-split-chapters - --no-write-srt --no-write-subs - --prefer-unsecure --prefer-insecure - --rate-limit RATE --limit-rate RATE - --split-tracks --split-chapters - --srt-lang LANGS --sub-langs LANGS - --trim-file-names LENGTH --trim-filenames LENGTH - --write-srt --write-subs - --yes-overwrites --force-overwrites - -#### Sponskrub Options -Support for [SponSkrub](https://github.com/faissaloo/SponSkrub) has been deprecated in favor of the `--sponsorblock` options - - --sponskrub --sponsorblock-mark all - --no-sponskrub --no-sponsorblock - --sponskrub-cut --sponsorblock-remove all - --no-sponskrub-cut --sponsorblock-remove -all - --sponskrub-force Not applicable - --no-sponskrub-force Not applicable - --sponskrub-location Not applicable - --sponskrub-args Not applicable - -#### No longer supported -These options may no longer work as intended - - --prefer-avconv avconv is not officially supported by yt-dlp (Alias: --no-prefer-ffmpeg) - --prefer-ffmpeg Default (Alias: --no-prefer-avconv) - -C, --call-home Not implemented - --no-call-home Default - --include-ads No longer supported - --no-include-ads Default - --write-annotations No supported site has annotations now - --no-write-annotations Default - -#### Removed -These options were deprecated since 2014 and have now been entirely removed - - -A, --auto-number -o "%(autonumber)s-%(id)s.%(ext)s" - -t, -l, --title, --literal -o "%(title)s-%(id)s.%(ext)s" - -# CONTRIBUTING -See [CONTRIBUTING.md](CONTRIBUTING.md#contributing-to-yt-dlp) for instructions on [Opening an Issue](CONTRIBUTING.md#opening-an-issue) and [Contributing code to the project](CONTRIBUTING.md#developer-instructions) - -# MORE -For FAQ see the [youtube-dl README](https://github.com/ytdl-org/youtube-dl#faq) +This README file was originally written by [Daniel Bolton](https://github.com/dbbolton) and is likewise released into the public domain. diff --git a/bin/yt-dlp b/bin/yt-dlp new file mode 100644 index 000000000..baecdeb0a --- /dev/null +++ b/bin/yt-dlp @@ -0,0 +1,6 @@ +#!/usr/bin/python + +import hypervideo_dl + +if __name__ == '__main__': + hypervideo_dl.main() diff --git a/build.sh b/build.sh new file mode 100644 index 000000000..324ee859a --- /dev/null +++ b/build.sh @@ -0,0 +1,104 @@ +#!/bin/bash + +# ============================================================================== +# FUNCTIONS - START +# ============================================================================== +VERSION='1.1.13' +BASEDIR="${2-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)}" + +run_rebrand() +{ + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Rebrand...' '\e[m' + + mv -T yt_dlp hypervideo_dl + mv bin/yt-dlp bin/hypervideo + + find . -type f \( \ + -iname "*" \ + ! -iname "build.sh" \ + ! -iname ".travis.yml" \ + ! -iname ".gitlab-ci.yml" \ + ! -path "./.git*" \) \ + -exec grep -rIl 'yt_dlp' {} + -exec sed -i 's|yt_dlp|hypervideo_dl|g' {} \; + + find . -name "tox.ini" -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + find . -name "setup.cfg" -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + find . -name "Makefile" -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + find . -name "*.in" -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + find docs -type f -exec sed -i 's|yt-dlp|hypervideo|g' {} \; + + # fixes URLs + find . -name "*.py" -type f -exec sed -i 's|hypervideo+test+video|yt-dlp+test+video|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://github.com/ytdl-org/hypervideo/|https://github.com/ytdl-org/yt-dlp/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://github.com/rg3/hypervideo/issues/|https://github.com/rg3/yt-dlp/issues/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|http://hypervideo.bandcamp.com/|http://yt-dlp.bandcamp.com/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|http://yt-dlp.bandcamp.com/track/hypervideo-test-song|http://yt-dlp.bandcamp.com/track/yt-dlp-test-song|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://www.dropbox.com/s/nelirfsxnmcfbfh/hypervideo|https://www.dropbox.com/s/nelirfsxnmcfbfh/yt-dlp|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|http://8tracks.com/ytdl/hypervideo-test-tracks-a|http://8tracks.com/ytdl/yt-dlp-test-tracks-a|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|http://phihag.de/2014/hypervideo/|http://phihag.de/2014/yt-dlp/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://hypervideo-demo.neocities.org/|https://yt-dlp-demo.neocities.org/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://travis-ci.org/ytdl-org/hypervideo/|https://travis-ci.org/ytdl-org/yt-dlp/|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|https://soundcloud.com/jaimemf/hypervideo-test|https://soundcloud.com/jaimemf/yt-dlp-test|g' {} \; + find . -name "*.py" -type f -exec sed -i 's|http://streamcloud.eu/skp9j99s4bpz/hypervideo_test_video|http://streamcloud.eu/skp9j99s4bpz/yt_dlp_test_video|g' {} \; + + find . -type f \( \ + -iname "*" \ + ! -iname "build.sh" \ + ! -iname ".travis.yml" \ + ! -iname ".gitlab-ci.yml" \ + ! -path "./.git*" \) \ + -exec grep -rIl 'YT-DLP' {} + -exec sed -i 's|YT-DLP|HYPERVIDEO|g' {} \; + + # Set current version + sed -i "s|__version.*|__version__ = '${VERSION}'|" hypervideo_dl/version.py + + # Set license + sed -i "s|UNLICENSE|CC0-1.0|" docs/conf.py || true +} + +run_clean() +{ + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Clean tempfiles...' '\e[m' + rm -rf -- hypervideo.1.temp.md hypervideo.1 hypervideo.bash-completion README.txt MANIFEST build/ dist/ .coverage cover/ hypervideo.tar.gz hypervideo_dl/extractor/lazy_extractors.py *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png CONTRIBUTING.md.tmp hypervideo hypervideo.exe + find . -name "*.pyc" -delete + find . -name "*.class" -delete +} + +run_make() +{ + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Making hypervideo...' '\e[m' + make hypervideo.tar.gz +} + +run_copy() +{ + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Genering build...' '\e[m' + install -d -m755 build + mv hypervideo.tar.gz "./build/hypervideo.tar.gz" + cd build && tar xzf hypervideo.tar.gz && + mv hypervideo "hypervideo-${VERSION}" && + tar -czvf "hypervideo-${VERSION}.tar.gz" "hypervideo-${VERSION}" && + rm -rf "hypervideo-${VERSION}" hypervideo.tar.gz + cd "$BASEDIR" || return + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Tarball in build/ directory' '\e[m' +} + +# Restore directory, compatibility reasons +run_reset() +{ + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Restore compatibility...' '\e[m' + mv -T hypervideo_dl yt_dlp + mv bin/hypervideo bin/yt-dlp + rm -v hypervideo + printf '%b%s%b%s%b\n' '\e[1;32m' '==> ' '\e[0m\033[1m' 'Please execute: "git checkout ." for complete restore' '\e[m' +} + +# ============================================================================== +# EXECUTION - START +# ============================================================================== +run_rebrand "$@" && run_clean "$@" + +run_make "$@" + +run_copy "$@" && run_reset "$@" diff --git a/devscripts/make_contributing.py b/devscripts/make_contributing.py index a06f8a616..d74462a3c 100755 --- a/devscripts/make_contributing.py +++ b/devscripts/make_contributing.py @@ -20,7 +20,7 @@ def main(): bug_text = re.search( r'(?s)#\s*BUGS\s*[^\n]*\s*(.*?)#\s*COPYRIGHT', readme).group(1) dev_text = re.search( - r'(?s)(#\s*DEVELOPER INSTRUCTIONS.*?)#\s*EMBEDDING yt-dlp', readme).group(1) + r'(?s)(#\s*DEVELOPER INSTRUCTIONS.*?)#\s*EMBEDDING YT-DLP', readme).group(1) out = bug_text + dev_text diff --git a/devscripts/make_issue_template.py b/devscripts/make_issue_template.py deleted file mode 100644 index 54043ef4e..000000000 --- a/devscripts/make_issue_template.py +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -import optparse - - -def read(fname): - with open(fname, encoding='utf-8') as f: - return f.read() - - -# Get the version without importing the package -def read_version(fname): - exec(compile(read(fname), fname, 'exec')) - return locals()['__version__'] - - -def main(): - parser = optparse.OptionParser(usage='%prog INFILE OUTFILE') - options, args = parser.parse_args() - if len(args) != 2: - parser.error('Expected an input and an output filename') - - infile, outfile = args - with open(outfile, 'w', encoding='utf-8') as outf: - outf.write( - read(infile) % {'version': read_version('yt_dlp/version.py')}) - - -if __name__ == '__main__': - main() diff --git a/devscripts/make_readme.py b/devscripts/make_readme.py index f2e08d7c6..f2e08d7c6 100755..100644 --- a/devscripts/make_readme.py +++ b/devscripts/make_readme.py diff --git a/devscripts/prepare_manpage.py b/devscripts/prepare_manpage.py index cea934949..d12ff4947 100644 --- a/devscripts/prepare_manpage.py +++ b/devscripts/prepare_manpage.py @@ -7,7 +7,7 @@ import re ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) README_FILE = os.path.join(ROOT_DIR, 'README.md') -PREFIX = r'''%yt-dlp(1) +PREFIX = r'''%YT-DLP(1) # NAME diff --git a/devscripts/update-formulae.py b/devscripts/update-formulae.py deleted file mode 100644 index 02b869304..000000000 --- a/devscripts/update-formulae.py +++ /dev/null @@ -1,37 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -import json -import re -import urllib.request - -# usage: python3 ./devscripts/update-formulae.py <path-to-formulae-rb> <version> -# version can be either 0-aligned (yt-dlp version) or normalized (PyPl version) - -filename, version = sys.argv[1:] - -normalized_version = '.'.join(str(int(x)) for x in version.split('.')) - -pypi_release = json.loads(urllib.request.urlopen( - 'https://pypi.org/pypi/yt-dlp/%s/json' % normalized_version -).read().decode()) - -tarball_file = next(x for x in pypi_release['urls'] if x['filename'].endswith('.tar.gz')) - -sha256sum = tarball_file['digests']['sha256'] -url = tarball_file['url'] - -with open(filename) as r: - formulae_text = r.read() - -formulae_text = re.sub(r'sha256 "[0-9a-f]*?"', 'sha256 "%s"' % sha256sum, formulae_text) -formulae_text = re.sub(r'url "[^"]*?"', 'url "%s"' % url, formulae_text) - -with open(filename, 'w') as w: - w.write(formulae_text) diff --git a/devscripts/update-version.py b/devscripts/update-version.py deleted file mode 100644 index c5bc83de9..000000000 --- a/devscripts/update-version.py +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -import subprocess -import sys -from datetime import datetime - -with open('yt_dlp/version.py') as f: - exec(compile(f.read(), 'yt_dlp/version.py', 'exec')) -old_version = locals()['__version__'] - -old_version_list = old_version.split('.') - -old_ver = '.'.join(old_version_list[:3]) -old_rev = old_version_list[3] if len(old_version_list) > 3 else '' - -ver = datetime.utcnow().strftime("%Y.%m.%d") - -rev = (sys.argv[1:] or [''])[0] # Use first argument, if present as revision number -if not rev: - rev = str(int(old_rev or 0) + 1) if old_ver == ver else '' - -VERSION = '.'.join((ver, rev)) if rev else ver - -try: - sp = subprocess.Popen(['git', 'rev-parse', '--short', 'HEAD'], stdout=subprocess.PIPE) - GIT_HEAD = sp.communicate()[0].decode().strip() or None -except Exception: - GIT_HEAD = None - -VERSION_FILE = f'''\ -# Autogenerated by devscripts/update-version.py - -__version__ = {VERSION!r} - -RELEASE_GIT_HEAD = {GIT_HEAD!r} -''' - -with open('yt_dlp/version.py', 'wt') as f: - f.write(VERSION_FILE) - -print('::set-output name=ytdlp_version::' + VERSION) -print(f'\nVersion = {VERSION}, Git HEAD = {GIT_HEAD}') @@ -97,7 +97,7 @@ def pycryptodome_module(): try: import Crypto # noqa: F401 print('WARNING: Using Crypto since Cryptodome is not available. ' - 'Install with: pip install pycryptodomex', file=sys.stderr) + 'Install with: pip install pycryptodome', file=sys.stderr) return 'Crypto' except ImportError: pass diff --git a/requirements.txt b/requirements.txt index a48b78d7a..b65d25456 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ mutagen -pycryptodomex +pycryptodome websockets brotli; platform_python_implementation=='CPython' brotlicffi; platform_python_implementation!='CPython' @@ -1,39 +1,29 @@ #!/usr/bin/env python3 - +# coding: utf-8 import os.path -import sys import warnings +import sys try: - from setuptools import Command, find_packages, setup + from setuptools import setup, Command, find_packages setuptools_available = True except ImportError: - from distutils.core import Command, setup + from distutils.core import setup, Command setuptools_available = False from distutils.spawn import spawn - -def read(fname): - with open(fname, encoding='utf-8') as f: - return f.read() - - # Get the version from yt_dlp/version.py without importing the package -def read_version(fname): - exec(compile(read(fname), fname, 'exec')) - return locals()['__version__'] - +exec(compile(open('yt_dlp/version.py').read(), 'yt_dlp/version.py', 'exec')) -VERSION = read_version('yt_dlp/version.py') DESCRIPTION = 'A youtube-dl fork with additional features and patches' LONG_DESCRIPTION = '\n\n'.join(( 'Official repository: <https://github.com/yt-dlp/yt-dlp>', '**PS**: Some links in this document will not work since this is a copy of the README.md from Github', - read('README.md'))) + open('README.md', encoding='utf-8').read())) -REQUIREMENTS = read('requirements.txt').splitlines() +REQUIREMENTS = ['mutagen', 'pycryptodome', 'websockets'] if sys.argv[1:2] == ['py2exe']: @@ -45,11 +35,11 @@ if sys.argv[1:2] == ['py2exe']: 'console': [{ 'script': './yt_dlp/__main__.py', 'dest_base': 'yt-dlp', - 'version': VERSION, + 'version': __version__, 'description': DESCRIPTION, 'comments': LONG_DESCRIPTION.split('\n')[0], 'product_name': 'yt-dlp', - 'product_version': VERSION, + 'product_version': __version__, }], 'options': { 'py2exe': { @@ -59,8 +49,6 @@ if sys.argv[1:2] == ['py2exe']: 'dist_dir': './dist', 'excludes': ['Crypto', 'Cryptodome'], # py2exe cannot import Crypto 'dll_excludes': ['w9xpopen.exe', 'crypt32.dll'], - # Modules that are only imported dynamically must be added here - 'includes': ['yt_dlp.compat._legacy'], } }, 'zipfile': None @@ -118,21 +106,16 @@ else: setup( name='yt-dlp', - version=VERSION, - maintainer='pukkandan', - maintainer_email='pukkandan.ytdlp@gmail.com', + version=__version__, + maintainer='Jesús E..', + maintainer_email='heckyel@hyperbola.info', + license='CC0-1.0', description=DESCRIPTION, long_description=LONG_DESCRIPTION, long_description_content_type='text/markdown', - url='https://github.com/yt-dlp/yt-dlp', + url='https://git.conocimientoslibres.ga/software/hypervideo.git', packages=packages, install_requires=REQUIREMENTS, - project_urls={ - 'Documentation': 'https://github.com/yt-dlp/yt-dlp#readme', - 'Source': 'https://github.com/yt-dlp/yt-dlp', - 'Tracker': 'https://github.com/yt-dlp/yt-dlp/issues', - 'Funding': 'https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md#collaborators', - }, classifiers=[ 'Topic :: Multimedia :: Video', 'Development Status :: 5 - Production/Stable', diff --git a/test/parameters.json b/test/parameters.json index bc4561374..06fe3e31b 100644 --- a/test/parameters.json +++ b/test/parameters.json @@ -44,5 +44,6 @@ "writesubtitles": false, "allsubtitles": false, "listsubtitles": false, + "socket_timeout": 20, "fixup": "never" } diff --git a/test/test_iqiyi_sdk_interpreter.py b/test/test_iqiyi_sdk_interpreter.py deleted file mode 100644 index 47c632a4e..000000000 --- a/test/test_iqiyi_sdk_interpreter.py +++ /dev/null @@ -1,44 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys -import unittest - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -from test.helper import FakeYDL, is_download_test -from yt_dlp.extractor import IqiyiIE - - -class WarningLogger: - def __init__(self): - self.messages = [] - - def warning(self, msg): - self.messages.append(msg) - - def debug(self, msg): - pass - - def error(self, msg): - pass - - -@is_download_test -class TestIqiyiSDKInterpreter(unittest.TestCase): - def test_iqiyi_sdk_interpreter(self): - ''' - Test the functionality of IqiyiSDKInterpreter by trying to log in - - If `sign` is incorrect, /validate call throws an HTTP 556 error - ''' - logger = WarningLogger() - ie = IqiyiIE(FakeYDL({'logger': logger})) - ie._perform_login('foo', 'bar') - self.assertTrue('unable to log in:' in logger.messages[0]) - - -if __name__ == '__main__': - unittest.main() diff --git a/test/test_update.py.disabled b/test/test_update.py.disabled deleted file mode 100644 index 85ac86692..000000000 --- a/test/test_update.py.disabled +++ /dev/null @@ -1,30 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys -import unittest - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -import json - -from yt_dlp.update import rsa_verify - - -class TestUpdate(unittest.TestCase): - def test_rsa_verify(self): - UPDATES_RSA_KEY = (0x9d60ee4d8f805312fdb15a62f87b95bd66177b91df176765d13514a0f1754bcd2057295c5b6f1d35daa6742c3ffc9a82d3e118861c207995a8031e151d863c9927e304576bc80692bc8e094896fcf11b66f3e29e04e3a71e9a11558558acea1840aec37fc396fb6b65dc81a1c4144e03bd1c011de62e3f1357b327d08426fe93, 65537) - with open(os.path.join(os.path.dirname(os.path.abspath(__file__)), 'versions.json'), 'rb') as f: - versions_info = f.read().decode() - versions_info = json.loads(versions_info) - signature = versions_info['signature'] - del versions_info['signature'] - self.assertTrue(rsa_verify( - json.dumps(versions_info, sort_keys=True).encode(), - signature, UPDATES_RSA_KEY)) - - -if __name__ == '__main__': - unittest.main() diff --git a/test/test_write_annotations.py.disabled b/test/test_write_annotations.py.disabled deleted file mode 100644 index c7cf199f6..000000000 --- a/test/test_write_annotations.py.disabled +++ /dev/null @@ -1,77 +0,0 @@ -#!/usr/bin/env python3 - -# Allow direct execution -import os -import sys -import unittest - -sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) - - -import xml.etree.ElementTree - -import yt_dlp.extractor -import yt_dlp.YoutubeDL -from test.helper import get_params, is_download_test, try_rm - - -class YoutubeDL(yt_dlp.YoutubeDL): - def __init__(self, *args, **kwargs): - super().__init__(*args, **kwargs) - self.to_stderr = self.to_screen - - -params = get_params({ - 'writeannotations': True, - 'skip_download': True, - 'writeinfojson': False, - 'format': 'flv', -}) - - -TEST_ID = 'gr51aVj-mLg' -ANNOTATIONS_FILE = TEST_ID + '.annotations.xml' -EXPECTED_ANNOTATIONS = ['Speech bubble', 'Note', 'Title', 'Spotlight', 'Label'] - - -@is_download_test -class TestAnnotations(unittest.TestCase): - def setUp(self): - # Clear old files - self.tearDown() - - def test_info_json(self): - expected = list(EXPECTED_ANNOTATIONS) # Two annotations could have the same text. - ie = yt_dlp.extractor.YoutubeIE() - ydl = YoutubeDL(params) - ydl.add_info_extractor(ie) - ydl.download([TEST_ID]) - self.assertTrue(os.path.exists(ANNOTATIONS_FILE)) - annoxml = None - with open(ANNOTATIONS_FILE, encoding='utf-8') as annof: - annoxml = xml.etree.ElementTree.parse(annof) - self.assertTrue(annoxml is not None, 'Failed to parse annotations XML') - root = annoxml.getroot() - self.assertEqual(root.tag, 'document') - annotationsTag = root.find('annotations') - self.assertEqual(annotationsTag.tag, 'annotations') - annotations = annotationsTag.findall('annotation') - - # Not all the annotations have TEXT children and the annotations are returned unsorted. - for a in annotations: - self.assertEqual(a.tag, 'annotation') - if a.get('type') == 'text': - textTag = a.find('TEXT') - text = textTag.text - self.assertTrue(text in expected) # assertIn only added in python 2.7 - # remove the first occurrence, there could be more than one annotation with the same text - expected.remove(text) - # We should have seen (and removed) all the expected annotation texts. - self.assertEqual(len(expected), 0, 'Not all expected annotations were found.') - - def tearDown(self): - try_rm(ANNOTATIONS_FILE) - - -if __name__ == '__main__': - unittest.main() diff --git a/test/versions.json b/test/versions.json deleted file mode 100644 index 6cccc2259..000000000 --- a/test/versions.json +++ /dev/null @@ -1,34 +0,0 @@ -{ - "latest": "2013.01.06", - "signature": "72158cdba391628569ffdbea259afbcf279bbe3d8aeb7492690735dc1cfa6afa754f55c61196f3871d429599ab22f2667f1fec98865527b32632e7f4b3675a7ef0f0fbe084d359256ae4bba68f0d33854e531a70754712f244be71d4b92e664302aa99653ee4df19800d955b6c4149cd2b3f24288d6e4b40b16126e01f4c8ce6", - "versions": { - "2013.01.02": { - "bin": [ - "http://youtube-dl.org/downloads/2013.01.02/youtube-dl", - "f5b502f8aaa77675c4884938b1e4871ebca2611813a0c0e74f60c0fbd6dcca6b" - ], - "exe": [ - "http://youtube-dl.org/downloads/2013.01.02/youtube-dl.exe", - "75fa89d2ce297d102ff27675aa9d92545bbc91013f52ec52868c069f4f9f0422" - ], - "tar": [ - "http://youtube-dl.org/downloads/2013.01.02/youtube-dl-2013.01.02.tar.gz", - "6a66d022ac8e1c13da284036288a133ec8dba003b7bd3a5179d0c0daca8c8196" - ] - }, - "2013.01.06": { - "bin": [ - "http://youtube-dl.org/downloads/2013.01.06/youtube-dl", - "64b6ed8865735c6302e836d4d832577321b4519aa02640dc508580c1ee824049" - ], - "exe": [ - "http://youtube-dl.org/downloads/2013.01.06/youtube-dl.exe", - "58609baf91e4389d36e3ba586e21dab882daaaee537e4448b1265392ae86ff84" - ], - "tar": [ - "http://youtube-dl.org/downloads/2013.01.06/youtube-dl-2013.01.06.tar.gz", - "fe77ab20a95d980ed17a659aa67e371fdd4d656d19c4c7950e7b720b0c2f1a86" - ] - } - } -}
\ No newline at end of file diff --git a/yt-dlp.sh b/yt-dlp.sh index 22a69250c..22a69250c 100755..100644 --- a/yt-dlp.sh +++ b/yt-dlp.sh diff --git a/yt_dlp/YoutubeDL.py b/yt_dlp/YoutubeDL.py index c6882d0d7..de8a8c4d2 100644 --- a/yt_dlp/YoutubeDL.py +++ b/yt_dlp/YoutubeDL.py @@ -47,7 +47,6 @@ from .postprocessor import ( MoveFilesAfterDownloadPP, get_postprocessor, ) -from .update import detect_variant from .utils import ( DEFAULT_OUTTMPL, IDENTITY, diff --git a/yt_dlp/__init__.py b/yt_dlp/__init__.py index 357be861b..0c68f8571 100644 --- a/yt_dlp/__init__.py +++ b/yt_dlp/__init__.py @@ -1,6 +1,7 @@ +#!/usr/bin/python f'You are using an unsupported version of Python. Only Python versions 3.6 and above are supported by yt-dlp' # noqa: F541 -__license__ = 'Public Domain' +__license__ = 'CC0-1.0' import getpass import itertools @@ -88,27 +89,28 @@ def print_extractor_information(opts, urls): out = '' if opts.list_extractors: - urls = dict.fromkeys(urls, False) - for ie in list_extractor_classes(opts.age_limit): - out += ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n' - if ie == GenericIE: - matched_urls = [url for url, matched in urls.items() if not matched] - else: - matched_urls = tuple(filter(ie.suitable, urls.keys())) - urls.update(dict.fromkeys(matched_urls, True)) - out += ''.join(f' {url}\n' for url in matched_urls) + for ie in list_extractors(opts.age_limit): + write_string(ie.IE_NAME + (' (CURRENTLY BROKEN)' if not ie.working() else '') + '\n', out=sys.stdout) + matchedUrls = [url for url in urls if ie.suitable(url)] + for mu in matchedUrls: + write_string(' ' + mu + '\n', out=sys.stdout) elif opts.list_extractor_descriptions: - _SEARCHES = ('cute kittens', 'slithering pythons', 'falling cat', 'angry poodle', 'purple fish', 'running tortoise', 'sleeping bunny', 'burping cow') - out = '\n'.join( - ie.description(markdown=False, search_examples=_SEARCHES) - for ie in list_extractor_classes(opts.age_limit) if ie.working() and ie.IE_DESC is not False) + for ie in list_extractors(opts.age_limit): + if not ie.working(): + continue + if ie.IE_DESC is False: + continue + desc = ie.IE_DESC or ie.IE_NAME + if getattr(ie, 'SEARCH_KEY', None) is not None: + _SEARCHES = ('cute kittens', 'slithering pythons', 'falling cat', 'angry poodle', 'purple fish', 'running tortoise', 'sleeping bunny', 'burping cow') + _COUNTS = ('', '5', '10', 'all') + desc += f'; "{ie.SEARCH_KEY}:" prefix (Example: "{ie.SEARCH_KEY}{random.choice(_COUNTS)}:{random.choice(_SEARCHES)}")' + write_string(desc + '\n', out=sys.stdout) elif opts.ap_list_mso: - out = 'Supported TV Providers:\n%s\n' % render_table( - ['mso', 'mso name'], - [[mso_id, mso_info['name']] for mso_id, mso_info in MSO_INFO.items()]) + table = [[mso_id, mso_info['name']] for mso_id, mso_info in MSO_INFO.items()] + write_string('Supported TV Providers:\n' + render_table(['mso', 'mso name'], table) + '\n', out=sys.stdout) else: return False - write_string(out, out=sys.stdout) return True @@ -457,15 +459,13 @@ def validate_options(opts): report_conflict('--playlist-random', 'playlist_random', '--lazy-playlist', 'lazy_playlist') report_conflict('--dateafter', 'dateafter', '--date', 'date', default=None) report_conflict('--datebefore', 'datebefore', '--date', 'date', default=None) - report_conflict('--exec-before-download', 'exec_before_dl_cmd', - '"--exec before_dl:"', 'exec_cmd', val2=opts.exec_cmd.get('before_dl')) + report_conflict('--exec-before-download', 'exec_before_dl_cmd', '"--exec before_dl:"', 'exec_cmd', opts.exec_cmd.get('before_dl')) report_conflict('--id', 'useid', '--output', 'outtmpl', val2=opts.outtmpl.get('default')) report_conflict('--remux-video', 'remuxvideo', '--recode-video', 'recodevideo') report_conflict('--sponskrub', 'sponskrub', '--remove-chapters', 'remove_chapters') report_conflict('--sponskrub', 'sponskrub', '--sponsorblock-mark', 'sponsorblock_mark') report_conflict('--sponskrub', 'sponskrub', '--sponsorblock-remove', 'sponsorblock_remove') - report_conflict('--sponskrub-cut', 'sponskrub_cut', '--split-chapter', 'split_chapters', - val1=opts.sponskrub and opts.sponskrub_cut) + report_conflict('--sponskrub-cut', 'sponskrub_cut', '--split-chapter', 'split_chapters', val1=opts.sponskrub and opts.sponskrub_cut) # Conflicts with --allow-unplayable-formats report_conflict('--add-metadata', 'addmetadata') @@ -474,7 +474,7 @@ def validate_options(opts): report_conflict('--embed-subs', 'embedsubtitles') report_conflict('--embed-thumbnail', 'embedthumbnail') report_conflict('--extract-audio', 'extractaudio') - report_conflict('--fixup', 'fixup', val1=opts.fixup not in (None, 'never', 'ignore'), default='never') + report_conflict('--fixup', 'fixup', val1=(opts.fixup or '').lower() in ('', 'never', 'ignore'), default='never') report_conflict('--recode-video', 'recodevideo') report_conflict('--remove-chapters', 'remove_chapters', default=[]) report_conflict('--remux-video', 'remuxvideo') @@ -678,11 +678,11 @@ def parse_options(argv=None): postprocessors = list(get_postprocessors(opts)) - print_only = bool(opts.forceprint) and all(k not in opts.forceprint for k in POSTPROCESS_WHEN[2:]) - any_getting = any(getattr(opts, k) for k in ( - 'dumpjson', 'dump_single_json', 'getdescription', 'getduration', 'getfilename', - 'getformat', 'getid', 'getthumbnail', 'gettitle', 'geturl' - )) + any_getting = (any(opts.forceprint.values()) or opts.dumpjson or opts.dump_single_json + or opts.geturl or opts.gettitle or opts.getid or opts.getthumbnail + or opts.getdescription or opts.getfilename or opts.getformat or opts.getduration) + + any_printing = opts.print_json final_ext = ( opts.recodevideo if opts.recodevideo in FFmpegVideoConvertorPP.SUPPORTED_EXTS @@ -700,10 +700,7 @@ def parse_options(argv=None): 'ap_mso': opts.ap_mso, 'ap_username': opts.ap_username, 'ap_password': opts.ap_password, - 'client_certificate': opts.client_certificate, - 'client_certificate_key': opts.client_certificate_key, - 'client_certificate_password': opts.client_certificate_password, - 'quiet': opts.quiet or any_getting or opts.print_json or bool(opts.forceprint), + 'quiet': (opts.quiet or any_getting or any_printing), 'no_warnings': opts.no_warnings, 'forceurl': opts.geturl, 'forcetitle': opts.gettitle, @@ -718,7 +715,7 @@ def parse_options(argv=None): 'forcejson': opts.dumpjson or opts.print_json, 'dump_single_json': opts.dump_single_json, 'force_write_download_archive': opts.force_write_download_archive, - 'simulate': (print_only or any_getting or None) if opts.simulate is None else opts.simulate, + 'simulate': (any_getting or None) if opts.simulate is None else opts.simulate, 'skip_download': opts.skip_download, 'format': opts.format, 'allow_unplayable_formats': opts.allow_unplayable_formats, @@ -867,6 +864,13 @@ def parse_options(argv=None): def _real_main(argv=None): + # Compatibility fixes for Windows + if sys.platform == 'win32': + # https://github.com/ytdl-org/youtube-dl/issues/820 + codecs.register(lambda name: codecs.lookup('utf-8') if name == 'cp65001' else None) + + workaround_optparse_bug9161() + setproctitle('yt-dlp') parser, opts, all_urls, ydl_opts = parse_options(argv) diff --git a/yt_dlp/downloader/hls.py b/yt_dlp/downloader/hls.py index 1e75c5e9c..f54b3f473 100644 --- a/yt_dlp/downloader/hls.py +++ b/yt_dlp/downloader/hls.py @@ -65,9 +65,9 @@ class HlsFD(FragmentFD): has_ffmpeg = FFmpegFD.available() no_crypto = not Cryptodome_AES and '#EXT-X-KEY:METHOD=AES-128' in s if no_crypto and has_ffmpeg: - can_download, message = False, 'The stream has AES-128 encryption and pycryptodomex is not available' + can_download, message = False, 'The stream has AES-128 encryption and pycryptodome is not available' elif no_crypto: - message = ('The stream has AES-128 encryption and neither ffmpeg nor pycryptodomex are available; ' + message = ('The stream has AES-128 encryption and neither ffmpeg nor pycryptodome are available; ' 'Decryption will be performed natively, but will be extremely slow') elif re.search(r'#EXT-X-MEDIA-SEQUENCE:(?!0$)', s): install_ffmpeg = '' if has_ffmpeg else 'install ffmpeg and ' diff --git a/yt_dlp/extractor/iqiyi.py b/yt_dlp/extractor/iqiyi.py index 35691ec20..5c316687c 100644 --- a/yt_dlp/extractor/iqiyi.py +++ b/yt_dlp/extractor/iqiyi.py @@ -6,13 +6,11 @@ import time from .common import InfoExtractor from ..compat import ( compat_str, - compat_urllib_parse_urlencode, compat_urllib_parse_unquote ) from .openload import PhantomJSwrapper from ..utils import ( clean_html, - decode_packed_codes, ExtractorError, float_or_none, format_field, @@ -37,135 +35,6 @@ def md5_text(text): return hashlib.md5(text.encode('utf-8')).hexdigest() -class IqiyiSDK: - def __init__(self, target, ip, timestamp): - self.target = target - self.ip = ip - self.timestamp = timestamp - - @staticmethod - def split_sum(data): - return compat_str(sum(map(lambda p: int(p, 16), list(data)))) - - @staticmethod - def digit_sum(num): - if isinstance(num, int): - num = compat_str(num) - return compat_str(sum(map(int, num))) - - def even_odd(self): - even = self.digit_sum(compat_str(self.timestamp)[::2]) - odd = self.digit_sum(compat_str(self.timestamp)[1::2]) - return even, odd - - def preprocess(self, chunksize): - self.target = md5_text(self.target) - chunks = [] - for i in range(32 // chunksize): - chunks.append(self.target[chunksize * i:chunksize * (i + 1)]) - if 32 % chunksize: - chunks.append(self.target[32 - 32 % chunksize:]) - return chunks, list(map(int, self.ip.split('.'))) - - def mod(self, modulus): - chunks, ip = self.preprocess(32) - self.target = chunks[0] + ''.join(map(lambda p: compat_str(p % modulus), ip)) - - def split(self, chunksize): - modulus_map = { - 4: 256, - 5: 10, - 8: 100, - } - - chunks, ip = self.preprocess(chunksize) - ret = '' - for i in range(len(chunks)): - ip_part = compat_str(ip[i] % modulus_map[chunksize]) if i < 4 else '' - if chunksize == 8: - ret += ip_part + chunks[i] - else: - ret += chunks[i] + ip_part - self.target = ret - - def handle_input16(self): - self.target = md5_text(self.target) - self.target = self.split_sum(self.target[:16]) + self.target + self.split_sum(self.target[16:]) - - def handle_input8(self): - self.target = md5_text(self.target) - ret = '' - for i in range(4): - part = self.target[8 * i:8 * (i + 1)] - ret += self.split_sum(part) + part - self.target = ret - - def handleSum(self): - self.target = md5_text(self.target) - self.target = self.split_sum(self.target) + self.target - - def date(self, scheme): - self.target = md5_text(self.target) - d = time.localtime(self.timestamp) - strings = { - 'y': compat_str(d.tm_year), - 'm': '%02d' % d.tm_mon, - 'd': '%02d' % d.tm_mday, - } - self.target += ''.join(map(lambda c: strings[c], list(scheme))) - - def split_time_even_odd(self): - even, odd = self.even_odd() - self.target = odd + md5_text(self.target) + even - - def split_time_odd_even(self): - even, odd = self.even_odd() - self.target = even + md5_text(self.target) + odd - - def split_ip_time_sum(self): - chunks, ip = self.preprocess(32) - self.target = compat_str(sum(ip)) + chunks[0] + self.digit_sum(self.timestamp) - - def split_time_ip_sum(self): - chunks, ip = self.preprocess(32) - self.target = self.digit_sum(self.timestamp) + chunks[0] + compat_str(sum(ip)) - - -class IqiyiSDKInterpreter: - def __init__(self, sdk_code): - self.sdk_code = sdk_code - - def run(self, target, ip, timestamp): - self.sdk_code = decode_packed_codes(self.sdk_code) - - functions = re.findall(r'input=([a-zA-Z0-9]+)\(input', self.sdk_code) - - sdk = IqiyiSDK(target, ip, timestamp) - - other_functions = { - 'handleSum': sdk.handleSum, - 'handleInput8': sdk.handle_input8, - 'handleInput16': sdk.handle_input16, - 'splitTimeEvenOdd': sdk.split_time_even_odd, - 'splitTimeOddEven': sdk.split_time_odd_even, - 'splitIpTimeSum': sdk.split_ip_time_sum, - 'splitTimeIpSum': sdk.split_time_ip_sum, - } - for function in functions: - if re.match(r'mod\d+', function): - sdk.mod(int(function[3:])) - elif re.match(r'date[ymd]{3}', function): - sdk.date(function[4:]) - elif re.match(r'split\d+', function): - sdk.split(int(function[5:])) - elif function in other_functions: - other_functions[function]() - else: - raise ExtractorError('Unknown function %s' % function) - - return sdk.target - - class IqiyiIE(InfoExtractor): IE_NAME = 'iqiyi' IE_DESC = '爱奇艺' @@ -246,47 +115,8 @@ class IqiyiIE(InfoExtractor): return ohdave_rsa_encrypt(data, e, N) - def _perform_login(self, username, password): - - data = self._download_json( - 'http://kylin.iqiyi.com/get_token', None, - note='Get token for logging', errnote='Unable to get token for logging') - sdk = data['sdk'] - timestamp = int(time.time()) - target = '/apis/reglogin/login.action?lang=zh_TW&area_code=null&email=%s&passwd=%s&agenttype=1&from=undefined&keeplogin=0&piccode=&fromurl=&_pos=1' % ( - username, self._rsa_fun(password.encode('utf-8'))) - - interp = IqiyiSDKInterpreter(sdk) - sign = interp.run(target, data['ip'], timestamp) - - validation_params = { - 'target': target, - 'server': 'BEA3AA1908656AABCCFF76582C4C6660', - 'token': data['token'], - 'bird_src': 'f8d91d57af224da7893dd397d52d811a', - 'sign': sign, - 'bird_t': timestamp, - } - validation_result = self._download_json( - 'http://kylin.iqiyi.com/validate?' + compat_urllib_parse_urlencode(validation_params), None, - note='Validate credentials', errnote='Unable to validate credentials') - - MSG_MAP = { - 'P00107': 'please login via the web interface and enter the CAPTCHA code', - 'P00117': 'bad username or password', - } - - code = validation_result['code'] - if code != 'A00000': - msg = MSG_MAP.get(code) - if not msg: - msg = 'error %s' % code - if validation_result.get('msg'): - msg += ': ' + validation_result['msg'] - self.report_warning('unable to log in: ' + msg) - return False - - return True + def _perform_login(self): + raise ExtractorError("iQiyi's non-free authentication algorithm has made login impossible", expected=True) def get_raw_data(self, tvid, video_id): tm = int(time.time() * 1000) diff --git a/yt_dlp/extractor/ivi.py b/yt_dlp/extractor/ivi.py index 699746943..f469a6adf 100644 --- a/yt_dlp/extractor/ivi.py +++ b/yt_dlp/extractor/ivi.py @@ -138,7 +138,7 @@ class IviIE(InfoExtractor): elif site == 353: continue elif not pycryptodome_found: - raise ExtractorError('pycryptodomex not found. Please install', expected=True) + raise ExtractorError('pycryptodome not found. Please install', expected=True) elif message: extractor_msg += ': ' + message raise ExtractorError(extractor_msg % video_id, expected=True) diff --git a/yt_dlp/options.py b/yt_dlp/options.py index 8c9a9bbb4..dddd5b15b 100644 --- a/yt_dlp/options.py +++ b/yt_dlp/options.py @@ -19,7 +19,6 @@ from .postprocessor import ( SponsorBlockPP, ) from .postprocessor.modify_chapters import DEFAULT_SPONSORBLOCK_CHAPTER_TITLE -from .update import detect_variant from .utils import ( OUTTMPL_TYPES, POSTPROCESS_WHEN, @@ -325,14 +324,6 @@ def create_parser(): action='version', help='Print program version and exit') general.add_option( - '-U', '--update', - action='store_true', dest='update_self', - help='Update this program to latest version') - general.add_option( - '--no-update', - action='store_false', dest='update_self', - help='Do not update (default)') - general.add_option( '-i', '--ignore-errors', action='store_true', dest='ignoreerrors', help='Ignore download and postprocessing errors. The download will be considered successful even if the postprocessing fails') diff --git a/yt_dlp/update.py b/yt_dlp/update.py deleted file mode 100644 index 8e34f2127..000000000 --- a/yt_dlp/update.py +++ /dev/null @@ -1,283 +0,0 @@ -import atexit -import hashlib -import json -import os -import platform -import subprocess -import sys -from zipimport import zipimporter - -from .compat import functools # isort: split -from .compat import compat_realpath -from .utils import Popen, shell_quote, traverse_obj, version_tuple -from .version import __version__ - -REPOSITORY = 'yt-dlp/yt-dlp' -API_URL = f'https://api.github.com/repos/{REPOSITORY}/releases/latest' - - -@functools.cache -def _get_variant_and_executable_path(): - """@returns (variant, executable_path)""" - if hasattr(sys, 'frozen'): - path = sys.executable - if not hasattr(sys, '_MEIPASS'): - return 'py2exe', path - if sys._MEIPASS == os.path.dirname(path): - return f'{sys.platform}_dir', path - return f'{sys.platform}_exe', path - - path = os.path.dirname(__file__) - if isinstance(__loader__, zipimporter): - return 'zip', os.path.join(path, '..') - elif (os.path.basename(sys.argv[0]) in ('__main__.py', '-m') - and os.path.exists(os.path.join(path, '../.git/HEAD'))): - return 'source', path - return 'unknown', path - - -def detect_variant(): - return _get_variant_and_executable_path()[0] - - -_FILE_SUFFIXES = { - 'zip': '', - 'py2exe': '_min.exe', - 'win32_exe': '.exe', - 'darwin_exe': '_macos', - 'linux_exe': '_linux', -} - -_NON_UPDATEABLE_REASONS = { - **{variant: None for variant in _FILE_SUFFIXES}, # Updatable - **{variant: f'Auto-update is not supported for unpackaged {name} executable; Re-download the latest release' - for variant, name in {'win32_dir': 'Windows', 'darwin_dir': 'MacOS', 'linux_dir': 'Linux'}.items()}, - 'source': 'You cannot update when running from source code; Use git to pull the latest changes', - 'unknown': 'It looks like you installed yt-dlp with a package manager, pip or setup.py; Use that to update', - 'other': 'It looks like you are using an unofficial build of yt-dlp; Build the executable again', -} - - -def is_non_updateable(): - return _NON_UPDATEABLE_REASONS.get(detect_variant(), _NON_UPDATEABLE_REASONS['other']) - - -def _sha256_file(path): - h = hashlib.sha256() - mv = memoryview(bytearray(128 * 1024)) - with open(os.path.realpath(path), 'rb', buffering=0) as f: - for n in iter(lambda: f.readinto(mv), 0): - h.update(mv[:n]) - return h.hexdigest() - - -class Updater: - def __init__(self, ydl): - self.ydl = ydl - - @functools.cached_property - def _new_version_info(self): - self.ydl.write_debug(f'Fetching release info: {API_URL}') - return json.loads(self.ydl.urlopen(API_URL).read().decode()) - - @property - def current_version(self): - """Current version""" - return __version__ - - @property - def new_version(self): - """Version of the latest release""" - return self._new_version_info['tag_name'] - - @property - def has_update(self): - """Whether there is an update available""" - return version_tuple(__version__) < version_tuple(self.new_version) - - @functools.cached_property - def filename(self): - """Filename of the executable""" - return compat_realpath(_get_variant_and_executable_path()[1]) - - def _download(self, name=None): - name = name or self.release_name - url = traverse_obj(self._new_version_info, ( - 'assets', lambda _, v: v['name'] == name, 'browser_download_url'), get_all=False) - if not url: - raise Exception('Unable to find download URL') - self.ydl.write_debug(f'Downloading {name} from {url}') - return self.ydl.urlopen(url).read() - - @functools.cached_property - def release_name(self): - """The release filename""" - label = _FILE_SUFFIXES[detect_variant()] - if label and platform.architecture()[0][:2] == '32': - label = f'_x86{label}' - return f'yt-dlp{label}' - - @functools.cached_property - def release_hash(self): - """Hash of the latest release""" - hash_data = dict(ln.split()[::-1] for ln in self._download('SHA2-256SUMS').decode().splitlines()) - return hash_data[self.release_name] - - def _report_error(self, msg, expected=False): - self.ydl.report_error(msg, tb=False if expected else None) - - def _report_permission_error(self, file): - self._report_error(f'Unable to write to {file}; Try running as administrator', True) - - def _report_network_error(self, action, delim=';'): - self._report_error(f'Unable to {action}{delim} Visit https://github.com/{REPOSITORY}/releases/latest', True) - - def check_update(self): - """Report whether there is an update available""" - try: - self.ydl.to_screen( - f'Latest version: {self.new_version}, Current version: {self.current_version}') - except Exception: - return self._report_network_error('obtain version info', delim='; Please try again later or') - - if not self.has_update: - return self.ydl.to_screen(f'yt-dlp is up to date ({__version__})') - - if not is_non_updateable(): - self.ydl.to_screen(f'Current Build Hash {_sha256_file(self.filename)}') - return True - - def update(self): - """Update yt-dlp executable to the latest version""" - if not self.check_update(): - return - err = is_non_updateable() - if err: - return self._report_error(err, True) - self.ydl.to_screen(f'Updating to version {self.new_version} ...') - - directory = os.path.dirname(self.filename) - if not os.access(self.filename, os.W_OK): - return self._report_permission_error(self.filename) - elif not os.access(directory, os.W_OK): - return self._report_permission_error(directory) - - new_filename, old_filename = f'{self.filename}.new', f'{self.filename}.old' - if detect_variant() == 'zip': # Can be replaced in-place - new_filename, old_filename = self.filename, None - - try: - if os.path.exists(old_filename or ''): - os.remove(old_filename) - except OSError: - return self._report_error('Unable to remove the old version') - - try: - newcontent = self._download() - except OSError: - return self._report_network_error('download latest version') - except Exception: - return self._report_network_error('fetch updates') - - try: - expected_hash = self.release_hash - except Exception: - self.ydl.report_warning('no hash information found for the release') - else: - if hashlib.sha256(newcontent).hexdigest() != expected_hash: - return self._report_network_error('verify the new executable') - - try: - with open(new_filename, 'wb') as outf: - outf.write(newcontent) - except OSError: - return self._report_permission_error(new_filename) - - try: - if old_filename: - os.rename(self.filename, old_filename) - except OSError: - return self._report_error('Unable to move current version') - try: - if old_filename: - os.rename(new_filename, self.filename) - except OSError: - self._report_error('Unable to overwrite current version') - return os.rename(old_filename, self.filename) - - if detect_variant() not in ('win32_exe', 'py2exe'): - if old_filename: - os.remove(old_filename) - else: - atexit.register(Popen, f'ping 127.0.0.1 -n 5 -w 1000 & del /F "{old_filename}"', - shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - - self.ydl.to_screen(f'Updated yt-dlp to version {self.new_version}') - return True - - @functools.cached_property - def cmd(self): - """The command-line to run the executable, if known""" - # There is no sys.orig_argv in py < 3.10. Also, it can be [] when frozen - if getattr(sys, 'orig_argv', None): - return sys.orig_argv - elif hasattr(sys, 'frozen'): - return sys.argv - - def restart(self): - """Restart the executable""" - assert self.cmd, 'Must be frozen or Py >= 3.10' - self.ydl.write_debug(f'Restarting: {shell_quote(self.cmd)}') - _, _, returncode = Popen.run(self.cmd) - return returncode - - -def run_update(ydl): - """Update the program file with the latest version from the repository - @returns Whether there was a successfull update (No update = False) - """ - return Updater(ydl).update() - - -# Deprecated -def update_self(to_screen, verbose, opener): - import traceback - - from .utils import write_string - - write_string( - 'DeprecationWarning: "yt_dlp.update.update_self" is deprecated and may be removed in a future version. ' - 'Use "yt_dlp.update.run_update(ydl)" instead\n') - - printfn = to_screen - - class FakeYDL(): - to_screen = printfn - - def report_warning(self, msg, *args, **kwargs): - return printfn(f'WARNING: {msg}', *args, **kwargs) - - def report_error(self, msg, tb=None): - printfn(f'ERROR: {msg}') - if not verbose: - return - if tb is None: - # Copied from YoutubeDL.trouble - if sys.exc_info()[0]: - tb = '' - if hasattr(sys.exc_info()[1], 'exc_info') and sys.exc_info()[1].exc_info[0]: - tb += ''.join(traceback.format_exception(*sys.exc_info()[1].exc_info)) - tb += traceback.format_exc() - else: - tb_data = traceback.format_list(traceback.extract_stack()) - tb = ''.join(tb_data) - if tb: - printfn(tb) - - def write_debug(self, msg, *args, **kwargs): - printfn(f'[debug] {msg}', *args, **kwargs) - - def urlopen(self, url): - return opener.open(url) - - return run_update(FakeYDL()) diff --git a/yt_dlp/utils.py b/yt_dlp/utils.py index 46a6c9fce..fba64be5a 100644 --- a/yt_dlp/utils.py +++ b/yt_dlp/utils.py @@ -980,10 +980,9 @@ def make_HTTPS_handler(params, **kwargs): def bug_reports_message(before=';'): - from .update import REPOSITORY - - msg = (f'please report this issue on https://github.com/{REPOSITORY}/issues?q= , ' - 'filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U') + msg = ('please report this issue on https://issues.hyperbola.info/ , ' + 'filling out the appropriate issue template. ' + 'Confirm you are on the latest version using pacman -Su') before = before.rstrip() if not before or before.endswith(('.', '!', '?')): diff --git a/ytdlp_plugins/extractor/__init__.py b/ytdlp_plugins/extractor/__init__.py deleted file mode 100644 index 3045a590b..000000000 --- a/ytdlp_plugins/extractor/__init__.py +++ /dev/null @@ -1,4 +0,0 @@ -# flake8: noqa: F401 - -# ℹ️ The imported name must end in "IE" -from .sample import SamplePluginIE diff --git a/ytdlp_plugins/extractor/sample.py b/ytdlp_plugins/extractor/sample.py deleted file mode 100644 index 82c0af459..000000000 --- a/ytdlp_plugins/extractor/sample.py +++ /dev/null @@ -1,14 +0,0 @@ -# ⚠ Don't use relative imports -from yt_dlp.extractor.common import InfoExtractor - - -# ℹ️ Instructions on making extractors can be found at: -# 🔗 https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-support-for-a-new-site - -class SamplePluginIE(InfoExtractor): - _WORKING = False - IE_DESC = False - _VALID_URL = r'^sampleplugin:' - - def _real_extract(self, url): - self.to_screen('URL "%s" sucessfully captured' % url) diff --git a/ytdlp_plugins/postprocessor/__init__.py b/ytdlp_plugins/postprocessor/__init__.py deleted file mode 100644 index 61099abbc..000000000 --- a/ytdlp_plugins/postprocessor/__init__.py +++ /dev/null @@ -1,4 +0,0 @@ -# flake8: noqa: F401 - -# ℹ️ The imported name must end in "PP" and is the name to be used in --use-postprocessor -from .sample import SamplePluginPP diff --git a/ytdlp_plugins/postprocessor/sample.py b/ytdlp_plugins/postprocessor/sample.py deleted file mode 100644 index 4563e1c11..000000000 --- a/ytdlp_plugins/postprocessor/sample.py +++ /dev/null @@ -1,26 +0,0 @@ -# ⚠ Don't use relative imports -from yt_dlp.postprocessor.common import PostProcessor - - -# ℹ️ See the docstring of yt_dlp.postprocessor.common.PostProcessor -class SamplePluginPP(PostProcessor): - def __init__(self, downloader=None, **kwargs): - # ⚠ Only kwargs can be passed from the CLI, and all argument values will be string - # Also, "downloader", "when" and "key" are reserved names - super().__init__(downloader) - self._kwargs = kwargs - - # ℹ️ See docstring of yt_dlp.postprocessor.common.PostProcessor.run - def run(self, info): - if info.get('_type', 'video') != 'video': # PP was called for playlist - self.to_screen(f'Post-processing playlist {info.get("id")!r} with {self._kwargs}') - elif info.get('filepath'): # PP was called after download (default) - filepath = info.get('filepath') - self.to_screen(f'Post-processed {filepath!r} with {self._kwargs}') - elif info.get('requested_downloads'): # PP was called after_video - filepaths = [f.get('filepath') for f in info.get('requested_downloads')] - self.to_screen(f'Post-processed {filepaths!r} with {self._kwargs}') - else: # PP was called before actual download - filepath = info.get('_filename') - self.to_screen(f'Pre-processed {filepath!r} with {self._kwargs}') - return [], info # return list_of_files_to_delete, info_dict |
