DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
Checklist
Provide a description that is worded well enough to be understood
When using .netrc authentication, if you have a password that contains a backslash character \, it will be treated as an escape character for the next character instead of the backslash literal.
Say for example, your password was a\b and the content of your .netrc file was
machine watchnebula login test@example.net password a\b
yt-dlp would then interpret the password as ab as b escaped just turns into b again. The same does not occur when using --username authentication. If you type a\b in the password prompt, yt-dlp interprets the backslash character as a literal backslash and the entire password as a\b.
I believe this is not intentional behavior since:
- Your readme section Authentication with netrc does not mention the need to manually escape backslashes in passwords
- Neither does the website about .netrc files you link to in the readme
- I couldn't find any information on the internet about the need to manually escape backslashes in passwords in
.netrc files.
In the Complete Verbose Output section of this issue, I have slightly modified the code in order to better demonstrate the issue. I have added a print(password) statement to the _get_login_info function in yt-dlp/yt_dlp/extractor/common.py just before
|
return username, password |
so we can see the difference in interpreting the password when running the verbose example with
--netrc and with
--username. See the line below
yt-dlp is up to date (stable@2023.07.06) in the respective verbose outputs.
In my example I use the nebula extractor since this is where I first noticed the issue, however from what I can tell the issue seems to be unrelated to that specific extractor but rather be a core issue.
In order to answer the questions of "How it could be fixed", I would suggest escaping backslashes in .netrc passwords so that backslash literals are still interpreted as backslash literals. In case I am wrong with my assessment and that is actually the intended behavior I would instead appreciate if the need to manually escape backslashes in passwords in netrc authentication was mentioned in the Authentication with netrc section in the readme.
Provide verbose output that clearly demonstrates the problem
Complete Verbose Output
# using .netrc authentication
~ yt-dlp -vU https://nebula.tv/videos/jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan --netrc
[debug] Command-line config: ['-vU', 'https://nebula.tv/videos/jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan', '--netrc']
[debug] User config "/home/dasug/.yt-dlp/config": []
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481]
[debug] Python 3.11.3 (CPython x86_64 64bit) - Linux-6.4.3-zen1-2-zen-x86_64-with-glibc2.37 (OpenSSL 3.1.1 30 May 2023, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, brotlicffi-1.0.9.2, certifi-2023.05.07, mutagen-1.46.0, secretstorage-3.3.3, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
yt-dlp is up to date (stable@2023.07.06)
ab
[Nebula] Logging in to Nebula with supplied credentials
WARNING: [Nebula] Authentication failed or rejected: HTTP Error 400: Bad Request
ERROR: [Nebula] jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan: This video is only available for registered users. Use --username and --password, --netrc-cmd, or --netrc (watchnebula) to provide account credentials
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 707, in extract
self.initialize()
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 613, in initialize
self._perform_login(username, password)
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/nebula.py", line 109, in _perform_login
self._nebula_api_token = self._perform_nebula_auth(username, password)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/nebula.py", line 33, in _perform_nebula_auth
self.raise_login_required(method='password')
File "/usr/lib/python3.11/site-packages/yt_dlp/extractor/common.py", line 1170, in raise_login_required
raise ExtractorError(msg, expected=True)
# using --username authentication
~ yt-dlp -vU https://nebula.tv/videos/jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan --username="test@example.net"
[debug] Command-line config: ['-vU', 'https://nebula.tv/videos/jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan', '--username=PRIVATE']
[debug] User config "/home/dasug/.yt-dlp/config": []
Type account password and press [Return]:
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.07.06 [b532a3481]
[debug] Python 3.11.3 (CPython x86_64 64bit) - Linux-6.4.3-zen1-2-zen-x86_64-with-glibc2.37 (OpenSSL 3.1.1 30 May 2023, glibc 2.37)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, brotlicffi-1.0.9.2, certifi-2023.05.07, mutagen-1.46.0, secretstorage-3.3.3, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.07.06, Current version: stable@2023.07.06
yt-dlp is up to date (stable@2023.07.06)
a\b
[Nebula] Logging in to Nebula with supplied credentials
[Nebula] Authorizing to Nebula
[Nebula] Extracting URL: https://nebula.tv/videos/jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan
[Nebula] jetlag-ep-4-we-played-a-96hour-game-of-capture-the-flag-across-japan: Fetching video meta data
^C
ERROR: Interrupted by user
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
Checklist
Provide a description that is worded well enough to be understood
When using
.netrcauthentication, if you have a password that contains a backslash character\, it will be treated as an escape character for the next character instead of the backslash literal.Say for example, your password was
a\band the content of your.netrcfile wasyt-dlp would then interpret the password as
abas b escaped just turns into b again. The same does not occur when using--usernameauthentication. If you typea\bin the password prompt, yt-dlp interprets the backslash character as a literal backslash and the entire password asa\b.I believe this is not intentional behavior since:
.netrcfiles.In the Complete Verbose Output section of this issue, I have slightly modified the code in order to better demonstrate the issue. I have added a
print(password)statement to the_get_login_infofunction inyt-dlp/yt_dlp/extractor/common.pyjust beforeyt-dlp/yt_dlp/extractor/common.py
Line 1347 in 9b16762
so we can see the difference in interpreting the password when running the verbose example with
--netrcand with--username. See the line belowyt-dlp is up to date (stable@2023.07.06)in the respective verbose outputs.In my example I use the
nebulaextractor since this is where I first noticed the issue, however from what I can tell the issue seems to be unrelated to that specific extractor but rather be a core issue.In order to answer the questions of "How it could be fixed", I would suggest escaping backslashes in
.netrcpasswords so that backslash literals are still interpreted as backslash literals. In case I am wrong with my assessment and that is actually the intended behavior I would instead appreciate if the need to manually escape backslashes in passwords in netrc authentication was mentioned in the Authentication with netrc section in the readme.Provide verbose output that clearly demonstrates the problem
yt-dlp -vU <your command line>)'verbose': TruetoYoutubeDLparams instead[debug] Command-line config) and insert it belowComplete Verbose Output