Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YouTube] Fix nsig challenge for player 3bb1f723 #32987

Merged
merged 14 commits into from
Dec 16, 2024
Merged

[YouTube] Fix nsig challenge for player 3bb1f723 #32987

merged 14 commits into from
Dec 16, 2024
+636 −202

Conversation

dirkf
Copy link
Contributor

@dirkf dirkf commented Dec 7, 2024

Boilerplate: mixed code, bug fix

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense, except for code from yt-dlp for which either this or the below has al;ready been asserted
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

YT player 3bb1f723 introduced a slightly differently nonsensical way of loading the nsig challenge JS, and the challenge required the previously unimplemented typeof operator.

This PR adapts the nsig extraction from yt-dlp/yt-dlp#11752, yt-dlp/yt-dlp# and #11750 and yt-dlp/yt-dlp#11751 (thx @bashonly) and implements the typeof operator for jsinterp.

Fixes #32986.

@dirkf dirkf mentioned this pull request Dec 7, 2024
Closed
10 tasks
@nicolaasjan nicolaasjan mentioned this pull request Dec 10, 2024
Open
6 tasks
@tansy tansy mentioned this pull request Dec 10, 2024
Closed
5 tasks
@dirkf
Copy link
Contributor Author

dirkf commented Dec 12, 2024

#32986 (comment)

dirkf added 5 commits December 12, 2024 11:42
@dirkf
[InfoExtractor] Use kwarg maxsplit for re.split
af7c0ff 
* May become kw-only in future Pythons
@dirkf
[jsinterp] Implement typeof operator
7e93935
@dirkf
[jsinterp] Fix and improve loose and strict equality operations
527c137 
* reimplement loose equality according to MDN (eg, 1 == "1")
* improve strict equality (eg, "abc" === "abc" but 'abc' is not 'abc')
* add tests for above
@dirkf
[jsinterp] Support multiple indexing (eg a[1][2])
7d3045c 
* extend single indexing with improved RE (should probably use/have used _separate_at_paren())
* fix some cases that should have given undefined, not throwing
* standardise RE group names
* support length of objects, like {1: 2, 3: 4, length: 42}
@dirkf
[jsinterp] Fix and improve arithmetic operations
9993e8c 
* addition becomes concat with a string operand
* improve handling of edgier cases
* arithmetic in float like JS (more places need cast to int?)
* increase test coverage
dirkf
dirkf commented Dec 12, 2024
View reviewed changes
test/test_jsinterp.py Outdated Show resolved Hide resolved
dirkf
dirkf commented Dec 12, 2024
View reviewed changes
test/test_jsinterp.py Outdated Show resolved Hide resolved
@nicolaasjan
Copy link

I just built youtube-dl with your changes proposed here, but now I get player 5b77d519. 🤔

And after that the usual 403 errors...

youtube-dl -v https://www.youtube.com/watch?v=lLSkbZ3-EOs
[debug] System config: []
[debug] User config: ['--rm-cache-dir', '-i', '-o', '/dev/shm/test-ytd/%(title)s.%(ext)s', '-f', 'bestvideo[height<=1080][ext=mp4][vcodec^=avc]+bestaudio[ext=m4a]/best[ext=mp4]/best', '--no-mtime', '--embed-thumbnail', '--force-ipv4', '--cookies', 'cookies.txt', '--user-agent', 'Mozilla/5.0 (Android 14; Mobile; rv:128.0) Gecko/128.0 Firefox/128.0']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=lLSkbZ3-EOs']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2024.12.12
[debug] Lazy loading extractors enabled
[debug] Single file build
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-5.15.0-126-generic-x86_64-with-glibc2.35 - OpenSSL 3.0.2 15 Mar 2022 - glibc 2.35
[debug] exe versions: ffmpeg N-118034-gd21134313f-20241209, ffprobe N-118034-gd21134313f-20241209, phantomjs 140310570063808, rtmpdump 2.4
[debug] Proxy map: {}
Removing cache dir /home/nico/.cache/youtube-dl ...
[youtube] lLSkbZ3-EOs: Downloading webpage
[youtube] lLSkbZ3-EOs: Downloading player 5b77d519
WARNING: [youtube] Falling back to generic n function search
[debug] [youtube] Decrypted nsig RDdH58oItEEPNExSu => B0ocRZ-v-RwcVQ
[debug] [youtube] Decrypted nsig 67PETO8bs4uba20cp => aOYdOj4hBjxeXg
[youtube] lLSkbZ3-EOs: Downloading API JSON
[youtube] lLSkbZ3-EOs: Downloading thumbnail ...
[youtube] lLSkbZ3-EOs: Writing thumbnail to: /dev/shm/test-ytd/全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】.jpg
[debug] Invoking downloader on 'https://rr2---sn-5hne6n6l.googlevideo.com/videoplayback?expire=1734050221&ei=TS1bZ7bAE87R6dsPivKwMA&ip=<REDACTED>&id=o-AMLmBH3JXNnWiOkMny-2Cj20KZJ7ol9ZWoNpW2rdU3Us&itag=299&aitags=133%2C134%2C135%2C136%2C160%2C242%2C243%2C244%2C247%2C278%2C298%2C299%2C302%2C303%2C394%2C395%2C396%2C397%2C398%2C399%2C597%2C598&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1734028621%2C&mh=jQ&mm=31%2C26&mn=sn-5hne6n6l%2Csn-4g5edndk&ms=au%2Conr&mv=m&mvi=2&pl=16&rms=au%2Cau&initcwndbps=4077500&bui=AfMhrI8C5fLPrENxUz-IvGhKbYreRWWvQwvyNtbKkVAe-1PoM5kmhp-8n95LNUu45UIQ4PfpNec91Kgv&spc=x-caUHHaqNcb3S40N4CQIHCO9sr-Sq9brcvfCu9vyE1ADPlL68Q9YHQAWT-xtrNhfA&vprv=1&svpuc=1&mime=video%2Fmp4&ns=bOiMdqRqrLGXuoLD81UwV1oQ&rqh=1&gir=yes&clen=136355177&dur=850.966&lmt=1726249510072208&mt=1734028431&fvip=5&keepalive=yes&fexp=51326932%2C51331020%2C51335594%2C51347747&c=MWEB&sefc=1&txp=5535434&n=aOYdOj4hBjxeXg&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgdZPEldBAdUemqrOGiipBMFEQKrIJ-8WeTNmppcdTx3MCIGjAIWymeYlXiGxpPkd6qhfxBnmEY1eVyGOjzYQ5LA6i&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRgIhAJqc3DyFqe14lmQlYq1azKLtbACxw7ZvX4-r6F414yVrAiEArB8sd0WZ4H3jq8xdJh_fq7cmfpiMdXlkkk2H4hzqfK8%3D'
[dashsegments] Total fragments: 14
[download] Destination: /dev/shm/test-ytd/全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】.f299.mp4
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
ERROR: giving up after 10 fragment retries
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/bin/youtube-dl.orig/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/__init__.py", line 477, in main
    _real_main(argv)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/__init__.py", line 467, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2253, in download
    res = self.extract_info(
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 1016, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 1860, in process_video_result
    self.process_info(new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 138, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2152, in process_info
    partial_success = dl(fname, new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2088, in dl
    return fd.download(name, new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/dash.py", line 78, in real_download
    self.report_error('giving up after %s fragment retries' % count)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/common.py", line 175, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 678, in report_error
    self.trouble(*args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 630, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

[debug] Invoking downloader on 'https://rr2---sn-5hne6n6l.googlevideo.com/videoplayback?expire=1734050221&ei=TS1bZ7bAE87R6dsPivKwMA&ip=<REDACTED>&id=o-AMLmBH3JXNnWiOkMny-2Cj20KZJ7ol9ZWoNpW2rdU3Us&itag=140&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&met=1734028621%2C&mh=jQ&mm=31%2C26&mn=sn-5hne6n6l%2Csn-4g5edndk&ms=au%2Conr&mv=m&mvi=2&pl=16&rms=au%2Cau&initcwndbps=4077500&bui=AfMhrI8C5fLPrENxUz-IvGhKbYreRWWvQwvyNtbKkVAe-1PoM5kmhp-8n95LNUu45UIQ4PfpNec91Kgv&spc=x-caUHHaqNcb3S40N4CQIHCO9sr-Sq9brcvfCu9vyE1ADPlL68Q9YHQAWT-xtrNhfA&vprv=1&svpuc=1&xtags=drc%3D1&mime=audio%2Fmp4&ns=bOiMdqRqrLGXuoLD81UwV1oQ&rqh=1&gir=yes&clen=13774638&dur=851.080&lmt=1726243429043316&mt=1734028431&fvip=5&keepalive=yes&fexp=51326932%2C51331020%2C51335594%2C51347747&c=MWEB&sefc=1&txp=5532434&n=aOYdOj4hBjxeXg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cxtags%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgc2RiT3rysZ1HpPnJwYuT0kAmzqNWCPNUitVdYm4x84wCIE-1OmWYJvcbY6pduFmDXdChFBmlJQ_B5mqPQJio92aE&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Crms%2Cinitcwndbps&lsig=AGluJ3MwRgIhAJqc3DyFqe14lmQlYq1azKLtbACxw7ZvX4-r6F414yVrAiEArB8sd0WZ4H3jq8xdJh_fq7cmfpiMdXlkkk2H4hzqfK8%3D'
[dashsegments] Total fragments: 2
[download] Destination: /dev/shm/test-ytd/全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】.m4a.f140-drc.m4a
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 403: Forbidden. Retrying fragment 1 (attempt 10 of 10)...
ERROR: giving up after 10 fragment retries
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/bin/youtube-dl.orig/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/__init__.py", line 477, in main
    _real_main(argv)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/__init__.py", line 467, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2253, in download
    res = self.extract_info(
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 868, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 875, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 982, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 1016, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 1860, in process_video_result
    self.process_info(new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 138, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2152, in process_info
    partial_success = dl(fname, new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 2088, in dl
    return fd.download(name, new_info)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/dash.py", line 78, in real_download
    self.report_error('giving up after %s fragment retries' % count)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/downloader/common.py", line 175, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 678, in report_error
    self.trouble(*args, **kwargs)
  File "/usr/local/bin/youtube-dl.orig/youtube_dl/YoutubeDL.py", line 630, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

@dirkf
Copy link
Contributor Author

dirkf commented Dec 13, 2024

Thanks for the test. I get the same result but format 18 is still working. It's just more #32905 apparently.

dirkf added 7 commits December 13, 2024 03:15
@dirkf
[jsinterp] Fix and improve split/join
4bf85ca 
* improve split/join edge cases
* correctly implement regex split (not like re.split)
@dirkf
[jsinterp] Fix and improve "methods"
7ab85d8 
* push, unshift return new length
* impove edge cases for push/pop, shift/unshift, forEach, indexOf, charCodeAt
* increase test coverage
@dirkf
[jsinterp] Strip /* comments */ when parsing
067ad74 
* NB: _separate() is looking creaky
@dirkf
[jsinterp] Add return hook for player 3bb1f723
2968201 
* set var `_ytdl_do_not_return` to a specific value in the scope of a function
* if an expression to be returned has that value, `return` becomes `void`
@dirkf
[YouTube] Handle player 3bb1f723
a93fff3 
* fix signature code extraction
* raise if n function returns input value
* add new tests from yt-dlp

Co-authored-by: bashonly
@dirkf
[YouTube] Pass nsig value as return hook, fixes player 3bb1f723
9bc8fcf
@dirkf
[YouTube] Simplify pattern for nsig function name extraction
3d64984
@bashonly
Copy link
Contributor

see also:

@dirkf
Copy link
Contributor Author

dirkf commented Dec 13, 2024

Thanks. I decided that all these a-zA-Z0-9_ were actually \w, even if that's only strictly true if re.A. But should they be jsinterp._NAME_RE (could be a useful export), or just \w$?

When yt-dl maintainers began to support Python 3, they failed to maintain the matching semantics of _search_regex(), which behaved as if re.A (existed and was set) under Python 2 unless re.U or (?u)... was used. So

  • here pattern matching works differently according to Python, which is bad, but normally makes no difference, while
  • in yt-dlp matching follows the built-in default re.U, which normally is not what was wanted even if it makes no difference.

Basically, the existing state means that Perly shorthands like \w, \b are not reliable, possibly why \w wasn't used instead of a-zA-Z0-9.

Surely it's not too late to make this change?

+        # first test can be omitted in yt-dlp
+        if sys.version_info[0] >= 3 and not (re.U & flags):
+            flags |= re.A  # default in Py2 but not defined

Then it's clear that an extractor must enable Unicode matching to find ℕ alongside n and N. Maybe _html_search_regex() should default to re.U, but we'd have to invent a compat_re_A = 256 or an extra kwarg that could be passed to disable that default across Pythons.

dirkf added 2 commits December 13, 2024 12:16
@dirkf
[YouTube] Fix signature function extraction for 2f1832d2
82b5256 
* `_` was omitted from patterns
* thx yt-dlp/yt-dlp#11801

Co-authored-by: bashonly
@dirkf
[YouTube] Always extract using MWEB API client
12bc377 
* temporary fix-up for 403 on download
* MWEB parameters from yt-dlp 2024-12-06
@dirkf
Copy link
Contributor Author

dirkf commented Dec 14, 2024

I've added a small interim (I hope) hack to address #32905 as well, since fixing #32986 on its own seems rather pointless.

Example log 
$ python -m youtube_dl -v -f worstvideo[ext=mp4]+worstaudio[ext=m4a] 'https://www.youtube.com/watch?v=lLSkbZ3-EOs' 
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-f', u'worstvideo[ext=mp4]+worstaudio[ext=m4a]', u'https://www.youtube.com/watch?v=lLSkbZ3-EOs']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 12bc3775d
[debug] Python 2.7.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.1.1w  11 Sep 2023 - glibc 2.15
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[youtube] lLSkbZ3-EOs: Downloading webpage
[youtube] lLSkbZ3-EOs: Downloading API JSON
[debug] [youtube] Decrypted nsig -fxLMP_-ns82IlJkdjG => NQpb9_72kLmiQA
[debug] [youtube] Decrypted nsig DWewDfYkwyVUl8ovgqD => SpP5NxHHfYLZ8Q
[debug] Invoking downloader on u'https://rr5---sn-cu-aigss.googlevideo.com/videoplayback?rms=au%2Cau&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&fexp=51326932%2C51331020%2C51335594%2C51347747&ei=bW1dZ5iYGNrZxN8P5ZTDuAE&ip=46.208.6.14&clen=3197241&id=o-AFmOOTZ42wa--h7SmdGX2b8yPXMapwWE4aRgyZlpLuDu&txp=5535434&svpuc=1&aitags=133%2C134%2C135%2C136%2C160%2C242%2C243%2C244%2C247%2C278%2C298%2C299%2C302%2C303%2C394%2C395%2C396%2C397%2C398%2C399%2C597%2C598&gir=yes&xpc=EgVo2aDSNQ%3D%3D&requiressl=yes&keepalive=yes&source=youtube&mv=m&sig=AJfQdSswRQIgdKWBAM9zH-cXsgOJh1gxnuGhZGpte5DftWQaZPQeH5QCIQDlGAl8kOuDDzsYxT_dntcETgh3GLbCU63gFhGJcpQnxQ%3D%3D&pcm2cms=yes&dur=850.966&ns=VTKLsQeDTDckVlJiCDE_74cQ&initcwndbps=3315000&vprv=1&lsig=AGluJ3MwRAIgZIpNfpXpRlz47poUqgoJMF-7PSbkfL2bDtOQldsSEBACIFS7xuYtFwVCxUrHnIF2Cg_ACohIOHjKlFvqf0heloTw&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Crms%2Cinitcwndbps&lmt=1726249325105236&c=MWEB&met=1734176109%2C&bui=AfMhrI9HqZf9IS1q7Ukj4P0dcY1y2s1daGkxr0HDqKvXV7cb42UJL1OsNzda7_skPolH_DvUY5ZYfgwy&mime=video%2Fmp4&fvip=4&rqh=1&itag=160&sefc=1&mm=31%2C29&mn=sn-cu-aigss%2Csn-cu-c9id&mh=jQ&n=SpP5NxHHfYLZ8Q&mt=1734175766&expire=1734197709&pl=25&ms=au%2Crdu&mvi=5'
[dashsegments] Total fragments: 1
[download] Destination: 全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.f160.mp4
[download] 100% of 3.05MiB in 00:02
[debug] Invoking downloader on u'https://rr5---sn-cu-aigss.googlevideo.com/videoplayback?rms=au%2Cau&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cbui%2Cvprv%2Csvpuc%2Cmime%2Cns%2Crqh%2Cgir%2Cclen%2Cdur%2Clmt&fexp=51326932%2C51331020%2C51335594%2C51347747&ei=bW1dZ5iYGNrZxN8P5ZTDuAE&ip=46.208.6.14&clen=3276152&id=o-AFmOOTZ42wa--h7SmdGX2b8yPXMapwWE4aRgyZlpLuDu&txp=5532434&svpuc=1&gir=yes&xpc=EgVo2aDSNQ%3D%3D&requiressl=yes&keepalive=yes&source=youtube&mv=m&sig=AJfQdSswRQIgaZ8TAuMkwER4Qfyjttf-jIGl0djaN3Qs2ELiaXSzQkACIQCO7iTrKjPGeAfmU2eTDpDnYN1Q7H30bJ0cQqxF5ixJTg%3D%3D&pcm2cms=yes&dur=851.150&ns=VTKLsQeDTDckVlJiCDE_74cQ&initcwndbps=3315000&vprv=1&lsig=AGluJ3MwRAIgZIpNfpXpRlz47poUqgoJMF-7PSbkfL2bDtOQldsSEBACIFS7xuYtFwVCxUrHnIF2Cg_ACohIOHjKlFvqf0heloTw&lsparams=met%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Crms%2Cinitcwndbps&lmt=1726243479804871&c=MWEB&met=1734176109%2C&bui=AfMhrI9HqZf9IS1q7Ukj4P0dcY1y2s1daGkxr0HDqKvXV7cb42UJL1OsNzda7_skPolH_DvUY5ZYfgwy&mime=audio%2Fmp4&fvip=4&rqh=1&itag=599&sefc=1&mm=31%2C29&mn=sn-cu-aigss%2Csn-cu-c9id&mh=jQ&n=SpP5NxHHfYLZ8Q&mt=1734175766&expire=1734197709&pl=25&ms=au%2Crdu&mvi=5'
[dashsegments] Total fragments: 1
[download] Destination: 全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.m4a.f599.m4a
[download] 100% of 3.12MiB in 00:03
[ffmpeg] Merging formats into "全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.f160.mp4' -i 'file:全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.m4a.f599.m4a' -c copy -map '0:v:0' -map '1:a:0' 'file:全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.temp.mp4'
Deleting original file 全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.f160.mp4 (pass -k to keep)
Deleting original file 全球金融大动荡，日本加息背刺美国，中国躺赢？【汤山老王】-lLSkbZ3-EOs.m4a.f599.m4a (pass -k to keep)
$

mk-pmb
mk-pmb reviewed Dec 15, 2024
View reviewed changes
youtube_dl/extractor/youtube.py
@@ -1914,9 +1943,50 @@ def _real_extract(self, url):
player_response = self._extract_yt_initial_variable(
webpage, self._YT_INITIAL_PLAYER_RESPONSE_RE,
video_id, 'initial player response')
if not player_response:
if False and not player_response:
Copy link
Contributor

@mk-pmb mk-pmb Dec 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "False and" looks like a temporary stopgap. It could thus use an inline comment explaining where to read about what problem it mitigates and what work is needed to fix it properly. Ideally the explanation would be a link to an issue thread.
Edit: Now seeing the "True or" a few lines ahead, it may be better to just set player_response = False and mark that as the stopgap, rather than having two bogus conditions.
Anyway, thanks for your great work! 👍

@dirkf
Copy link
Contributor Author

dirkf commented Dec 15, 2024

Apparently the kewl kidz are just putting comments like this in the commit message and using git blame .... But if this turns out to be less temporary I'll clean it up.

@dirkf dirkf merged commit d55d1f4 into ytdl-org:master Dec 16, 2024
14 checks passed
@mk-pmb
Copy link
Contributor

mk-pmb commented Dec 17, 2024 •
edited
Loading

I trust your choice in this case.
That said, I feel a need to explain to later readers of this thread (those that didn't read "kewl kidz" as tongue-in-cheek) why relying on git blame should be a rare exception:
I was one of those myself, and it usually works for a while, until several commits for other changes in that line pile up and then code gets moved to another place. Then tracing back the git blame becomes true detective's work. And a few years later you're no longer sure what part exactly of the commit the bullet in the commit message referred to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mk-pmb mk-pmb mk-pmb left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@dirkf @nicolaasjan @bashonly @mk-pmb