YouTube 403 forbidden error on audio urls #32882
Comments
|
I dug a little deeper and the issue is that the url provided by youtube-dl is not accessible and gives a 403 error. This seems to be very random. This seems like a new development since the commit I linked above. Could youtube-dl validate that the url it gives is accessible before returning it? Possibly with some retry logic? I guess it may be too complex to validate all the format URLs. |
|
Could you post the original youtube url you're fetching so we could see what youtube-dl is returning/do some debugger-work? |
|
@aiur-adept sure, it can occur on seemingly any video. Here's two examples: It seems like there is some kind of rate limiting happening, because if I try again in a little bit it will work. I guess these might be related: |
ReenigneArcher
changed the title
|
Ah yeah, I've just seen the behaviour. It failed with 403 then after about 10 minutes it worked. Weird that they're sending a 403 when it's likely a rate limit. I'm not sure how to solve this as having the process wait for 10 minutes on a 403 isn't really a good solution. |
|
If you are post-processing the download URLs, you could try using the equivalent back-up URL (specified in its query parameters) on unexpected 403, or 4xx generally. This has so far been deemed too hard in yt-dl[p], as it would need a custom downloader; also AFAIK no-one has conclusively shown that the backup URL work better than the original one. The algorithm for generating the backup URL is in an old issue (also at yt-dlp) which I expect I can find if you can't. |
|
I tried a small retry loop in my code as well, and it might have helped... still testing though. I also tried not using a cookie file after seeing this: yt-dlp/yt-dlp#10046 ... that also may have helped... still testing that too.
From what I can see this is kind of standard for google products. @dirkf, I am not using youtube-dl to do the downloading, and I'm handling that on my own. I did see this yt-dlp/yt-dlp#4100 (comment) I'm not sure I 100% understand though. |
|
@dirkf @ReenigneArcher should we close this issue? |
|
What's the algorithm for generating the backup url? |
|
Probably this PR is the best source. |
|
No luck: LizardByte/Themerr-plex@81df3d2 |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as resolved.
This comment was marked as resolved.
|
I found that the issue was at least partially because of m4a chosen as the best audio format. If I force it with 'bestaudio[ext=webm]' then previously 403 error'd URLs download with no issues. Still not a good solution because there's a few URLs with no webm format. |
|
@JstaClarkey thank you for that information. I'm also using m4a. I guess I will need to incorporate ffmpeg to solve this. |
|
This very much looks like the poToken experiment. |
|
Yes, I've made a WIP that tests for The multiple client thing is conceptually straightforward (ie, instead of a single result to be traversed there is a list of results) but also brings in some related catch-up issues like extractor args. |
|
@dirkf I'll be happy to test when it's ready. My CI tests about 30 videos, so it's pretty easy to hit this error. |
|
My initial testing gives somewhat strange results. Getting the title from each item in a playlist brings up the poToken warning quite regularly. Downloading item 1 of the list works fine regardless of whether the warning appears. Similar results with the two videos above. |
|
Do you have a live branch for this? |
|
Continued in #32905. |
Checklist
Question
Did something change in the YouTube audio formats, or are they returned differently than before? After merging e1b3fa2 into my codebase, I am getting (somewhat) random failures, but still very repeatable. I wasn't getting these errors with the previous version of ytdl, but there was a new error (#32842 (comment)) so had to update.
Plex is giving me 406 (unacceptable) errors when I try to upload the audio.
Here is my code for getting the audio url: https://github.com/LizardByte/Themerr-plex/blob/c948caa73991e05cebe8b24b88bce5ccb6bede4c/Contents/Code/youtube_dl_helper.py#L133-L161
I tested locally, and I'm still getting audio urls, but could be in formats that aren't supported by Plex. Last update for Plex was July 9th, so nothing on their end changed in the last few days.
Any ideas?
The text was updated successfully, but these errors were encountered: