Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bandlabextractor] added new bandlab extractor for track, album, and playlist… #31206

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

dt-rush
Copy link

@dt-rush dt-rush commented Aug 28, 2022

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

Added a bandlab extractor. Resolves #31184

@dt-rush dt-rush mentioned this pull request Aug 28, 2022
5 tasks
@dt-rush
Copy link
Author

dt-rush commented Aug 29, 2022

@dirkf the first run of the tests failed on one specific test because there was a utf8 string. I changed the test for the extractor to use a song which doesn't have utf8 chars in the name. All tests should pass if run again.

@dirkf
Copy link
Contributor

dirkf commented Aug 29, 2022

You should be able to put back the Unicode test data now.

@dt-rush
Copy link
Author

dt-rush commented Aug 31, 2022

@dirkf no worries. It doesn't really make the test any more robust. It just happened i chose a song with a unicode title. Should be clear to merge from my side :)

@itzTheMeow
Copy link

is there a status on this? seems good to go

@dt-rush
Copy link
Author

dt-rush commented Sep 21, 2022

Yep it can be merged any time.

Copy link
Contributor

@dirkf dirkf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work!

I've what may seem to be a lot of suggestions, but basically they come down to two things:

  1. in new extractors, _TESTS is preferred to _TEST;
  2. unifying the extraction code for the album and playlist extractors, which are almost identical.

In the second (7-part) change, you could also change the order of the two classes so that the derived IE is later.

youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
@itzTheMeow
Copy link

@dt-rush any status on this?

@dt-rush
Copy link
Author

dt-rush commented Jan 28, 2023

Hey @dirkf ! Thanks for the suggestions! I just updated the branch. Changes:

  • The playlist and album extractors are merged into a single class since the logic is virtually identical except for using the string "albums" or "collections" in one or the other.

  • _TEST is _TESTS

  • we now use try_get() for the various metadata fields (I initially thought, the extractor should break if the format changes, but after reading the README a little more, I see the logic)

Cheers!

@itzTheMeow
Copy link

nice nice! eager to see this merged

dirkf
dirkf previously requested changes Jan 28, 2023
Copy link
Contributor

@dirkf dirkf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's check the CI tests after I fix (I hope) the Flake8 issues.

There are still some unresolved points from the previous review, plus the new ones here.

youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/extractors.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
youtube_dl/extractor/bandlab.py Outdated Show resolved Hide resolved
@dt-rush
Copy link
Author

dt-rush commented Jan 28, 2023

@dirkf fixed!

@orithecapper
Copy link

bump

@dirkf dirkf mentioned this pull request Dec 2, 2023
11 tasks
@aiur-adept
Copy link
Contributor

Hey all, this should be clear to merge. This is my new account (I'm @dt-rush). @dirkf, can you merge this please? Thank you! :)

Comment on lines 141 to 143
resource_id = self._match_id(url)
kind_regex = re.compile(self._VALID_URL)
kind = kind_regex.match(url).group('kind')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
resource_id = self._match_id(url)
kind_regex = re.compile(self._VALID_URL)
kind = kind_regex.match(url).group('kind')
resource_id, kind = self._match_valid_url(url).group('id', 'kind')

@dirkf
Copy link
Contributor

dirkf commented Jan 23, 2024

Download tests fail with 403 on API request.

In the browser: {"errorCode":403,"message":"Current user does not have permission for this action."}

@dirkf dirkf dismissed their stale review January 23, 2024 04:37

Changes made

@itzTheMeow
Copy link

itzTheMeow commented Mar 30, 2024

@aiur-adept the tracks i provided in the original example (and the ones being used for the tests) have since been privated/removed so they should probably be changed to existing tracks/albums

that'd probably be the reason for the 403

@aiur-adept
Copy link
Contributor

@itzTheMeow @dirkf it seems the helper methods from utils.py no longer exist. Were they moved somewhere else?

@aiur-adept
Copy link
Contributor

(for example, traverse_obj)

@dirkf
Copy link
Contributor

dirkf commented Jul 25, 2024

In this PR it might be necessary to pull the latest yt-dl source.

In case anyone is trying to use the extractor with yt-dlp, they moved traverse_obj() and friends to utils/traversal.py. We have a traversal.py that just redirects the imports to utils.py.

@aiur-adept
Copy link
Contributor

@dirkf thanks, i'm gonna rebase on upstream master and try to get this working

@aiur-adept
Copy link
Contributor

#32881 has been pushed with fixed test data. This PR can close now in preference of that one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BandLab
5 participants