Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Determine how to pull PyPI packages #10

Open
nightlark opened this issue Dec 5, 2024 · 2 comments
Open

Determine how to pull PyPI packages #10

nightlark opened this issue Dec 5, 2024 · 2 comments
Assignees
@swest50

Comments

@nightlark
Copy link
Collaborator

nightlark commented Dec 5, 2024 •
edited
Loading

Determine how to pull a list of PyPI packages and information on the packages.
@nightlark
Copy link
Collaborator Author

Steven: Working on scrapping using simple API. Brute force download and scrape all data. Mapping import name to package name is not a direct mapping -- beautifulsoup4 package is imported as bs4.

Focus on mapping based on folders/files in binary wheels.

@nightlark
Copy link
Collaborator Author

12/19: Script currently pulls PyPI API information. Working on extending it to include downloading/parsing wheel file contents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@swest50 swest50

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

data_scraping-dev LLNL/dapper
2 participants
@nightlark @swest50