Read model weights #91
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Separate Safetensor from the weights * Rename test tensors to include type * Rename ModelWeights to Weights * Throw error for unsupported data types * Remove model weights from LanguageModel.Configurations
|
Weights are now accessible independently of configurations for flexibility and without MLModel or conversion. |
|
That's very cool @shavit! May I ask what use case you have in mind for this? Edit: Oh, I see it's for your embeddings work. Any highlights on the general architecture you're considering? |
|
Yes
The plan is to use models directly from local files or the hub, and maybe support more types like mlx/numpy. |
pcuenca
approved these changes
Dec 12, 2024
|
|
||
| func testLoadWeightsFromFileURL() async throws { | ||
| let repo = "google/bert_uncased_L-2_H-128_A-2" | ||
| let modelDir = try await hubApi.snapshot(from: repo) |
There was a problem hiding this comment.
I'd suggest you use matching globs to only download the two files we need and avoid additional downloads. There's a version that supports multiple patterns.
Enable safe access to keys.
pcuenca
approved these changes
Dec 14, 2024
|
No, that's it. |
|
Thanks a lot @shavit! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Read model weights from safetensors.
Related #86