Skip to content

Commit

Permalink
finetuning reminder
Browse files Browse the repository at this point in the history
  • Loading branch information
bclavie committed Dec 19, 2024
1 parent 5bdcd72 commit 0404578
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion modernbert.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ In this blog post we introduced the ModernBERT models, a new state-of-the-art fa

ModernBERT demonstrates that encoder-only models can be improved by modern methods. They continue to offer very strong performance on some tasks, providing an extremely attractive size/performance ratio.

More than anything, we’re really looking forward to seeing what creative ways to use these models the community will come up with! To encourage this, we’re opening a call for demos until January 10th, 2025: the 5 best ones will get added to this post in a showcase section and win a $100 (or local currency equivalent) Amazon gift card, as well as a 6-month HuggingFace Pro subscription! (If you need a hint to get started, here’s a demo we thought about: code similarity HF space)
More than anything, we’re really looking forward to seeing what creative ways to use these models the community will come up with! To encourage this, we’re opening a call for demos until January 10th, 2025: the 5 best ones will get added to this post in a showcase section and win a $100 (or local currency equivalent) Amazon gift card, as well as a 6-month HuggingFace Pro subscription! If you need a hint to get started, here’s a demo we thought about: code similarity HF space! And remember, this is an encoder model, so all the coolest downstream applications will likely require some sort of fine-tuning (on real or perhaps decoder-model synthetic data?). Thankfully, there's lots of cool frameworks out there to support fine-tuning encoders: [🤗Transformers](https://huggingface.co/docs/transformers/en/index) itself for various tasks, including classification, [GliNER](https://github.com/urchade/GLiNER) for zero-shot Named Entity Recognition, or [Sentence-Transformers](https://sbert.net/) for retrieval and similarity tasks!

## Links

Expand Down

0 comments on commit 0404578

Please sign in to comment.