Skip to content

Pull requests: huggingface/nanotron

New pull request New
31 Open 141 Closed
31 Open 141 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fp8
#266 opened Dec 18, 2024 by xrsrke Loading…
[Feature] Support resume ZeRO1 in a new data parallelism size
#263 opened Dec 10, 2024 by xrsrke Loading…
Fix wrong initialization of lr scheduler
#256 opened Nov 29, 2024 by kylematoba Loading…
2
[NEW] Llama3.2 weight converters 🦙
#255 opened Nov 28, 2024 by TJ-Solergibert Loading…
6 tasks
4
Add shuffling in Nanotron for subsequent epochs when data is repeated
#247 opened Nov 24, 2024 by Lauler Loading…
3
Fix initial_lr when resuming training
#243 opened Nov 17, 2024 by Lauler Loading…
6
Load random states from checkpoint
#238 opened Nov 2, 2024 by gritukan Loading…
1
lighteval support after checkpoint, UX refactor
#222 opened Aug 24, 2024 by eliebak Loading…
@3outeille
41
Refactor pre tokenization tool
#219 opened Aug 21, 2024 by eliebak Loading…
@3outeille
refacto generate + use simpler rotary for inference
#202 opened Jun 25, 2024 by 3outeille Loading…
2
Created interconnect benchmark before the training
#200 opened Jun 22, 2024 by RamenBuddha Loading…
Fix: Update wrong typing on the function get_local_ranks
#194 opened Jun 10, 2024 by morgangiraud Loading…
1
Move MoE Implementation into src/, add Load Balancing Losses
#192 opened Jun 6, 2024 by haeggee Loading…
1 task done
Migrate internal brrr to nanotron
#186 opened May 27, 2024 by NouamaneTazi • Draft
[Feature] Monitor model states during training
#183 opened May 25, 2024 by xrsrke Loading…
Fix overflow in nanosets with big datasets
#182 opened May 23, 2024 by jquesnelle Loading…
1
Ring attention
#181 opened May 23, 2024 by zzhhjjj Loading…
4
FEAT: Adding 1.58bit LLMs training architecture in nanotron
#180 opened May 23, 2024 by MekkCyber • Draft
8
Llama3 conversion scripts 🦙
#174 opened May 20, 2024 by TJ-Solergibert Loading…
9 tasks done
13
[Feature] Mixture of Depths
#171 opened May 15, 2024 by xrsrke • Draft
[Feature] Infini Attention
#169 opened May 14, 2024 by xrsrke Loading…
Core attention
#168 opened May 13, 2024 by zzhhjjj Loading…
llama tests
#157 opened Apr 30, 2024 by zzhhjjj Loading…
21
Fix TestContext warning
#156 opened Apr 29, 2024 by AleHD Loading…
Checkpoint 1.3 backwards compatibility
#152 opened Apr 25, 2024 by AleHD Loading…
3 tasks done
2
ProTip! Type g i on any issue or pull request to go back to the issue listing page.