Code

[Feature] Mixture of Depths#171

Code

Draft

xrsrke wants to merge 2 commits intomainhuggingface/nanotron:mainfrom xrsrke/mixture-of-depthhuggingface/nanotron:xrsrke/mixture-of-depthCopy head branch name to clipboard

+284-2

Commits

Commits on May 15, 2024

• add MoD block

  xrsrke

  committed
  3c2e92a

• add support llama tp

  xrsrke

  committed
  fc37517