Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

finetune_cosmopedia.sh如何训练出来8B模型 #27

Open
RuipingWang1986 opened this issue May 20, 2024 · 1 comment
Open

finetune_cosmopedia.sh如何训练出来8B模型 #27

RuipingWang1986 opened this issue May 20, 2024 · 1 comment

Comments

@RuipingWang1986
Copy link

您好，根据您的指导，我目前已经可以跑通finetune_cosmopedia脚本的训练，但是我发现生成的模型和之前使用的llama2-7B模型规模基本是相同的，并没有增大；另外，按照您论文里提到的，应该是可以自己设计添加多少层，但我在训练脚本中也没有发现。
@season1blue
Copy link

我也是同样的问题，请问扩展参数的那部分代码是还没有放出来吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@season1blue @RuipingWang1986