Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support 360 Zhinao-7B #3804

Closed
wants to merge 1 commit into from
Closed

Support 360 Zhinao-7B #3804

wants to merge 1 commit into from

Conversation

bingnandu
Copy link

@bingnandu bingnandu commented May 18, 2024

What does this PR do?

添加对360Zhinao的支持:
1、当使用360Zhinao模型进行chat时,stop_words里面需要加上"<eod>",这个是模型所需要的,跟tokenizer有关,否则会输出异常,表现为持续输出乱码;
2、微调时需加上"o_proj","gate_proj","down_proj","qkv_proj","up_proj",否则loss的下降会出现异常。

Before submitting

@hiyouga
Copy link
Owner

hiyouga commented May 18, 2024

添加模型请先添加我微信:hiyouga

@hiyouga hiyouga added the wontfix This will not be worked on label May 27, 2024
@hiyouga hiyouga closed this May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants