Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate gemini 2.0 flash & phi 4 #250

Open
BLACKGAMER1221 opened this issue Dec 14, 2024 · 2 comments
Open

Evaluate gemini 2.0 flash & phi 4 #250

BLACKGAMER1221 opened this issue Dec 14, 2024 · 2 comments

Comments

@BLACKGAMER1221
Copy link

No description provided.

@the-crypt-keeper
Copy link
Owner

@BLACKGAMER1221 I don't actually have access to Gemini API, If you've got keys please run:

interview-litellm.py --model gemini/gemini-2.0-flash-exp --interview senior

Then ./bulk_eval.sh to perform evaluation and open a PR with the result/eval*.json that pops out.

Phi4 still hasn't got any "official weights" released on HF 🤔 I've evaled some of the leaked GGUF and they're bad, but I dont think that's fair since the conversion may not have been done properly.

@BLACKGAMER1221
Copy link
Author

hi i did (gemini-2.0-flash-exp & gemini-2.0-flash-thinking-exp)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants