Skip to content

Conversation

@abukhoy
Copy link
Contributor

@abukhoy abukhoy commented Dec 9, 2025

The following models have been successfully tested without using model cache:

  • TinyLlama/TinyLlama-1.1B-Chat-v1.0
  • gpt2
  • Salesforce/codegen-350M-mono
  • microsoft/Phi-3-mini-4k-instruct
  • tiiuae/falcon-7b
  • Qwen/Qwen2-0.5B
  • Qwen/Qwen3-0.6B
  • bigcode/starcoder2-3b
  • Felladrin/Minueza-32M-Base
  • wtang06/mpt-125m-c4
  • hakurei/gpt-j-random-tinier
  • meta-llama/Llama-3.2-1B
  • unsloth/gemma-2b
  • unsloth/gemma-2-2b
  • TheBloke/TinyLlama-1.1B-Chat-v0.3-AWQ (AWQ model)
  • neuralmagic/Llama-3.2-3B-Instruct-FP8 (float quantized compressed-tensor per tensor for weights and activations)
  • neuralmagic/Qwen2-0.5B-Instruct-FP8 (FP8 quant method, static, LM head ignored)
  • ibm-granite/granite-3.1-2b-instruct
  • ibm-granite/granite-guardian-3.1-2b
  • allenai/OLMo-2-0425-1B

Signed-off-by: Abukhoyer Shaik <[email protected]>
@abukhoy abukhoy marked this pull request as draft December 15, 2025 06:49
@quic-rishinr quic-rishinr marked this pull request as ready for review December 16, 2025 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant