Edit Models filters

Model Tree

Apps

Docker Model Runner

Unsloth Desktop

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

613

Base only

Active filters: llm-compressor

kaitchup/LFM2.5-1.2B-Thinking-autoround-W4A16

0.7B • Updated Jan 22 • 8

kaitchup/GLM-4.7-Flash-FP8-Dynamic

30B • Updated Jan 23 • 14

RedHatAI/Phi-4-reasoning-FP8-dynamic

Text Generation • 15B • Updated Apr 28 • 58 • 1

dtometzki/Qwen3-30B-A3B-awq-sym

Text Generation • 5B • Updated Jan 28 • 180

JongYeop/Llama-3.1-8B-Instruct-NVFP4-W4A4

5B • Updated Jan 29 • 26

JongYeop/Llama-3.1-70B-Instruct-NVFP4-W4A4

41B • Updated Feb 2 • 3

JongYeop/Llama-3.1-8B-Instruct-INT8-W8A8

8B • Updated Feb 3 • 2

JongYeop/Llama-3.1-8B-Instruct-INT8-W8A8-Dynamic-Per-Token

8B • Updated Feb 3 • 3

JongYeop/Llama-3.1-8B-Instruct-FP8-W8A8-Dynamic-Per-Token

8B • Updated Feb 3 • 3

bullpoint/Qwen3-Coder-Next-AWQ-4bit

Text Generation • 14B • Updated Feb 3 • 37.6k • 30

rtj1/Qwen2.5-0.5B-AWQ-FP8-Dynamic

Text Generation • 0.6B • Updated Feb 10 • 6

rtj1/Qwen2.5-0.5B-AWQ-FP8-Block

Text Generation • 0.6B • Updated Feb 10 • 8

ludovicoYIN/MiniMax-M2-BF16

Text Generation • 229B • Updated Feb 8 • 16 • 1

vistralis/Qwen3-4B-NVFP4

Text Generation • 3B • Updated Feb 7 • 19 • 1

vistralis/Qwen3-4B-FP8

Text Generation • 4B • Updated Feb 7 • 9

vistralis/Qwen3-4B-INT8

Text Generation • 4B • Updated Feb 7 • 14

vistralis/Qwen3-8B-INT8

Text Generation • 8B • Updated Feb 7 • 11

a-ivanovitch/Qwen3-4B-Instruct-2507-AWQ-W4A16

Text Generation • 4B • Updated Feb 12 • 404

ludovicoYIN/MiniMax-M2-BF16-W4A16

Text Generation • 34B • Updated Feb 13 • 22

mratsim/MiniMax-M2.5-BF16-INT4-AWQ

Text Generation • 39B • Updated Feb 17 • 161 • 43

ekurtic/granite-4.0-h-small-FP8-block

Text Generation • 32B • Updated Feb 14 • 13

ekurtic/granite-4.0-h-small-FP8-dynamic

Text Generation • 32B • Updated Feb 14 • 6

kaitchup/Nanbeige4.1-3B-MXFP4

2B • Updated Feb 16 • 4 • 1

kaitchup/Nanbeige4.1-3B-NVFP4

3B • Updated Feb 17 • 39

kaitchup/Nanbeige4.1-3B-FP8-Dynamic

4B • Updated Feb 17 • 6

kaitchup/Nanbeige4.1-3B-AWQ-W4A16-ASYM

4B • Updated Feb 17 • 10 • 1

mratsim/MiniMax-M2.5-FP8-INT4-AWQ

Text Generation • 39B • Updated Feb 17 • 1.25k • 21

RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic

Image-Text-to-Text • 404B • Updated Jul 23 • 1.33k • 6

Sehyo/Qwen3.5-397B-A17B-NVFP4

Updated Mar 2 • 635 • 7

prithivMLmods/ZwZ-8B-FP8

Image-Text-to-Text • 9B • Updated Feb 19 • 7 • 1