Felix

numb3r3

·

AI & ML interests

None yet

Recent Activity

reacted to JonnaMat's post with 🚀 16 days ago

⚡ FlashHead: Fast LM Head Inference - Now a Simple vLLM Plugin flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just: ``` pip install flash-head vllm serve embedl/Qwen3-1.7B-FlashHead-W4A16 ``` ✨ The plugin activates automatically via vLLM's `vllm.general_plugins` entry point. No source patches, no custom imports. 🧩 Supported models (full collection): https://huggingface.co/Qwen Qwen3, https://huggingface.co/meta-llama Llama3, https://huggingface.co/google Gemma3, https://huggingface.co/nvidia Cosmos-Reason2 - BF16 and W4A16 variants. https://huggingface.co/collections/embedl/flashhead 📊 https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks 🔧 Benchmark it yourself: ``` vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 # Baseline comparison FLASHHEAD_ENABLED=0 vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1 ``` FlashHead shines at low batch sizes; the typical real-time / on-device use case. 🚀

View all activity

Organizations

upvoted a collection about 2 years ago

Mini Pretrain Datasets

9 items • Updated Jul 9, 2024 • 12

upvoted a paper about 2 years ago

Jina CLIP: Your CLIP Model Is Also Your Text Retriever

Paper • 2405.20204 • Published May 30, 2024 • 37

upvoted a paper over 2 years ago

Gecko: Versatile Text Embeddings Distilled from Large Language Models

Paper • 2403.20327 • Published Mar 29, 2024 • 47