f16 path changed
Browse files
{Qwen3-1.7B-Q5_K_S → Qwen3-1.7B-f16-Q5_K_S}/README.md
RENAMED
|
@@ -17,7 +17,7 @@ base_model: Qwen/Qwen3-1.7B
|
|
| 17 |
author: geoffmunn
|
| 18 |
---
|
| 19 |
|
| 20 |
-
# Qwen3-1.7B:Q5_K_S
|
| 21 |
|
| 22 |
Quantized version of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) at **Q5_K_S** level, derived from **f16** base weights.
|
| 23 |
|
|
@@ -123,7 +123,7 @@ Stop sequences: `<|im_end|>`, `<|im_start|>`
|
|
| 123 |
Importing directly into Ollama should work, but you might encounter this error: `Error: invalid character '<' looking for beginning of value`.
|
| 124 |
In this case try these steps:
|
| 125 |
|
| 126 |
-
1. `wget https://huggingface.co/geoffmunn/Qwen3-1.7B/resolve/main/Qwen3-1.7B-f16%3AQ5_K_S.gguf`
|
| 127 |
2. `nano Modelfile` and enter these details:
|
| 128 |
```text
|
| 129 |
FROM ./Qwen3-1.7B-f16:Q5_K_S.gguf
|
|
@@ -162,7 +162,7 @@ Here’s how you can query this model via API using `curl` and `jq`. Replace the
|
|
| 162 |
|
| 163 |
```bash
|
| 164 |
curl http://localhost:11434/api/generate -s -N -d '{
|
| 165 |
-
"model": "hf.co/geoffmunn/Qwen3-1.7B:Q5_K_S",
|
| 166 |
"prompt": "Respond exactly as follows: Write a short limerick about a robot who loves gardening.",
|
| 167 |
"temperature": 0.8,
|
| 168 |
"top_p": 0.95,
|
|
|
|
| 17 |
author: geoffmunn
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# Qwen3-1.7B-f16:Q5_K_S
|
| 21 |
|
| 22 |
Quantized version of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) at **Q5_K_S** level, derived from **f16** base weights.
|
| 23 |
|
|
|
|
| 123 |
Importing directly into Ollama should work, but you might encounter this error: `Error: invalid character '<' looking for beginning of value`.
|
| 124 |
In this case try these steps:
|
| 125 |
|
| 126 |
+
1. `wget https://huggingface.co/geoffmunn/Qwen3-1.7B-f16/resolve/main/Qwen3-1.7B-f16%3AQ5_K_S.gguf`
|
| 127 |
2. `nano Modelfile` and enter these details:
|
| 128 |
```text
|
| 129 |
FROM ./Qwen3-1.7B-f16:Q5_K_S.gguf
|
|
|
|
| 162 |
|
| 163 |
```bash
|
| 164 |
curl http://localhost:11434/api/generate -s -N -d '{
|
| 165 |
+
"model": "hf.co/geoffmunn/Qwen3-1.7B-f16:Q5_K_S",
|
| 166 |
"prompt": "Respond exactly as follows: Write a short limerick about a robot who loves gardening.",
|
| 167 |
"temperature": 0.8,
|
| 168 |
"top_p": 0.95,
|