Trying to use with ollama
Error: 400 Bad Request: template error: template: :92: function "content" not defined
any ideas?
Ollama doesn't use the same kind of jinja templates as the supported clients; I believe it uses a Go-specific form of template. This is why I abandoned Ollama in favor of llama.cpp; I got tired of trying to translate jinja dialetcs when I wanted to run something not provided directly by Ollama.
yeah that's so annoying, it states that it uses jinja now but i guess it doesn't really
ollama is so much easier for memory management when you're running multiple models though
I think llama-swap can do the same kinda management and model swapping job for llama.cpp :) Haven't used it myself because I haven't had the need yet, but maybe that will help you.
llama-swap is amazing, the config is a little less user friendly, but you have your LLM's for that, just have them write the config files