How to use NemoStation/Marlin-2B with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForCausalLM processor = AutoProcessor.from_pretrained("NemoStation/Marlin-2B", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("NemoStation/Marlin-2B", trust_remote_code=True)
The video capability is cool, but can you also perform inference with images or just text?
Also would freezing the vision encoder work to train the LLM part of the model but keep its capabilities to use the vision encoder?
· Sign up or log in to comment
