Instructions to use ChanceFocus/finma-7b-nlp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ChanceFocus/finma-7b-nlp with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ChanceFocus/finma-7b-nlp")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("ChanceFocus/finma-7b-nlp") model = AutoModelForCausalLM.from_pretrained("ChanceFocus/finma-7b-nlp") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use ChanceFocus/finma-7b-nlp with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ChanceFocus/finma-7b-nlp" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChanceFocus/finma-7b-nlp", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ChanceFocus/finma-7b-nlp
- SGLang
How to use ChanceFocus/finma-7b-nlp with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ChanceFocus/finma-7b-nlp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChanceFocus/finma-7b-nlp", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ChanceFocus/finma-7b-nlp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ChanceFocus/finma-7b-nlp", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ChanceFocus/finma-7b-nlp with Docker Model Runner:
docker model run hf.co/ChanceFocus/finma-7b-nlp
Tokenizer
Can you please upload tokenizer-related files? Thank you. I am getting the following error.
OSError: Can't load tokenizer for 'ChanceFocus/finma-7b-nlp'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'ChanceFocus/finma-7b-nlp' is the correct path to a directory containing all relevant files for a LlamaTokenizerFast tokenizer.
Absolutely, my apologies for the inconvenience. We previously stored the tokenizer in the backbone model directory, which might have led to this confusion. We will promptly upload our tokenizer and update our README and model card to rectify this issue. Please stay tuned and thank you for your patience!
Thank you.
Thank you for your understanding. I'm happy to inform you that we have now updated our usage instructions and uploaded the tokenizer-related files for FinMA-7B-NLP. We have also thoroughly tested it to ensure that it works seamlessly now. We apologize for any inconvenience caused and appreciate your patience. Please feel free to give it another try. Thank you!