File size: 1,550 Bytes
21035f8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 | ---
pipeline_tag: text-classification
tags:
- sentiment-analysis
- transformer
- custom
- pytorch
- trained-from-scratch
datasets:
- stanfordnlp/imdb
- stanfordnlp/sentiment140
- SetFit/sst5
- financial_phrasebank
- tweet_eval
language:
- en
license: mit
---
# Sentiment Transformer — tango
A small (≈13M parameter) transformer encoder trained **entirely from scratch** for
3-class sentiment analysis (negative / neutral / positive).
## Architecture
Pre-layer-norm transformer encoder with [CLS] pooling and a linear classification head.
Built with pure `torch.nn` — no pretrained weights.
| Parameter | Value |
|---|---|
| Hidden dim | 256 |
| FFN dim | 1024 |
| Layers | 6 |
| Heads | 8 |
| Max seq len | 256 |
| Vocab size | 16000 |
| Labels | NEGATIVE, NEUTRAL, POSITIVE |
| Precision | bf16 mixed-precision |
## Training Data
Trained on a combined corpus of:
- **IMDB** (50k movie reviews)
- **Sentiment140** (1M tweets)
- **Yelp** (1M reviews)
- **SST-5** (fine-grained → 3-class)
- **Financial PhraseBank** (finance headlines)
- **TweetEval** (SemEval-2017 tweets)
## Usage
```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline
model = AutoModelForSequenceClassification.from_pretrained(
"Impulse2000/sentiment-transformer", trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("Impulse2000/sentiment-transformer")
pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
print(pipe("This movie was absolutely fantastic!"))
```
|