Arabic
arabic
tokenizer
morphology
nlp
dialect
df-arc / tokenizer.json
fr3on's picture
vocab_size Increased from 64000 to 128000
34fc84b verified
This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )
SHA256:
2c80d2f37438968d6e27081d42c5dc04da9bb631ccab5a87fc02cebf24a3f689
Pointer size:
132 Bytes
·
Size of remote file:
8.86 MB
·
Xet hash:
da90bffd75a81ca1e21d63e853efe97c7f17e864cf4e3e5c01ca2693a2dcc859

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.