Upload Qwen2ForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,28 @@
+{
+  "_name_or_path": "./merged_model",
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 8192,
+  "initializer_range": 0.02,
+  "intermediate_size": 29568,
+  "max_position_embeddings": 32768,
+  "max_window_layers": 80,
+  "model_type": "qwen2",
+  "num_attention_heads": 64,
+  "num_hidden_layers": 80,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.1",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151646
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 151645,
+  "transformers_version": "4.48.1",
+  "use_cache": false
+}

model-00001-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:157896f26462aca119a9f0d1b67bc68813dc44a196979da1fc92cf13b6a6ef2e
+size 4969136264

model-00002-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63bd6990dc69a7e6137014514a151851d4baaa6bedefb6490a5193dad104162f
+size 4114761856

model-00003-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14cf299d5d81f414d7ec04903ed593efabc3a371bd88896efe452883b8bf8e7b
+size 4479624664

model-00004-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:597d14383319c8a4ac41fac109048dd907f508fa7be91cf6ada0cdf6ea4b4520
+size 4479624664

model-00005-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b0c85db0d7ceb0d8f79980c6dae84fddccc385759dbb794bc27af584fdbd738
+size 4815276384

model-00006-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fcf8211326f04a31e0e1e234da7926f63ff974bdf5d021f50ef3d8dc9b4293d
+size 4748060240

model-00007-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a20ec9c1e51845aff44c5de3e7a34da8b7bb449d1200f63b137123cf62f7db68
+size 4479624664

model-00008-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c52eb9b6f435ebfa9c8509b761e1520f242c8537a3b8c02d6674c0c0ca20e4d
+size 4815276384

model-00009-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdcbd31d7b1753270c802cb83249a91c513c6a8675bc1bf583903631e8ed2020
+size 4748060256

model-00010-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b798828a69f3fd08ba187d13c893b62ed8b615d4becf7ec2067749ce1e90d67
+size 4479624672

model-00011-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e8520e0a3a96978279d687b5e06bc0bb7af4b764822d80bdf3e2dee04246464
+size 4815276400

model-00012-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b91c01d7235868f2c5d76ea7d347406ac0dc98bd9eb63c36303cb7af7161e0a
+size 4748060248

model-00013-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77fd648c5324905f45d08027ec6469c420008aa5663672cd67ba3b519ba0bdf4
+size 4479624672

model-00014-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69515cf0aa3b0f4134f589aed162182ade5c26ba4451d04ff0e1c8ed6bb888d1
+size 4815276400

model-00015-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:331e5c77f0a1cd18e78c6788cc93c7bc08af94c8a3bc18916c8e5392e44cd92c
+size 4748060248

model-00016-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d44ac8e3500a9a77201ce2cec93c39985b890a748b838b80c25a551817ef7e25
+size 4479624672

model-00017-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9de400aab7737e009a556ee77ef5fd5a883fe1c1eef2560fb18e72114d548465
+size 4815276400

model-00018-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18eb20d0185bb3a898c936744c6b30f220fcb4ea734ee8634f86d64996c93882
+size 4748060248

model-00019-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b17c6986d2dbdeb5e0641d83cf926d0141d4e728e21a6c055f3eda214a6346e7
+size 4479624672

model-00020-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8074ec043c013a5d910c123c95bc033de2138347ee90b6ee8f83f877a6709c1
+size 4815276400

model-00021-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5084c06a3d5370865789f96e948282fb1fa526fade3657c03bbcf45f583c71a
+size 4748060248

model-00022-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16b7df51c869fe40343a0a3deb0a1724fb79e8536783ee303672644def5ce9a4
+size 4479624672

model-00023-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6041ef070beab28f373fc328bfd6e44706f9aad051eabd842a5aec6bf0d6758f
+size 4815276400

model-00024-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5b00b3f9502e7c1d063d0fd0f6912405fff10b521437dded7944e3c13d459f0
+size 4748060248

model-00025-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32e1ef7247377338fbbdeee8698ea6e63df8eca92f1517bcf3339725e9fb5a70
+size 4479624672

model-00026-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b56ee4a8a9c74bce9a8bdef0601147e72e1af8fc579e1a1ec9abf368583a6e7
+size 4815276400

model-00027-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5ace4f396a64d2f68c5686c6998da56563ebe0ef756a2bf47bee537dff25fc6
+size 4748060248

model-00028-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79393d507ce318d99d09a60c7991aa0ed5fdc0eb948e4841d118a9422583ec53
+size 4479624672

model-00029-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e097c8a081965f256b95b954fbb379aba54206242aed556e65dc5155ab14d716
+size 4815276400

model-00030-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52dfdd10d8421ec5e758f89f08dad6ee79b4318d5354b8192bce85914892ae22
+size 4748060248

model-00031-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7aca76a42665b6ddf3251b708ba9f8483ea574fa7548b103fce2743bb111b4fa
+size 4479624672

model-00032-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:413a05e45e6e6a5178da4f3056f2024a501c9ba082ab9bcefc49f757dc8d7ee0
+size 4815276400

model-00033-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4cb6fe29a3b33892f00be7cddf825719339d908506d247cbcf09314cadfe2bdf
+size 4748060248

model-00034-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89c6edc1521069d0616795f8b64883243e98dc53aed5e18d88b065eefdf5f5fb
+size 4479624672

model-00035-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b2dae2e88210b515fd3233ae9c131e2ed83bee8d9248472405535069a536162
+size 4815276400

model-00036-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc839ff155aefd5426c9691a79f8b2295b470ced02ec774cddc57eae06378bfd
+size 4748060248

model-00037-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:233a73264028898380008703f3455ce7da71214cc4fe6bb5f159915f832b2109
+size 4479624672

model-00038-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23b435adc9cc30ef56c8faa917fcaad69e6dba7172064e294c23d19683259018
+size 4815276400

model-00039-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:599c04377cb5a6e6f1b70a6ec39a1504215c3513974b8b5d782ac38551b20be3
+size 4748060248

model-00040-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d1744e738ea97073a3572e6f3693f08826edb3b925bee75ac95b1ae90149d03
+size 4479624672

model-00041-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:427acf501104f1d3aa185f8ffda2a1723997616a9dd47991df996d5180f6be42
+size 4815276400

model-00042-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d4ec9a83f07fcb44e7a5fc7b4cb181ceb2c7529e1bcce77425a82fd1e3d4f97
+size 4748060248

model-00043-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:263488dbc23b34e923814d86fc73f7722d7e22ebcbbbb77b3c615f431b7aa9ac
+size 4479624672

model-00044-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e4825ead355877043f3f76c2655874bd1d6bb8490cd3fb89deb9352da1119d6
+size 4815276400

model-00045-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04e708a40c93cd0a65803ccb4b4e8d038cc41911f6eabdf3d649e616c396c2aa
+size 4748060248

model-00046-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9251bc3c016c7168c175605ccff9e1172390b98b4b662448d9653548cf54d21e
+size 4479624672

model-00047-of-00063.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c70f57b7933833c2fd0576555c20c55a91138be285de985c5db2a08d852f7a3d
+size 4815276400