model documentation

#2
by nazneen - opened
Files changed (1) hide show
  1. README.md +170 -5
README.md CHANGED
@@ -1,21 +1,186 @@
 
1
  ---
2
  language:
3
  - ru
 
4
  tags:
5
  - PyTorch
6
  - Transformers
7
  - bert
8
  - exbert
9
  pipeline_tag: fill-mask
10
- thumbnail: "https://github.com/sberbank-ai/model-zoo"
11
- license: apache-2.0
12
  ---
13
 
14
- # ruBert-large
15
- Model was trained by [SberDevices](https://sberdevices.ru/) team.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  * Task: `mask filling`
17
  * Type: `encoder`
18
  * Tokenizer: `bpe`
19
  * Dict size: `120 138`
20
  * Num Parameters: `178 M`
21
- * Training Data Volume `30 GB`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
  ---
3
  language:
4
  - ru
5
+ license: apache-2.0
6
  tags:
7
  - PyTorch
8
  - Transformers
9
  - bert
10
  - exbert
11
  pipeline_tag: fill-mask
12
+ thumbnail: https://github.com/sberbank-ai/model-zoo
 
13
  ---
14
 
15
+
16
+ # Model Card for ruBert-large
17
+
18
+
19
+ # Model Details
20
+
21
+ ## Model Description
22
+
23
+
24
+ - **Developed by:** Sberbank-ai
25
+ - **Shared by [Optional]:** Hugging Face
26
+ - **Model type:** Fill-Mask
27
+ - **Language(s) (NLP):** ru
28
+ - **License:** apache-2.0
29
+ - **Related Models:** exbert
30
+ - **Parent Model:** bert
31
+ - **Resources for more information:**
32
+ - [GitHub Repo](https://github.com/sberbank-ai/model-zoo)
33
+ - [Associated Paper](https://arxiv.org/abs/1810.04805)
34
+
35
+ # Uses
36
+
37
+ ## Direct Use
38
+
39
+ Fill-Mask
40
+
41
+ ## Downstream Use [Optional]
42
+
43
+
44
+ More information needed.
45
+
46
+
47
+ ## Out-of-Scope Use
48
+
49
+ The model should not be used to intentionally create hostile or alienating environments for people.
50
+
51
+
52
+ # Bias, Risks, and Limitations
53
+
54
+
55
+ Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
56
+
57
+
58
+ ## Recommendations
59
+
60
+
61
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recomendations.
62
+
63
+
64
+ # Training Details
65
+
66
+ ## Training Data
67
+
68
+ More information needed
69
+
70
+ ## Training Procedure
71
+
72
+
73
+
74
+ ### Preprocessing
75
+
76
+ More information needed
77
+
78
+ ### Speeds, Sizes, Times
79
+
80
  * Task: `mask filling`
81
  * Type: `encoder`
82
  * Tokenizer: `bpe`
83
  * Dict size: `120 138`
84
  * Num Parameters: `178 M`
85
+ * Training Data Volume `30 GB`
86
+
87
+
88
+ # Evaluation
89
+
90
+
91
+
92
+ ## Testing Data, Factors & Metrics
93
+
94
+ ### Testing Data
95
+
96
+
97
+
98
+ More information needed
99
+
100
+ ### Factors
101
+
102
+ More information needed
103
+
104
+ ### Metrics
105
+
106
+ More information needed
107
+
108
+ ## Results
109
+
110
+ | Model | Task | Type | Tokenizer | Dict size | Num Parameters | Training Data Volume |
111
+ |-----------------|----------------------|-----------------|-----------|-----------|-----------------|----------------------|
112
+ | ruBERT-large | mask filling | encoder | bpe | 120 138 | 427 M | 30 GB |
113
+
114
+ # Model Examination
115
+
116
+ More information needed
117
+ # Environmental Impact
118
+
119
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
120
+
121
+ - **Hardware Type:** More information needed
122
+ - **Hours used:** More information needed
123
+ - **Cloud Provider:** More information needed
124
+ - **Compute Region:** More information needed
125
+ - **Carbon Emitted:** More information needed
126
+
127
+ # Technical Specifications [optional]
128
+
129
+ ## Model Architecture and Objective
130
+
131
+ More information needed
132
+
133
+ ## Compute Infrastructure
134
+ More information needed
135
+
136
+ ### Hardware
137
+
138
+ More information needed
139
+
140
+ ### Software
141
+
142
+ More information needed
143
+
144
+
145
+ # Citation
146
+
147
+
148
+ **BibTeX:**
149
+
150
+ More information needed
151
+
152
+ **APA:**
153
+
154
+ More information needed
155
+
156
+ # Glossary [optional]
157
+
158
+ More information needed
159
+
160
+ # More Information [optional]
161
+
162
+ More information needed
163
+ # Model Card Authors [optional]
164
+
165
+ Sberbank-ai in collaberation with Ezi Ozoani and the Hugging Face team
166
+
167
+ # Model Card Contact
168
+ More information needed
169
+
170
+ # How to Get Started with the Model
171
+
172
+ Use the code below to get started with the model.
173
+
174
+ <details>
175
+ <summary> Click to expand </summary>
176
+
177
+ ```python
178
+ from transformers import AutoTokenizer, AutoModelForTokenClassification
179
+
180
+ tokenizer = AutoTokenizer.from_pretrained("asahi417/tner-xlm-roberta-base-ontonotes5")
181
+
182
+ model = AutoModelForTokenClassification.from_pretrained("asahi417/tner-xlm-roberta-base-ontonotes5")
183
+
184
+ ```
185
+ </details>
186
+