GUE_EMP_H3K79me3-seqsight_65536_512_47M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H3K79me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4374
• F1 Score: 0.8193
• Accuracy: 0.8193

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5042	1.1	200	0.4523	0.8134	0.8135
0.4575	2.21	400	0.4432	0.8135	0.8135
0.453	3.31	600	0.4381	0.8068	0.8079
0.4367	4.42	800	0.4333	0.8096	0.8107
0.4327	5.52	1000	0.4302	0.8134	0.8145
0.423	6.63	1200	0.4528	0.8043	0.8065
0.4233	7.73	1400	0.4418	0.8010	0.8031
0.4154	8.84	1600	0.4534	0.7936	0.7961
0.4116	9.94	1800	0.4231	0.8144	0.8145
0.4052	11.05	2000	0.4394	0.8028	0.8037
0.4028	12.15	2200	0.4245	0.8196	0.8197
0.397	13.26	2400	0.4251	0.8144	0.8148
0.3917	14.36	2600	0.4285	0.8201	0.8200
0.3907	15.47	2800	0.4296	0.8129	0.8131
0.3827	16.57	3000	0.4302	0.8171	0.8169
0.3821	17.68	3200	0.4380	0.8186	0.8187
0.3754	18.78	3400	0.4418	0.8105	0.8110
0.371	19.89	3600	0.4367	0.8177	0.8176
0.3684	20.99	3800	0.4477	0.8107	0.8110
0.3639	22.1	4000	0.4422	0.8158	0.8159
0.3605	23.2	4200	0.4480	0.8144	0.8145
0.3561	24.31	4400	0.4502	0.8163	0.8166
0.3478	25.41	4600	0.4584	0.8175	0.8173
0.3503	26.52	4800	0.4596	0.8121	0.8121
0.3491	27.62	5000	0.4524	0.8113	0.8117
0.3407	28.73	5200	0.4644	0.8110	0.8117
0.3349	29.83	5400	0.4509	0.8151	0.8152
0.3364	30.94	5600	0.4585	0.8171	0.8169
0.3328	32.04	5800	0.4492	0.8199	0.8197
0.3307	33.15	6000	0.4530	0.8164	0.8166
0.3277	34.25	6200	0.4746	0.8175	0.8173
0.3223	35.36	6400	0.4711	0.8181	0.8183
0.3192	36.46	6600	0.4757	0.8187	0.8187
0.3178	37.57	6800	0.4753	0.8139	0.8141
0.3153	38.67	7000	0.4703	0.8165	0.8169
0.3129	39.78	7200	0.4812	0.8196	0.8197
0.3105	40.88	7400	0.4763	0.8143	0.8141
0.3064	41.99	7600	0.4652	0.8180	0.8180
0.306	43.09	7800	0.4787	0.8145	0.8145
0.3041	44.2	8000	0.4898	0.8150	0.8152
0.3014	45.3	8200	0.4882	0.8173	0.8173
0.3005	46.41	8400	0.4859	0.8173	0.8173
0.3006	47.51	8600	0.4895	0.8143	0.8145
0.2973	48.62	8800	0.4882	0.8124	0.8124
0.2961	49.72	9000	0.4937	0.8140	0.8141
0.3008	50.83	9200	0.4829	0.8128	0.8131
0.2934	51.93	9400	0.4918	0.8133	0.8135
0.2928	53.04	9600	0.4910	0.8149	0.8148
0.2936	54.14	9800	0.4936	0.8156	0.8155
0.2934	55.25	10000	0.4941	0.8135	0.8135

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2