GUE_EMP_H3K79me3-seqsight_65536_512_47M-L8_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H3K79me3 dataset. It achieves the following results on the evaluation set:

• Loss: 0.4309
• F1 Score: 0.8185
• Accuracy: 0.8187

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.5148	1.1	200	0.4601	0.8090	0.8089
0.4624	2.21	400	0.4479	0.8057	0.8065
0.4592	3.31	600	0.4440	0.8066	0.8076
0.4474	4.42	800	0.4400	0.8033	0.8044
0.4463	5.52	1000	0.4429	0.8030	0.8048
0.4386	6.63	1200	0.4471	0.8024	0.8048
0.4403	7.73	1400	0.4353	0.8077	0.8089
0.4328	8.84	1600	0.4478	0.8019	0.8041
0.4305	9.94	1800	0.4266	0.8190	0.8190
0.4265	11.05	2000	0.4371	0.8041	0.8051
0.4265	12.15	2200	0.4269	0.8185	0.8183
0.4207	13.26	2400	0.4243	0.8151	0.8155
0.4176	14.36	2600	0.4245	0.8184	0.8183
0.4192	15.47	2800	0.4285	0.8111	0.8117
0.414	16.57	3000	0.4283	0.8175	0.8173
0.4149	17.68	3200	0.4244	0.8161	0.8162
0.4094	18.78	3400	0.4262	0.8172	0.8176
0.4091	19.89	3600	0.4239	0.8140	0.8141
0.4087	20.99	3800	0.4302	0.8091	0.8100
0.4076	22.1	4000	0.4246	0.8108	0.8114
0.4059	23.2	4200	0.4253	0.8144	0.8148
0.4057	24.31	4400	0.4300	0.8124	0.8131
0.3982	25.41	4600	0.4299	0.8159	0.8162
0.4019	26.52	4800	0.4289	0.8184	0.8187
0.4036	27.62	5000	0.4294	0.8112	0.8121
0.3975	28.73	5200	0.4243	0.8114	0.8121
0.3938	29.83	5400	0.4255	0.8134	0.8138
0.3966	30.94	5600	0.4280	0.8160	0.8162
0.3953	32.04	5800	0.4275	0.8214	0.8214
0.3972	33.15	6000	0.4261	0.8150	0.8155
0.3931	34.25	6200	0.4297	0.8170	0.8173
0.3914	35.36	6400	0.4287	0.8140	0.8145
0.393	36.46	6600	0.4275	0.8181	0.8183
0.3901	37.57	6800	0.4299	0.8136	0.8141
0.3893	38.67	7000	0.4314	0.8153	0.8159
0.3881	39.78	7200	0.4304	0.8184	0.8187
0.3886	40.88	7400	0.4277	0.8189	0.8190
0.3859	41.99	7600	0.4314	0.8162	0.8166
0.3869	43.09	7800	0.4308	0.8169	0.8173
0.3859	44.2	8000	0.4329	0.8149	0.8155
0.3839	45.3	8200	0.4341	0.8159	0.8162
0.3871	46.41	8400	0.4291	0.8184	0.8187
0.3848	47.51	8600	0.4327	0.8172	0.8176
0.3837	48.62	8800	0.4334	0.8164	0.8169
0.383	49.72	9000	0.4334	0.8158	0.8162
0.388	50.83	9200	0.4328	0.8160	0.8166
0.3826	51.93	9400	0.4316	0.8169	0.8173
0.3819	53.04	9600	0.4315	0.8166	0.8169
0.3815	54.14	9800	0.4318	0.8170	0.8173
0.3831	55.25	10000	0.4325	0.8166	0.8169

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2