GUE_EMP_H4ac-seqsight_65536_512_47M-L1_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H4ac dataset. It achieves the following results on the evaluation set:

• Loss: 0.5702
• F1 Score: 0.7087
• Accuracy: 0.7085

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.6417	0.93	200	0.5973	0.6815	0.6812
0.6039	1.87	400	0.6004	0.6859	0.6871
0.5916	2.8	600	0.5816	0.7040	0.7038
0.5863	3.74	800	0.5817	0.7044	0.7041
0.5795	4.67	1000	0.5882	0.7058	0.7062
0.5751	5.61	1200	0.5956	0.7051	0.7067
0.5732	6.54	1400	0.5767	0.7128	0.7126
0.5653	7.48	1600	0.5786	0.7123	0.7120
0.5723	8.41	1800	0.5774	0.7119	0.7117
0.5682	9.35	2000	0.5854	0.7109	0.7117
0.5614	10.28	2200	0.5768	0.7124	0.7123
0.5653	11.21	2400	0.5738	0.7158	0.7158
0.5605	12.15	2600	0.5763	0.7136	0.7138
0.559	13.08	2800	0.5887	0.7114	0.7126
0.5598	14.02	3000	0.5760	0.7146	0.7150
0.5565	14.95	3200	0.5703	0.7176	0.7176
0.5541	15.89	3400	0.5891	0.7101	0.7120
0.552	16.82	3600	0.5692	0.7192	0.7191
0.5579	17.76	3800	0.5672	0.7212	0.7211
0.5528	18.69	4000	0.5698	0.7187	0.7188
0.5492	19.63	4200	0.5783	0.7161	0.7170
0.5525	20.56	4400	0.5653	0.7226	0.7226
0.5496	21.5	4600	0.5951	0.7070	0.7103
0.5495	22.43	4800	0.5678	0.7221	0.7223
0.5521	23.36	5000	0.5792	0.7182	0.7196
0.5458	24.3	5200	0.5668	0.7237	0.7238
0.5497	25.23	5400	0.5603	0.7257	0.7255
0.5482	26.17	5600	0.5680	0.7232	0.7235
0.5479	27.1	5800	0.5718	0.7214	0.7223
0.5439	28.04	6000	0.5623	0.7295	0.7293
0.5477	28.97	6200	0.5758	0.7186	0.7196
0.5463	29.91	6400	0.5683	0.7237	0.7240
0.5461	30.84	6600	0.5867	0.7164	0.7185
0.5448	31.78	6800	0.5662	0.7250	0.7252
0.5426	32.71	7000	0.5676	0.7240	0.7243
0.5419	33.64	7200	0.5682	0.7239	0.7246
0.5439	34.58	7400	0.5696	0.7216	0.7223
0.5425	35.51	7600	0.5626	0.7284	0.7284
0.5385	36.45	7800	0.5638	0.7287	0.7287
0.5443	37.38	8000	0.5762	0.7198	0.7211
0.5399	38.32	8200	0.5670	0.7270	0.7276
0.5409	39.25	8400	0.5653	0.7284	0.7287
0.5439	40.19	8600	0.5633	0.7277	0.7279
0.5406	41.12	8800	0.5669	0.7262	0.7267
0.5393	42.06	9000	0.5684	0.7268	0.7273
0.543	42.99	9200	0.5738	0.7209	0.7220
0.5384	43.93	9400	0.5725	0.7238	0.7246
0.5406	44.86	9600	0.5664	0.7266	0.7270
0.542	45.79	9800	0.5679	0.7259	0.7264
0.5386	46.73	10000	0.5694	0.7248	0.7255

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2