GUE_EMP_H4-seqsight_65536_512_47M-L32_f

This model is a fine-tuned version of mahdibaghbanzadeh/seqsight_65536_512_47M on the mahdibaghbanzadeh/GUE_EMP_H4 dataset. It achieves the following results on the evaluation set:

• Loss: 0.2723
• F1 Score: 0.8953
• Accuracy: 0.8953

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Score	Accuracy
0.3593	2.17	200	0.2935	0.8873	0.8871
0.2808	4.35	400	0.2879	0.8960	0.8960
0.2632	6.52	600	0.2894	0.8888	0.8891
0.252	8.7	800	0.2906	0.8888	0.8884
0.2352	10.87	1000	0.2793	0.9110	0.9110
0.2293	13.04	1200	0.2952	0.8901	0.8898
0.2172	15.22	1400	0.2890	0.8948	0.8946
0.2113	17.39	1600	0.3144	0.8909	0.8905
0.2004	19.57	1800	0.3055	0.8945	0.8946
0.1942	21.74	2000	0.3162	0.8907	0.8905
0.1835	23.91	2200	0.3497	0.8696	0.8693
0.1786	26.09	2400	0.3230	0.8819	0.8816
0.1698	28.26	2600	0.3381	0.8858	0.8857
0.1611	30.43	2800	0.3506	0.8852	0.8850
0.1532	32.61	3000	0.3809	0.8799	0.8802
0.1489	34.78	3200	0.3671	0.8791	0.8789
0.1385	36.96	3400	0.3798	0.8786	0.8782
0.1347	39.13	3600	0.3871	0.8758	0.8754
0.1278	41.3	3800	0.4102	0.8761	0.8761
0.1241	43.48	4000	0.4262	0.8790	0.8789
0.1173	45.65	4200	0.4611	0.8715	0.8720
0.1122	47.83	4400	0.4375	0.8797	0.8795
0.11	50.0	4600	0.4266	0.8786	0.8789
0.1039	52.17	4800	0.4801	0.8736	0.8734
0.1057	54.35	5000	0.4509	0.8775	0.8775
0.0953	56.52	5200	0.4760	0.8717	0.8713
0.0926	58.7	5400	0.5029	0.8683	0.8679
0.0903	60.87	5600	0.4814	0.8722	0.8720
0.0863	63.04	5800	0.5023	0.8729	0.8727
0.0856	65.22	6000	0.5227	0.8670	0.8665
0.0833	67.39	6200	0.5262	0.8677	0.8672
0.0783	69.57	6400	0.5150	0.8695	0.8693
0.0761	71.74	6600	0.5296	0.8734	0.8734
0.0727	73.91	6800	0.5547	0.8704	0.8700
0.0705	76.09	7000	0.5961	0.8663	0.8658
0.0718	78.26	7200	0.5728	0.8608	0.8604
0.0666	80.43	7400	0.5711	0.8695	0.8693
0.0657	82.61	7600	0.5681	0.8652	0.8652
0.0638	84.78	7800	0.5880	0.8697	0.8693
0.0616	86.96	8000	0.5926	0.8695	0.8693
0.0638	89.13	8200	0.5964	0.8641	0.8638
0.0638	91.3	8400	0.5819	0.8708	0.8706
0.0594	93.48	8600	0.5993	0.8680	0.8679
0.0574	95.65	8800	0.5968	0.8675	0.8672
0.0586	97.83	9000	0.5952	0.8640	0.8638
0.0584	100.0	9200	0.6028	0.8614	0.8611
0.0583	102.17	9400	0.6088	0.8640	0.8638
0.0575	104.35	9600	0.6062	0.8682	0.8679
0.0576	106.52	9800	0.6077	0.8668	0.8665
0.0553	108.7	10000	0.6073	0.8667	0.8665

Framework versions

• PEFT 0.9.0
• Transformers 4.38.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2