t5-small-squad-qg

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2888

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss
0.6959	0.9259	50	0.4512
0.3276	1.8519	100	0.3444
0.3088	2.7778	150	0.3162
0.2913	3.7037	200	0.3071
0.2824	4.6296	250	0.2994
0.3002	5.5556	300	0.2937
0.2718	6.4815	350	0.2942
0.2772	7.4074	400	0.2934
0.2782	8.3333	450	0.2906
0.2786	9.2593	500	0.2910
0.2753	10.1852	550	0.2907
0.2746	11.1111	600	0.2897
0.2722	12.0370	650	0.2889
0.2719	12.9630	700	0.2890
0.2768	13.8889	750	0.2884
0.2706	14.8148	800	0.2888

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.1+cu121
• Datasets 2.13.1
• Tokenizers 0.19.1