t5-small-finetuned-xsum

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.0804
• Rouge1: 21.7575
• Rouge2: 8.5919
• Rougel: 17.3288
• Rougelsum: 20.4481
• Gen Len: 18.8222

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	45	2.3869	21.4625	7.7924	16.4408	19.7799	18.7667
No log	2.0	90	2.3161	22.2793	8.0559	17.0177	20.6462	18.8444
No log	3.0	135	2.2576	21.9986	7.8751	16.7895	20.2286	18.6778
No log	4.0	180	2.2061	21.9707	8.2401	16.9102	20.2145	18.6333
No log	5.0	225	2.1667	22.1615	8.3056	17.0849	20.48	18.8222
No log	6.0	270	2.1350	21.942	8.5934	17.2273	20.4631	18.7444
No log	7.0	315	2.1102	21.8541	8.664	17.2851	20.4798	18.7444
No log	8.0	360	2.0939	21.967	8.675	17.4126	20.5475	18.8222
No log	9.0	405	2.0841	21.824	8.6682	17.3674	20.4822	18.8222
No log	10.0	450	2.0804	21.7575	8.5919	17.3288	20.4481	18.8222

Framework versions

• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1