switch-base-8-finetuned-samsum

This model is a fine-tuned version of google/switch-base-8 on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.4606
• Rouge1: 46.5651
• Rouge2: 23.2378
• Rougel: 39.4484
• Rougelsum: 43.1011
• Gen Len: 17.0183

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.8829	1.0	3683	1.5154	46.3805	23.0982	39.0612	43.0142	17.6296
1.6207	2.0	7366	1.4578	47.7434	24.9471	40.6481	44.351	17.2066
1.442	3.0	11049	1.4360	47.6903	24.9954	40.713	44.3487	17.0501
1.3103	4.0	14732	1.4396	48.4517	25.7725	41.5212	45.1211	16.9071
1.2393	5.0	18415	1.4445	48.4002	25.8727	41.5361	45.0467	16.9804

Framework versions

• Transformers 4.25.1
• Pytorch 1.13.1+cu116
• Datasets 2.8.0
• Tokenizers 0.13.2