Post
1610
We explore extremely low-weight merger as an alternative to fine-tuning. Merge formula details here:
grimjim/kukulemon-v3-soul_mix-32k-7B
grimjim/kukulemon-v3-soul_mix-32k-7B
Jim Lai
grimjim
AI & ML interests
Experimenting with 7B-9B parameter text completion models. Not all models are intended for direct use, but for educational and/or merge purposes.
Organizations
Posts
4
Post
1261
I propose "merge densification", a style of merger which attempts to transfer the benefits of a denser model to a base model. The model weight in this case is 0.02, which is atypically small for mergers, but high compared to the learning rate used during training. In this case, the expectation is more creative text-generation. More details below:
grimjim/kunoichi-lemon-royale-v3-32K-7B
grimjim/kunoichi-lemon-royale-v3-32K-7B
Collections
4
Models helpfully quantized by others as well as myself
-
grimjim/kuno-kunoichi-v1-DPO-v2-SLERP-7B-GGUF
Text Generation • Updated • 108 • 1 -
Lewdiculous/kuno-kunoichi-v1-DPO-v2-SLERP-7B-GGUF-IQ-Imatrix
Updated • 340 • 9 -
grimjim/kuno-kunoichi-v1-DPO-v2-SLERP-7B-8.0bpw_h8_exl2
Text Generation • Updated • 3 -
grimjim/kukulemon-7B-GGUF
Text Generation • Updated • 761 • 1
Models uploaded in safetensors format. Currently all are merges.
models
61
grimjim/Llama-3-Luminurse-v0.1-OAS-8B-GGUF
Text Generation
•
Updated
grimjim/Llama-3-Luminurse-v0.1-OAS-8B
Text Generation
•
Updated
•
4
•
3
grimjim/kukulemon-v3-soul_mix-32k-7B
Text Generation
•
Updated
•
155
•
3
grimjim/kukulemon-v3-soul_mix-32k-7B-GGUF
Text Generation
•
Updated
•
19
grimjim/koboldai-holodeck-extended-32k-7B
Text Generation
•
Updated
•
17
grimjim/koboldai-erebus-extended-32k-7B
Text Generation
•
Updated
•
18
•
2
grimjim/llama-3-aaditya-OpenBioLLM-8B
Text Generation
•
Updated
•
9
grimjim/kunoichi-lemon-royale-v3-32K-7B
Text Generation
•
Updated
•
224
•
3
grimjim/kunoichi-lemon-royale-v3-32K-7B-GGUF
Text Generation
•
Updated
•
46
•
1
grimjim/wizard-elem-to-32k-7B
Text Generation
•
Updated