crewai use case
I'm interested in your function calling fine tune of llama3 70b. I'm particularly interested to know if anyone has it working with crewai as the original instruct model isn't working for me. Frequently failing to call other agents correctly etc.
A few thoughts:
Can you try out the free/public 8B llama 3 model I made for function calling? if that works, then 70B will just be better.
You should know that the way Trelis function calling models work is by using a custom chat template. This must be used for formatting prompts as it injects the function metadata. You should check out this vid to understand more: https://www.youtube.com/watch?v=hHn_cV5WUDI
I scanned crewai - it's not entirely clear to me but it seems the agents are pre-set (i.e. you can't swap in a custom model). However, you can hit a custom endpoint/api - so maybe you could run a custom model on runpod and hit that model with crewai.
If you have more details on exactly what you're doing and why crewai can't do it with the default agents, that can maybe give me more context to help.
Holding off for the moment as GGUF llama 3 models seem to not be performing well due to BPE tokenisation error. When that is fixed in llama.cpp I'll revisit. Thank you for the information.