Can this be dropped into openai compatible apis to support function calling / tools in langchain/langraph?
Can this be used with langchain / langchain tools through openai compatible apis? I would normally test this on my own, but since its paywalled I cannot. I am looking for an local model to run langchain/langgraph as an openai alternative, but most function calling models are not capable of function calling and coding in a compatible way.
Howdy @rakataprime , can you post one sample prompt (completely formatted, including bos, eos etc. tokens) here for me to see. That will help me tell you either a) this can work or b) what I need to do to adjust it to work.
here is a full end to end example that I have used for testing function calling with langgraph. https://github.com/rakataprime/local_llm_langgraph. I will use this to generate/format some sample prompts tomorrow including the tokens.