Trelis

/

deepseek-coder-33b-instruct-function-calling-v2

like 2

Hi Grant, thanks for having provided a clear example.

I've read through and the responses all seem reasonable.

What kind of response were you expecting to the second question?

Hi Ronan, for the second question, I was expecting the model to return with function format, something like
{
"function": "write_file",
"arguments": {
"filename": "out.py",
"contents": "print('Hello World!')\n"
}
}
Is there anyway to achieve this?

A few thoughts:

1. "Write a file, creating it if necessary. If the file exists, it is overwritten."
   This function description is unclear because a language model cannot "create" a file, and it's confusing what the difference is between "write" and "create". Probably it's best to call it "save_file" and put the description as "Function to call if asked to save a file.". If you further want the language model to know whether files can be overwritten or not, then you need to provide a list of files to the LLM and further explain that in the description.

2. "Write a python program to out.py to print hello world"

There isn't a clear way for the LLM to know whether to just write a response (i.e. respond) with the python code OR use the write_file function. Again, using save_file as the function name will probably help because then you could say something like "Write a python program that prints -hello world- and save that program as out.py.".

BTW, chatgpt is very good at helping to refine your function defintions and prompts. You coud paste everything you wrote into chatgpt, tell it you are feeding into an LLM that is fine-tuned for function calling, and ask how to improve the function description and prompt.

Hi Ronan,

Just for comparison, I tried the same prompt provided before with gpt-3.5-turbo-0613 model using tool calls, this is what I get:

"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_xz8NjzhqKiZA8UJkDR2KQvpd",
"type": "function",
"function": {
"name": "write_file",
"arguments": "{\n "filename": "out.py",\n "contents": "print('Hello, world!')"\n}"
}
}
]
},
"logprobs": null,
"finish_reason": "tool_calls"
}
]

I should have said this before, but the v3 version of this model is now recommended. I have updated the model card (which I should have before).

I have also done some testing on the v3 model with your functions and prompt. It works well.

Here is a formatted version of the conversation I had:

user: Write a python program to out.py to print hello world

function_call: {
    "name": "write_file",
    "arguments": {
        "filename": "out.py",
        "contents": "print('Hello, world')"
    }
}

function_response: File 'out.py' written successfully.

And here is the raw prompt I fed in (with functions). Note that this format is for the v3 model (not the v2 model here):

<｜begin▁of▁sentence｜>You have access to the following functions. Use them if required:

[
    {
        "type": "function",
        "function": {
            "name": "write_file",
            "description": "Write a file, creating it if necessary. If the file exists, it is overwritten.",
            "parameters": {
                "type": "object",
                "properties": {
                    "filename": {
                        "type": "string",
                        "description": "name of the file"
                    },
                    "contents": {
                        "type": "string",
                        "description": "contents of the file"
                    }
                },
                "required": [
                    "filename",
                    "contents"
                ]
            }
        }
    }
]

### Instruction:
Write a python program to out.py to print hello world
### Response:

add a note on chat_template