{{ message }}
Repository navigation
Chat and Tool discussion, all models supported #10973
Description
I don't know if it is a bug.
When i used this prompt :
[
{
"role": "user",
"content": "# SYSTEM INSTRUCTIONS\n{{smart_spec_skill}}\n# ASSOCIATED USER MESSAGE\n{{user_message}}"
},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": "read_file",
"arguments": { "path": ".smart.ai/config.yml" }
}
}
]
},
{
"role": "user",
"tool_call_id": "call_1",
"name": "read_file",
"content": "{{read_file_config_result}}"
}
]
My tets is ok with deepseek, but gemini try to call the tool (it does'nt undestand that it is already done)
To get working i need the following prompt dedicated to gemini:
[
{
"role": "user",
"parts": [
{
"text": "# SYSTEM INSTRUCTIONS\n{{smart_spec_skill}}\n# ASSOCIATED USER MESSAGE\n{{user_message}}"
}
]
},
{
"role": "model",
"parts": [
{
"functionCall": {
"id": "call-1",
"name": "read_file",
"args": { "path": ".smart.ai/config.yml" }
},
"thoughtSignature": "skip_thought_signature_validator"
}
]
},
{
"role": "user",
"parts": [
{
"functionResponse": {
"id": "call-1",
"name": "read_file",
"response": {
"result": "{{read_file_config_result}}"
}
}
}
]
}
]
Is it normal ?
Promptfoo does not manage the difference for the moment ?
Activity
ebukae05 commentedon Sep 21, 2026
yo i can look into it seems interesting.
fletort commentedon Sep 23, 2026
It is more a question/decision:
Is promptfoo wants to make an abstraction of each specific json (google, anthropic, openai), for exemple for discussion with tool call ?
Or not ?
We can have an automatic mapping to the Model (Prompt management) and also on the reply (response of the llm).
In my case I have a test that check if the model calls a specific tool.... I need to adapt all the assert to be ok with openai json et gemini json....
PS: if yes i can contribute... i have already a local "mapping" for this first case.