I used yairpatch's gguf mxfp4 version, found this model is very smart,but tool call fails a lot,what about original model?

#11
by tarjintor - opened

I can only use gguf so I can use memory+vram together

tarjintor changed discussion title from I used gguf mxfp4 version, found this model is very smart,but tool call fails a lot,what about original model? to I used yairpatch's gguf mxfp4 version, found this model is very smart,but tool call fails a lot,what about original model?
JD Open Source org

Hi, @tarjintor . Glad to hear your feedback. Would you provide more specific toolcall examples?

Hi, @tarjintor . Glad to hear your feedback. Would you provide more specific toolcall examples?

here is my prompts:

你好,现在有一些数据在sqlite的表中,sqlite数据库文件位置为/tmp/mydata.db,下面我们要一步步来交互式的探索,尝试分析这些模型的信息,请先查看一下表结构,不要进一步操作.你可以调用python解释器执行代码,这个解释器底层是jupyter kernel,多次执行代码之间的状态可以保存

here is my toolcall:

jupyter_tool_test = {
        "type": "function",        
        "name": "python",
        "description": f"""Call a Python interpreter with some Python code that will be ran.
Use this tool to execute Python code in your chain of thought. 
The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).
When you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment. 
python will respond with the output of the execution or time out after {TIMEOUT} seconds. 
Internet access for this session is UNKNOWN. Depends on the cluster.
""",
        "parameters": {
            "type": "object",
            "properties": {
                "code": {
                    "type": "string",
                    "description": "The Python code to run",
                },
            },
            "required": ["code"],
        },
    }

this works fine for model same size level like glm 4.7 flash,nvidia's Nemotron 30b,and larger models all works fine

and what I mean fail is the model didn't tool call immediately in this step,but need me to send "继续",then tool call in next step

Sign up or log in to comment