Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ChatCompletionsToolCall missing index field #43611

Open
3 tasks
anuraaga opened this issue Dec 26, 2024 · 0 comments
Open
3 tasks

[BUG] ChatCompletionsToolCall missing index field #43611

anuraaga opened this issue Dec 26, 2024 · 0 comments
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@anuraaga
Copy link

anuraaga commented Dec 26, 2024

Describe the bug

OpenAI response for a tool call contains a field index. This is particularly important for streaming tool call requests because the tool calls are returned individually within chunks. Without the index, it is not clear which tool call a particular chunk is referring to. While currently it seems OpenAI only returns id in the first chunk of a tool call and that could be used as a delimiter, I think that behavior could change in the future and isn't intended to be how we delimit tool calls in streaming.

JSON response:

{
  "id" : "chatcmpl-AiWm43aBO7oeXWZ3bOGNh48zGSfDN",
  "object" : "chat.completion.chunk",
  "created" : 1735175780,
  "model" : "gpt-4o-mini-2024-07-18",
  "system_fingerprint" : "fp_0aa8d3e20b",
  "choices" : [ {
    "index" : 0,
    "delta" : {
      "tool_calls" : [ {
        "index" : 0,
        "id" : "call_c5QfAjfoKgdc8TLCZwCLqe75",
        "type" : "function",
        "function" : {
          "name" : "get_weather",
          "arguments" : ""
        }
      } ]
    },
    "logprobs" : null,
    "finish_reason" : null
  } ]
}

index field not parsed:
https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/models/ChatCompletionsFunctionToolCall.java#L71

Exception or Stack Trace
Add the exception log and stack trace if available

To Reproduce

Execute a streaming parallel tool call request. OpenAI docs only have one for streaming tool calls, and parallel tool calls, not parallel streaming tool calls but they can be combined to exercise the functionality.

https://platform.openai.com/docs/guides/function-calling#advanced-usage

Code Snippet
Add the code snippet that causes the issue.

    private static class StringField {
        private String type = "string";

        private String description;

        public StringField(String description) {
            this.description = description;
        }
    }

    private static class GetWeatherFunctionProperties {
        StringField
                location = new StringField("The location to get the current temperature for");
    }

    private static class GetWeatherFunctionParameterDefinition {
        private String type = "object";

        private List<String> required = List.of("location");

        private boolean additionalProperties = false;

        private GetWeatherFunctionProperties
                properties = new GetWeatherFunctionProperties();
    }

    void streamParallelToolsAndCaptureMessageContent(boolean useResponseApi) {
        FunctionDefinition functionDefinition = new FunctionDefinition("get_weather");
        GetWeatherFunctionParameterDefinition
                parameters = new GetWeatherFunctionParameterDefinition();
        functionDefinition.setParameters(BinaryData.fromObject(parameters));

        List<ChatRequestMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant providing weather updates."));
        chatMessages.add(new ChatRequestUserMessage("What is the weather in New York City and London?"));

        ChatCompletionsOptions options = new ChatCompletionsOptions(chatMessages)
                .setTools(List.of(new ChatCompletionsFunctionToolDefinition(functionDefinition)));


        IterableStream<ChatCompletions> chatCompletionsStream = openai.client.getChatCompletionsStream(TEST_CHAT_MODEL, options);

Expected behavior
A clear and concise description of what you expected to happen.

ChatCompletions.choices.delta.toolCalls have index populated

Issue is obvious from ChatCompletionsFunctionToolCall code so will leave out remaining.

Screenshots
If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

  • OS: [e.g. iOS]
  • IDE: [e.g. IntelliJ]
  • Library/Libraries: [e.g. com.azure:azure-core:1.16.0 (groupId:artifactId:version)]
  • Java version: [e.g. 8]
  • App Server/Environment: [e.g. Tomcat, WildFly, Azure Function, Apache Spark, Databricks, IDE plugin or anything special]
  • Frameworks: [e.g. Spring Boot, Micronaut, Quarkus, etc]

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

  • verbose dependency tree (mvn dependency:tree -Dverbose)
  • exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Bug Description Added
  • Repro Steps Added
  • Setup information Added
@github-actions github-actions bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Dec 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
Development

No branches or pull requests

1 participant