Structured Outputs not supporting meta-llama/Llama-3.2-3B-Instruct-Turbo

[antoniomtz]

Even thought it's stated that meta-llama/Llama-3.2-3B-Instruct-Turbo is supported for Structured Outputs, this code returns the following error:

 const extract = await together.chat.completions.create({
    messages: [
      {
        role: "system",
        content:
          "The following is a list of items from a menu. Only answer in JSON.",
      },
      {
        role: "user",
        content: menuItems!,
      },
    ],
    model: "meta-llama/Llama-3.2-3B-Instruct-Turbo",
    // @ts-expect-error - this is not typed in the API
    response_format: { type: "json_object", schema: jsonSchema },
  });

Error:

error: {
    id: 'npsvkwr-57nCBj-92f800485c83c4bf',
    error: {
      message: "Input validation error: grammar is not valid: grammar is not valid: Grammar must have a 'properties' field",
      type: 'invalid_request_error',
      param: null,
      code: null
    }
  }
}

The same code works with meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo.

[blainekasten]

Hey @antoniomtz sorry for the late response. I've tested this model today and JSON mode worked for me. Can you share what the content: menuItems is? Maybe something unique about the data you sent in.