Currently, the Gemini adapter can encounter issues when processing streams in :json_schema mode and when dealing with Ecto's generic :map types.
1. Stream Processing for :json_schema Mode:
When using stream: true with mode: :json_schema for the Gemini adapter, the Instructor.Adapters.Gemini.parse_stream_chunk_for_mode/2 function can raise a FunctionClauseError. This occurs because the stream from Gemini includes metadata or terminal chunks (e.g., indicating finishReason: "STOP") that do not match the expected data-bearing structure (candidates -> content -> parts -> text).
A more robust handling mechanism is needed to gracefully ignore these non-data chunks, allowing the stream processing to complete successfully.
Example Error (before fix):
** (FunctionClauseError) no function clause matching in Instructor.Adapters.Gemini.parse_stream_chunk_for_mode/2
The following arguments were given to Instructor.Adapters.Gemini.parse_stream_chunk_for_mode/2:
# 1
:json_schema
# 2
%{"candidates" => [%{"content" => %{"role" => "model"}, "finishReason" => "STOP", ...}]}
2. :map Type Incompatibility with Gemini json_schema Mode:
Gemini's responseSchema in json_schema mode has strict requirements for object definitions. Specifically, it appears that:
* It does not support additionalProperties for defining generic maps with arbitrary keys.
* It requires that any nested object defined within the schema must have a non-empty properties field.
This creates an incompatibility with Ecto's generic :map and {:map, type} types, which naturally translate to JSON schema objects that either use additionalProperties or might have an empty properties map (e.g., {"type": "object", "properties": {}}).
The Instructor.Adapters.Gemini.normalize_json_schema/1 function currently includes a client-side raise to prevent sending schemas with empty properties for nested objects, as these are rejected by the Gemini API. This raise is informative:
Invalid JSON Schema: object with no properties at path: [...]. Gemini does not support empty objects. This is likely because it uses a naked :map type without any fields at [...]. Try switching to an embedded schema instead.
Expected Behavior / Solution:
- For stream processing, the Gemini adapter should add a fallback clause to
parse_stream_chunk_for_mode/2 for :json_schema mode to return an empty string for unmatchable/non-data chunks.
- The client-side
raise in normalize_json_schema for empty properties should be maintained as it correctly identifies an incompatibility and guides the user.
- Documentation for the Gemini adapter should clearly state the limitation regarding generic
:map types in :json_schema mode and recommend using Ecto's embeds_one/embeds_many with explicit schemas for nested objects.
- The test suite should be updated to use
embeds_one/embeds_many for map-like structures specifically when testing the Gemini adapter in json_schema mode, while allowing other adapters to be tested with generic :map types if they support them.
Impact:
These changes will improve the reliability of streaming with Gemini in json_schema mode and provide clearer guidance to users about schema design for compatibility with Gemini's structured output features.
Currently, the Gemini adapter can encounter issues when processing streams in
:json_schemamode and when dealing with Ecto's generic:maptypes.1. Stream Processing for
:json_schemaMode:When using
stream: truewithmode: :json_schemafor the Gemini adapter, theInstructor.Adapters.Gemini.parse_stream_chunk_for_mode/2function can raise aFunctionClauseError. This occurs because the stream from Gemini includes metadata or terminal chunks (e.g., indicatingfinishReason: "STOP") that do not match the expected data-bearing structure (candidates -> content -> parts -> text).A more robust handling mechanism is needed to gracefully ignore these non-data chunks, allowing the stream processing to complete successfully.
Example Error (before fix):
2.
:mapType Incompatibility with Geminijson_schemaMode:Gemini's
responseSchemainjson_schemamode has strict requirements for object definitions. Specifically, it appears that:* It does not support
additionalPropertiesfor defining generic maps with arbitrary keys.* It requires that any nested object defined within the schema must have a non-empty
propertiesfield.This creates an incompatibility with Ecto's generic
:mapand{:map, type}types, which naturally translate to JSON schema objects that either useadditionalPropertiesor might have an emptypropertiesmap (e.g.,{"type": "object", "properties": {}}).The
Instructor.Adapters.Gemini.normalize_json_schema/1function currently includes a client-sideraiseto prevent sending schemas with emptypropertiesfor nested objects, as these are rejected by the Gemini API. Thisraiseis informative:Expected Behavior / Solution:
parse_stream_chunk_for_mode/2for:json_schemamode to return an empty string for unmatchable/non-data chunks.raiseinnormalize_json_schemafor emptypropertiesshould be maintained as it correctly identifies an incompatibility and guides the user.:maptypes in:json_schemamode and recommend using Ecto'sembeds_one/embeds_manywith explicit schemas for nested objects.embeds_one/embeds_manyfor map-like structures specifically when testing the Gemini adapter injson_schemamode, while allowing other adapters to be tested with generic:maptypes if they support them.Impact:
These changes will improve the reliability of streaming with Gemini in
json_schemamode and provide clearer guidance to users about schema design for compatibility with Gemini's structured output features.