Anthropic tool calls and extraction fails for single arguments

[willbakst]

Description

It looks like when given a single argument it always returns the input as a string instead of as a single-key dictionary.

Ideally it would return the correct input as part of the tool call; however, if it isn't easy to force this then we can always just try parsing the returns string as json and dynamically grabbing the field name.

Python, Mirascope & OS Versions, related packages (not required)

mirascope=v0.13.3

[willbakst]

Looks like this is actually an issue with Anthropic tool calls not returning list as a list but as a str.

This isn't really fixable on our end since the item could be deeply nested and would require a full scan of the object. Better to request that Anthropic fix this.

anthropics/anthropic-sdk-python#507

[willbakst]

Update: we can get around this through prompt engineering.

Opus works out of the box, but for Sonnet and Haiku adding something like "MUST BE A LIST" seems to work.

[willbakst]

Update, this is something the Anthropic team is aware of but isn't something they can do except with updates to the model. Nothing we can do on our end to "fix" this unfortunately.

The near term solution on our end is to simply prompt engineer to get the result (i.e. add something like MUST BE A LIST). Not ideal, but works. Something the user should do, not something we should include in Mirascope (since it's dependent on the user's schema). We could try to prompt engineer this under the hood, but that would just add unknown/unseen tokens and likely not work as well as a more specific prompt since it would necessarily be general.