Tool mistook long form content for a question

[timpaul]

Here's a fun one. The tool took the contents of the 'Documents we need' section on this page and turned it into a multiple choice question!

Then, it took the content at the start of the next section, 'What you need to do now', and turned it into an address question:

Finally, it took the content at the end of that section and made a text question:

[timpaul]

I tried adding the following to the tool description:

• pages: An array of the questions in the form. A question typically consists of a question title, some optional hint text and one or more form fields. Form fields typically look like boxes.

It did this!:

So, it took the first sentence on the page as the question title, then used the checkboxes from the declaration panel as the question fields. There were no other questions generated.

[timpaul]

I isolated the first part of the form that generates the error:

Now it generates 3 pages, one for each bullet.

I think this might reveal what is happening. Claude is returning the most question-like things it can see in the image, evcen if they are not very question-like at all. It doesn't know how to say "There are no questions here".

[timpaul]

I've somewhat addressed this now:

This commit adds the ability to recognise if an image isn't a form, by (amongst other things) changing the prompt text to this:

"Is this a form? It's only a form if it contains form field boxes. If it is a form, extract the questions from it using the extract_form_questions tool."

[timpaul]

OK, now it does this:

(It gets the signature and date too).

It's interesting - it's not a good replication of the original form, but it DOES work - in that it gets the user to make a series of explicit declarations.

[timpaul]

Closing this for now, until new examples turn up