AI Assist - Generate Image Description Behavior Issue

[raldea]

Found a behavior issue with the native AI Assist - Generate Image Description. When initially updating the field while it's empty, the AI Assist is working correctly. Once I publish and click the Generate Image Description - AI Assist again, an error appears. It only works again when you empty the alt field.

Steps to reproduce the behavior:

1. In your Sanity Intance, AI Assist should be enabled
2. In a document add the following code field

 defineField({
      name: 'featuredImage',
      title: 'Featured Image',
      type: 'image',
      group: 'editorial',
      fields: [
        defineField({
          type: 'string',
          name: 'alt',
          title: 'Alternative text',
        }),
        defineField({
          type: 'url',
          name: 'href_url',
          title: 'Image Link',
          validation: Rule => Rule.uri({
            scheme: ['http', 'https'],
          }),
        }),
      ],
      options: {
        hotspot: true,
        aiAssist: {
          imageDescriptionField: 'alt',
        },
      },
    }),

3. Upload an image and the initial AI Image Description generation should be executed automatically
4. Publish the document
5. Hover back to the alt field and click AI Assist > Generate Image Description
6. An error will occur
7. Empty the alt field and go back to step 5 again, it should work

Issue is in step 6

Screenshots
If applicable, add screenshots to help explain your problem.

ai-assist-issue.mp4

Which versions of Sanity are you using?

@sanity/cli (global) 5.11.0 (up to date)
@sanity/asset-utils 2.3.0 (up to date)
@sanity/assist 5.0.4 (up to date)
@sanity/cli 5.11.0 (up to date)
@sanity/color-input 6.0.3 (latest: 6.0.4)
@sanity/document-internationalization 4.1.1 (latest: 5.0.2)
@sanity/eslint-config-studio 6.0.0 (up to date)
@sanity/icons 3.7.4 (up to date)
@sanity/table 2.0.1 (up to date)
@sanity/ui 3.1.11 (up to date)
@sanity/vision 5.11.0 (up to date)
sanity 5.11.0 (up to date)

What operating system are you using?
Macos 15.1.1

Which versions of Node.js / npm are you running?
npm -v && node -v
10.9.2
v22.13.0

[pedrobonamin]

Hello @raldea thank you for reporting this issue.
I think AI assist is trying to patch the draft document but the draft doesn't exist yet because it was just published. It looks like we need to update the backend that handles this to do a different type of mutation.
Will report once this is updated, thank you for the great details on this report.

1. Draft was created.
2. Edit from AI assist works because it targets the draft.
3. Draft is published, published is created and draft deleted.
4. Edit from AI assist fails in the draft document.