v0 Model: Incorrect Image Message Format (Multimodal Support)

[Pterjudin]

v0 models (categorized under openAICompatible) fail to process images. The model reports it cannot see the message even though it advertises multimodal support.

This occurs because v0 falls back to the default model options, which set:

specialToolFormat: undefined

When specialToolFormat is undefined, CortexIDE routes messages through the Anthropic-style XML formatter:

prepareMessages_XML_tools()

This is not the format v0 expects, so the model cannot detect any attached image.

---

Root Cause

• v0 model names do not match any OpenAI recognition patterns in:

  extensiveModelOptionsFallback()
  

• Because of that, v0 falls back to:

  defaultModelOptions
  

  which sets:

  specialToolFormat: undefined
  

• undefined → routes to the Anthropic-style image format:

  {
    type: 'image',
    source: {
      type: 'base64',
      media_type: mediaType,
      data: base64String
    }
  }

• v0 does not use:

  • OpenAI-style:

    { type: 'image_url', image_url: { url: dataUrl } }

  • Anthropic-style:

    { type: 'input_image', source: {...} }

• As a result, v0 receives the wrong image payload, and the model cannot “see” the image.

---

Proposed Fix

1. Add a new v0-style formatter option

In specialToolFormat type definitions, add:

specialToolFormat?: 'openai-style' | 'anthropic-style' | 'gemini-style' | 'v0-style'

---

2. Create a dedicated formatter

Create a new function:

prepareMessages_v0_tools()

This formatter will generate the exact image structure v0 expects (to be confirmed after API testing).

---

3. Add v0 detection

Inside modelOptionsFallback() or extensiveModelOptionsFallback(), detect v0 model names:

if (lower.includes('v0')) {
  return {
    ...defaultModelOptions,
    recognizedModelName: 'v0',
    specialToolFormat: 'v0-style'
  }
}

---

4. Update message routing

In prepareOpenAIOrAnthropicMessages() add:

else if (specialToolFormat === 'v0-style') {
    llmChatMessages = prepareMessages_v0_tools(messages)
}

---

Files Involved

src/vs/workbench/contrib/cortexide/common/modelCapabilities.ts
src/vs/workbench/contrib/cortexide/browser/convertToLLMMessageService.ts
src/vs/workbench/contrib/cortexide/common/sendLLMMessageTypes.ts

(sendLLMMessageTypes.ts only if v0 needs a custom message type.)

---

Next Steps

1. Confirm v0’s exact expected image-messaging format

   • Via v0’s API documentation
   • Or by sending known OpenAI/Anthropic structures and comparing failures

2. Implement the formatter once the current VS Code stable migration is complete
   (Avoiding merge conflicts and workbench regressions.)

---

Priority

High — this blocks all multimodal (image) support for v0 models.