About cogvlm

[jifeng35]

Hello, I'm using your VIE score to assess my own diffusion model, it helps me a lot.
But when I turn to use cogvlm to test my model, I found it does not follow the prompt at all.
Here is the output of the cogvlm:

result_SC: {
    "output": [
    { "score": 9, "reasoning": "The edited image has successfully changed the double poneytail to single poneytails, maintaining the character's overall appearance." },
    { "-": "There is no significant difference between the original and edited images." }
    ]
    }
result_PQ: The image has a high level of naturalness with a score of 9. The details, lighting, and shadows are well-rendered, giving a realistic feel to the scene. As for artifacts, there are minimal distortions, and the image appears to be of high quality.

So my opinion is removing this model since it cannot follow the prompt!
😩

[vinesmsuic]

Hi, thanks for the comment! Yes, older models like cogvlm cannot follow the prompt properly, as stated in the paper. I suggest using some more recent model or even Gemini 2.5 Pro (you can integrate it into vllm, should be pretty easy to do so).