Difficulty in Generating Rationales Using blip2-flan-t5-xxl

[lky-violet]

Dear Author,

Thank you for open-sourcing your code and sharing your valuable work.

I have been reading your paper with great interest and attempting to replicate your results. According to the paper, you used blip2-flan-t5-xxl to generate the reasoning rationales. However, during my replication attempts, I found it challenging to generate similar intermediate rationale files.

I wanted to ask:

1. Is blip2-flan-t5-xxl indeed the correct model to use, or might there have been a different model involved?
2. Do specific prompt designs play a crucial role in obtaining the desired rationales?

If possible, could you kindly provide examples of the prompts you used? It would be incredibly helpful for reproducing the results more accurately.

Thank you for your time and for your contributions to the community. I look forward to your response.

[ZSZH2019]

Thank you for your interest in our work and for taking the time to replicate our results.
To address your questions:

1. Yes, blip2-flan-t5-xxl is used during the generation of reasoning rationales, specifically in the "Visual Recognition to Obtain Visual Complements" phase.
2. We follow the official prompts for blip-2 in QA tasks and do not design any special prompts tailored for our task.
   Here is an example of one of the prompts we use:
   Question: {sub-question} Answer:
   Additionally, please note that we do not directly input all the results of Negative-Space Prompting into blip-2. Only the sub-questions where the answer is Uncertain are independently input into blip-2 for further processing.

I hope this clarifies your concerns. Please feel free to reach out if you have any further questions!