On the Cycle Consistency of Image-Text Mappings

Thank you for the insightful feedback and the helpful suggestions.

Q1. How does model choice affect the correlation between cycle consistency and downstream performance?

As requested, we report the Pearson correlation coefficient per model. The correlation is consistently strong for most models ($R^2 > 0.65$), except for BLIP2-2.7B and LLaVA-OV-0.5B with lower coefficients of 0.349 and 0.241, respectively. We attribute the low correlation to their use of small-scale, less-performant language models (OPT-2.7B, Qwen2-0.5B) as pre-trained backbones, which may cause poorer text reconstruction.

Furthermore, we have updated Figures 6, 7, 10 results to report cycle consistency averaged across all models, instead of just fixing one model in the pipeline. We also extend the analysis to include all four combinations, additionally comparing text quality (descriptiveness, hallucination) and image quality (prompt-following) with both image and text cycle consistency. We observe that both cycles exhibit a strong correlation across modalities, with text cycle consistency being more prominent.

Q2. Add more tasks (e.g., VQA) and check if cycle consistency can be a proxy of task accuracy.

As requested, we report correlation between cycle consistency and VQA performance. Note that this is an evaluation of model performance, whereas VQA without V measures the descriptiveness of text descriptions. We evaluate VQA performance on MMBench and MME, with MME divided into perception and cognition categories. Cycle consistency is computed on the sDCI dataset and averaged across five different text-to-image models with 3 different random seeds. The table below reports the Pearson correlation coefficient ($R^2$) between VQA scores and cycle consistency. Image cycle consistency strongly correlates with MMBench and MME (cognition), but weaker with other benchmarks. This is somewhat surprising as MME (perception) shows a strong correlation with cycle consistency in our VQA without V experiment. This discrepancy likely stems from the differences in these tasks: VQA evaluates the model’s ability to answer diverse questions about an image, while VQA without V evaluates whether the text descriptions include detailed answers to those questions. We have also added the correlation plots to the Appendix as Figure 18.

Q3. How Section 5 takeaways can be used for model training.

It is an interesting question how to incorporate the results of Section 5 (sensitivity to prompt style) for better model training. Because image-text mappings are one-to-many (or even many-to-many), some variance is expected and even desirable for sampling over different outputs. The degree to which two text prompts are equivalent could be even a model design choice if introduced as a type of augmentation during training, so that different sentences with the same sentiment map to the same images (this is similar to image augmentation for vision models).

Note that we have extended our analysis to study how random seed selection, prompt and caption style, and temperature sampling contribute to variance in cycle consistency (updated Section 5). Table 2 shows that image-to-text models exhibit higher variance due to temperature sampling but remain relatively robust to changes in prompt style. In contrast, text-to-image models are significantly more sensitive to caption style than to random seed sampling.

Q4. Using a strong MLLM for measuring captioning performance.

We clarify that we use a strong LLM for measuring captioning performance instead of CIDEr score. We use a VQA Benchmark dataset (such as MME, MMStar) and ask each image-to-text model to produce descriptions for the images in the benchmark dataset. Then, we ask an LLM to answer the VQA question based on the output text descriptions (instead of the images, which would be traditional VQA). This measure for captioning performance has several advantages over CIDEr scores. Firstly, CIDEr scores compare caption outputs against human annotated captions, which are often short and lack detail. Secondly, we are able to determine what kind of information are included in different captions, and what aspects image-to-text models are better at describing than others (based on the VQA sub-categories). Figure 7 demonstrates that cycle consistency strongly correlates with descriptiveness in the generated captions.

Q5. What do the authors think about building unified image-to-text and text-to-image models while enforcing cycle consistency?

Based on our findings, along with evidence of existing high-performing models which already incorporate cycle consistency in training [1-4], we believe that enforcing cycle consistency can be a helpful training objective. For text-to-image models, DALLE-3 [1] and SD3 [2] are trained on descriptive captions generated by an image captioning model. This process can be formalized as $\text{argmin}_G \ L(I, G(F(I)))$ where $I$ is the input image, $F$ is the image-to-text model, and $G$ is the text-to-image model. Training a text-to-image model on synthetic captions is equivalent to enforcing image cycle consistency relative to the fixed image captioner. For vision-language models (VLMs), Synth2 [3] trains a VLM using data from a pre-trained text-to-image model, while ITIT [4] jointly trains image-to-text and text-to-image models to be cycle-consistent. By injecting cycle consistency during training, both Synth2 and ITIT achieve high performance with significantly fewer data examples than state-of-the-art models. We have included this discussion in the introduction (Section 1) to further motivate our analysis of cycle consistency in current models.

[1] Improving image generation with better captions. Betker et al., 2023.

[2] Scaling rectified flow transformers for high-resolution image synthesis. Esser et al., ICML 2024.

[3] Synth2: Boosting visual-language models with synthetic captions and image embeddings. Sharifzadeh et al., 2024.

[4] Leveraging unpaired data for vision-language generative models via cycle consistency. Li et al., ICLR 2024.

Q3. Baselines for cycle consistency.

We add the following baselines and comparisons in Figure 16:

1. For image cycle consistency, we compare against reconstructing an image given human annotated text. For this baseline, each image in the DCI dataset is paired with a “short caption” provided by a human annotator. We compare $\text{DreamSim}(I, G(T_\text{human}))$ with $\text{DreamSim}(I, G(F(I))$, comparing human text $T_\text{human}$ against generated text F(I), where $G$ is the image-to-text model and $G$ is the text-to-image model. Figure 16 shows that synthetic text surpasses human text beyond a certain point due to superior descriptiveness, highlighting its effectiveness as a substitute for human text in training large models.
2. For text cycle consistency, we compare against reconstructing text given a natural image. Specifically, we compare $\text{SBERT}(T, F(I))$ with $\text{SBERT}(T, F(G(T))$, where $I$ is the real image paired with the input text $T$. Figure 16 shows that synthetic images achieve better text cycle consistency compared to real images. Figure 17 visualizes text cycle consistency from a real image vs. synthetic image. Compared to real images containing more complex details, synthetic images only generate details described in the input text which occupy larger areas of the generated image. Therefore, such details are easier to reconstruct for the image-to-text model, resulting in better text reconstruction.

Q4. Analysis of the divergence of image-to-text models.

As mentioned in Q2, the updated Section 5 studies the variance in cycle consistency (formerly called divergence). Specifically, analysis on how 1) prompt style and 2) temperature sampling affects variance in cycle consistency relates to variance from image-to-text models.

[1] Cyclegan, a master of steganography, Chu et al., NIPS “Machine Deception” Workshop, 2017.

[2] A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions, Urbanek et al., CVPR 2024.

Thank you for the insightful feedback and the helpful suggestions.

Q1. It is questionable that satisfying cycle consistency guarantees better performance.

Thank you for pointing this out and we will clarify this in the paper. We agree that cycle consistency is likely a consequence of better model capabilities – updated Section 3 analyzes contributing factors for enhanced cycle consistency. We are not suggesting that cycle consistency is necessarily a contributing factor to performance, unless models are known to incorporate cycle consistency during their training (e.g., SD3 and DALLE-3).

On the other hand, it's still questionable that satisfying cycle consistency guarantees better T2I&I2T performance. (e.g. A perfect Image->Text->Image reconstruction can be achieved if the I2T model writes down all pixel values in one long string. A perfect Text->Image->Text reconstruction can be achieved if the T2I model produces the image that contains the entire input text visually.)

While we agree that there are trivial solutions to the I2T2I and T2I2T cycles, we do not encounter a failure mode to this degree, since it is unlikely that models are trained with these kind of examples (i.e., outputting pixel values in text or generating text descriptions in pixels). Instead, we demonstrate that higher cycle consistency is generally achieved by higher-quality generated text and images, i.e., more descriptive text with less hallucinations and images generated faithfully to the text and containing fine-grained details (see updated Section 4). Apart from the rare scenario suggested by the reviewer, we demonstrate in Section 5 that cycle consistency is sensitive to subtle perturbations in language (i.e., prompt style). Additionally, Section 4.2 (Figure 9) shows instances where high image cycle consistency is obtained using short captions lacking details. While such cases are not uncommon for image cycle consistency, we do not observe similar cases in text cycle consistency.

Q2. How divergence and sensitivity analysis relate to cycle consistency. Fewer tags don’t necessarily contain less information.

We agree that previous Section 4 (divergence) lacks proper control of caption length and information - instead it varies the number of elements described. Furthermore, Section 5 (sensitivity) combined changes in length and style during prompt rewriting, making it difficult to isolate their respective impacts on cycle consistency. To address these shortcomings and better relate Sections 4 and 5 to cycle consistency, we make the following changes:

1. Updated Section 4.4 studies cycle consistency as a function of caption length. Captions from the Densely Captioned Images dataset [2] are summarized into varying lengths (5, 10, 20, 30, and 50 words) using LLaMA3-8B-Instruct. Figure 12 shows that cycle consistency improves as captions become more descriptive, especially for the higher performing models FLUX-Time and SD3.
2. Updated Section 5 studies the variance in cycle consistency (formerly called divergence). Unlike the previous experiment, we address the effect of caption length separately in Section 4.4, and focus on sources of variance in this section. Specifically, we analyze how random seed selection, prompt style, and temperature sampling contribute to this variance. Table 2 shows that image-to-text models exhibit higher variance due to temperature sampling but remain relatively robust to changes in prompt style. In contrast, text-to-image models are significantly more sensitive to prompt style than to random seed sampling.
3. Congruent with the tags experiment, we report diversity as a function of caption density, measured by DreamSim pairwise distance between generated images using 10 different random seeds for the same caption. However, we observe inconsistent trends across the text-to-image models (see updated Figure 19). This discrepancy likely stems from differences in the experiment design: previously Section 4 used hierarchically created captions with tags, often altering meaning by introducing new elements, while the caption density experiment used summarized versions of the same caption with varying levels of detail. By focusing on caption density rather than tags, we aim to better understand how caption descriptiveness influences cycle consistency.

Thank you for the insightful feedback and the helpful suggestions.

Q1. Analysis is hand-wavy for Table 1 and 2.

As suggested by Reviewers go1x, QcCd, and 31TL, we have included an analysis of factors contributing to cycle consistency in the updated Section 3. The main findings are highlighted as follows:

1. Cycle consistency improves with LLM scale. An image-to-text model consists of a vision encoder, a projector, and a large language model (LLM). Scaling the vision transformer (ViT) for the vision encoder is reported to enhance performance [1], yet a simple MLP projection remains the dominant approach [2-5]. Since no model provides open-sourced weights with varying vision encoder scales, we focus our analysis on ablating the scale of the LLM. Figure 4 demonstrates that scaling the LLM enhances image and text cycle consistency across all image-to-text model families. Figure 5 visualizes this effect—despite being trained on the same dataset and architecture, only InternVL2-40B successfully captures both the color and the presence of a corner turret.
2. Cycle consistency improves with re-captioned dataset quality. Table 1 demonstrates that the quality of the re-captioned dataset (e.g., dataset re-captioned by GPT-4V, LLaVA1.6-34B) plays an important role in improving image cycle consistency, often outperforming models trained on larger datasets annotated by less-performant models (e.g., BLIP). On the other hand, text cycle consistency shows little difference between the LLaVA models, as the input text from sDCI often lacks fine-grained detail (evidenced in Figure 16) compared to longer and more descriptive synthetic captions, such as those produced by LLaVA1.6 and LLaVA-OV. We believe higher-quality human annotations and text-to-image models with longer context would enhance the analysis of text cycle consistency. We exclude InternVL2 from this analysis as its pre-training dataset details are not disclosed. We detail differences in architecture, scale, and dataset in Table 1, 5, and 6.

Q2. Failure to address causality.

We agree that cycle consistency is likely a consequence of better model capabilities – updated Section 3 analyzes contributing factors for enhanced cycle consistency. We are not suggesting that cycle consistency is a contributing factor to performance.

However, this does not diminish the practical utility of cycle consistency as a tool to study model capabilities. Our goal is to show that cycle consistency is an emergent property in image-to-text and text-to-image models, and strongly correlates with various desired properties such as descriptiveness and reduced hallucination in text, and enhanced prompt-following in images (Section 4). Based on our findings, along with evidence of existing high-performing models which already incorporate cycle consistency in training [6-9], we believe that enforcing cycle consistency can be a helpful training objective for future research. For instance, text-to-image models such as DALLE-3 [6] and SD3 [7] report training on descriptive captions generated by an image captioning model. This process can be formalized as $\text{argmin}_G \ L(I, G(F(I)))$ where $I$ is the input image, $F$ is the image-to-text model and $G$ is the text-to-image model. Training a text-to-image model on synthetic captions is equivalent to enforcing image cycle consistency relative to the fixed image captioner. For vision-language models (VLMs), Synth2 [8] trains a VLM using data from a pretrained text-to-image model, while ITIT [9] jointly trains image-to-text and text-to-image models to be cycle-consistent. By injecting cycle consistency during training, both Synth2 and ITIT achieve high performance with significantly fewer data examples than state-of-the-art models.

Q3. The claim that "more descriptive text leads to a broader distribution of generated images" is not convincing.

We agree that the existing experiment does not properly control caption length – instead it varies the number of elements described. To address this issue, we make the following changes:

1. We replace Section 4 with a new experiment that studies cycle consistency as a function of caption density (see updated Section 4.4). We better control the amount of information by summarizing captions from the DCI dataset [10] into varying lengths (5, 10, 20, 30, and 50 words) using LLaMA3-8B-Instruct. We find that cycle consistency improves as captions become more descriptive and dense, especially for the higher performing models FLUX-Time and SD3.

2. Congruent with the tags experiment, we report diversity as a function of caption density, measured by DreamSim pairwise distance between generated images using 10 different random seeds for the same caption. However, we observe inconsistent trends across the text-to-image models (see updated Figure 19). This discrepancy likely stems from differences in the experiment design: previous Section 4 used hierarchically created captions with tags, often altering meaning by introducing new elements, while the caption density experiment used summarized versions of the same caption with varying levels of detail. By focusing on caption density rather than tags, we aim to better understand how caption descriptiveness influences cycle consistency.

Q4. Unaddressed claims in the abstract.

Thank you for pointing these claims out and we intend to clarify them both here and in the updated abstract:

1. Analyze failure cases: We provide examples of failure cases of cycle consistency in Figures 23 and 24. Failures include: synthetic images with artifacts or implausible generations but little effect on captions, descriptions of non-existent objects, endpoint model failures (i.e., the intermediate image or text representation is reasonable but the endpoint model creates inaccuracies which affect reconstruction). Many of these mistakes can be attributed to model error and usually affect text cycle consistency much more than image, mainly because images generated from incorrect captions often have lower cycle consistency, whereas image-to-text models do not always notice inaccuracies in synthetic images. There are also examples, typically for image cycle consistency, where information is not explicitly conveyed by the intermediate text, but the image reconstruction is nearly successful. To explain this, we can attribute cycle consistency as both a function of the intermediate representation and the input. For example, for image cycle consistency we find it is easy to reconstruct common or cliche images in very short captions. We show examples in Figure 9.
2. How descriptiveness affects achieving cycle consistency: We modify Section 4.4 to study caption density and cycle consistency, and find that more descriptive text positively influences cycle consistency as detailed in Q3.
3. There are no explorations of training cycle-consistent models in the paper: Our intent for Section 5 was to study variance of cycle consistency, as both image-to-text and text-to-image mappings can be stochastic due to random seed choice, temperature sampling, and prompt wording. We apologize for causing confusion and we have updated the abstract for clarity.

Q5. Is image-text cycle consistency a meaningful metric for model development?

Yes, we believe that image-text cycle consistency is a meaningful metric for model development. In the update paper, we show results indicating that more informative and descriptive captions correlate with cycle consistency, and similarly for generated images which are informative and faithful to their input text prompts. Qualitatively we highlight examples where better cycle consistency aligns with more preservation of information in Figures 2, 3, and 8. We also provide evidence of existing models that incorporate cycle consistency in training [6-9] in Q2.

[1] Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. PMLR 2023.

[2] Visual instruction tuning. NeurIPS 2023.

[3] Improved baselines with visual instruction tuning. 2023.

[4] Internvl-2.0. OpenGVLab, 2024. https://internvl.github.io/blog/2024-07-02-InternVL-2.0/

[5] Llava-onevision: Easy visual task transfer. 2024.

[6] Improving image generation with better captions. 2023.

[7] Scaling rectified flow transformers for high-resolution image synthesis. ICML 2024.

[8] Synth2: Boosting visual-language models with synthetic captions and image embeddings. Sharifzadeh et al., 2024.

[9] Leveraging unpaired data for vision-language generative models via cycle consistency. Li et al., ICLR 2024.

[10] A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions, CVPR 2024.

Q3. Model ablation for measuring cycle consistency.

As suggested, we update Figure 6, 7, and 10 to report cycle consistency averaged across all models, instead of just fixing one model in the pipeline. We also extend the analysis to include all four combinations, additionally comparing text quality (descriptiveness, hallucination) and image quality (prompt-following) with both image and text cycle consistency. We observe that both cycles exhibit a strong correlation across modalities, with text cycle consistency being more prominent.

As requested, we report the Pearson correlation coefficient per model. The correlation is consistently strong for most models ($R^2 > 0.65$), except for BLIP2-2.7B and LLaVA-OV-0.5B with lower coefficients of 0.349 and 0.241, respectively. We attribute the low correlation to their use of small-scale, less-performant language models (OPT-2.7B, Qwen2-0.5B) as pre-trained backbones, which may cause poorer text reconstruction.

Q4. Does cycle consistency correlate with data quality?

Yes! Thanks to your valuable suggestions, we have discovered that the quality of the re-captioned dataset is associated withcycle consistency (mentioned in Q1). Table 1 details the re-captioned dataset for each model and their cycle consistency.

[1] Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. Li et al., PMLR 2023.

[2] Visual instruction tuning. Liu et al., NeurIPS 2023.

[3] Improved baselines with visual instruction tuning. Liu et al., 2023.

[4] Internvl-2.0. OpenGVLab Team, 2024. https://internvl.github.io/blog/2024-07-02-InternVL-2.0/

[5] Llava-onevision: Easy visual task transfer. Li et al., 2024.

[6] A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions, Urbanek et al., CVPR 2024.

Thank you for the insightful feedback and the helpful suggestions.

Q1. What factors contribute to the observed differences in cycle consistency?

As suggested by Reviewers go1x, QcCd, and 31TL, we have included an analysis of factors contributing to cycle consistency in the updated Section 3. The main findings are highlighted as follows:

1. Cycle consistency improves with LLM scale. An image-to-text model consists of a vision encoder, a projector, and a large language model (LLM). Scaling the vision transformer (ViT) for the vision encoder is reported to enhance performance [1], yet a simple MLP projection remains the dominant approach [2-5]. Since no model provides open-sourced weights with varying vision encoder scales, we focus our analysis on ablating the scale of the LLM. Figure 4 demonstrates that scaling the LLM enhances image and text cycle consistency across all image-to-text model families. Figure 5 visualizes this effect—despite being trained on the same dataset and architecture, only InternVL2-40B successfully captures both the color and the presence of a corner turret.

2. Cycle consistency improves with re-captioned dataset quality. Table 1 demonstrates that the quality of the re-captioned dataset (e.g., dataset re-captioned by GPT-4V, LLaVA1.6-34B) plays an important role in improving image cycle consistency, often outperforming models trained on larger datasets annotated by less-performant models (e.g., BLIP). On the other hand, text cycle consistency shows little difference between the LLaVA models, as the input text from sDCI often lacks fine-grained detail (evidenced in Figure 16) compared to longer and more descriptive synthetic captions, such as those produced by LLaVA1.6 and LLaVA-OV. We believe higher-quality human annotations and text-to-image models with longer context would enhance the analysis of text cycle consistency. We exclude InternVL2 from this analysis as its pre-training dataset details are not disclosed. We find minimal differences in objective functions across models: most image-to-text models use visual instruction tuning with auto-regressive objectives except BLIP2, and most text-to-image models are LDMs, except Stable Diffusion 3 with rectified flow. Therefore, we exclude analysis of the objective function. As suggested by the reviewer, we detail differences in architecture, scale, and dataset in Table 1, 5, and 6.

Q2. It is unclear how Section 4 and 5 relate to cycle consistency.

Previously Section 4 lacked proper control of caption length (mentioned by 31TL, op89) and Section 5 combined changes in length and style during caption rewriting, making it difficult to isolate their respective impacts on cycle consistency. To address these shortcomings and better relate Sections 4 and 5 to cycle consistency, we make the following changes:

1. Updated Section 4.4 studies cycle consistency as a function of caption length. Captions from the Densely Captioned Images dataset [6] are summarized into varying lengths (5, 10, 20, 30, and 50 words) using LLaMA3-8B-Instruct. Figure 12 shows that cycle consistency improves as captions become more descriptive, especially for the higher performing models FLUX-Time and SD3.
2. Updated Section 5 studies the variance in cycle consistency (formerly called sensitivity). Unlike the previous experiment, we address the effect of caption length separately in Section 4.4, and focus on sources of variance in this section. Specifically, we analyze how random seed selection, prompt style, and temperature sampling contribute to this variance. Table 2 shows that image-to-text models exhibit higher variance due to temperature sampling but remain relatively robust to changes in prompt style. In contrast, text-to-image models are significantly more sensitive to prompt style than to random seed sampling. Note that we excluded InternVL2-40B from measuring cycle consistency due to lack of compute, and we will add it to the final manuscript.
3. Congruent with the tags experiment, updated Figure19 shows diversity as a function of caption density, measured by DreamSim pairwise distance between generated images using 10 different random seeds for the same caption. However, we observe inconsistent trends across the text-to-image models. This discrepancy likely stems from differences in the experiment design: previously Section 4 used hierarchically created captions with tags, which altered meaning by introducing new elements. Instead, the caption density experiment uses summarized versions of the same caption with varying levels of detail. By focusing on caption density rather than tags, we aim to better understand how caption descriptiveness influences cycle consistency.

Thank you for the insightful review and the helpful feedback.

Q1. What factors affect cycle consistency?

As suggested by Reviewers go1x, QcCd, and 31TL, we have included an analysis of factors contributing to cycle consistency in the updated Section 3. The main findings are highlighted as follows:

1. Cycle consistency improves with LLM scale. An image-to-text model consists of a vision encoder, a projector, and a large language model (LLM). Scaling the vision transformer (ViT) for the vision encoder is reported to enhance performance [1], yet a simple MLP projection remains the dominant approach [2-5]. Since no model provides open-sourced weights with varying vision encoder scales, we focus our analysis on ablating the scale of the LLM. Figure 4 demonstrates that scaling the LLM enhances image and text cycle consistency across all image-to-text model families. Figure 5 visualizes this effect—despite being trained on the same dataset and architecture, only InternVL2-40B successfully captures both the color and the presence of a corner turret.
2. Cycle consistency improves with re-captioned dataset quality. Table 1 demonstrates that the quality of the re-captioned dataset (e.g., dataset re-captioned by GPT-4V, LLaVA1.6-34B) plays an important role in improving image cycle consistency, often outperforming models trained on larger datasets annotated by less-performant models (e.g., BLIP). On the other hand, text cycle consistency shows little difference between the LLaVA models, as the input text from sDCI often lacks fine-grained detail (evidenced in Figure 16) compared to longer and more descriptive synthetic captions, such as those produced by LLaVA1.6 and LLaVA-OV. We believe higher-quality human annotations and text-to-image models with longer context would enhance the analysis of text cycle consistency. We exclude InternVL2 from this analysis as its pre-training dataset details are not disclosed. We detail differences in architecture, scale, and dataset in Tables 1, 5, 6.

Q2. Why specific combinations of I2T and T2I models perform differently in image and text cycle consistency.

Most of the model combinations perform similarly on our updated results using the Densely Captioned Images dataset. However, there are exceptions: LLaVA1.5-13B achieves higher text cycle consistency than image cycle consistency relative to the other VLMs, whereas LLaVA-OV-7B has a higher consistency on image over text, as shown in Figure 14,15 (Appendix). We think this may be due to several reasons: LLaVA1.5 captions tend to point out less specific details, whereas other models try to pinpoint exact locations or meanings of photographs. Of course this is desirable for describing real images, but leads to variability when reconstructing text from synthetic images.. Secondly, LLaVA1.5 uses Vicuna-1.5 as its LLM which is a finetuned version of Llama 2. The sDCI captions which we use as our text inputs, are summaries of the full captions from DCI created by Llama2. Because the input text and output texts are created by similar models, it is likely that this contributes to their high alignment scores.

Q3. The paper does not propose solutions for prompt sensitivity.

We clarify that we are not proposing a new method, , but aim to provide an empirical study on cycle consistency in image-text mappings.

As suggested by Reviewers QcCd and uoP89, we extend our analysis to study how random seed selection, prompt and caption style, and temperature sampling contribute to variance in cycle consistency (updated Section 5). Table 2 shows that image-to-text models exhibit higher variance due to temperature sampling but remain relatively robust to changes in prompt style. In contrast, text-to-image models are significantly more sensitive to caption style than to random seed sampling. Note that we excluded InternVL2-40B due to lack of compute, and we will add it to the final manuscript.

Q4. MS COCO captions often lack detailed descriptions of the images.

We agree with the reviewer’s suggestion and have replaced MS COCO with Densely Captioned Images (DCI) dataset [6], which features higher-resolution images annotated with more dense captions. Due to the limited prompt length of text-to-image models, we use sDCI, i.e., LLM-summarized DCI captions to fit 77 tokens, and sample 1K from the train split. Average image resolution and number of CLIP tokens per caption are as follows:

Improving dataset quality revealed key factors influencing cycle consistency (updated Section 3.2). We thank the reviewer for the insightful suggestion.