LABridge: Text–Image Latent Alignment Framework via Mean-Conditioned OU Process

We sincerely thank the reviewer for their constructive feedback and insightful comments. We address the specific weaknesses below with additional clarifications and new experimental results.

---

1. On the Ablation of the OU Bridge vs. Other Processes

We appreciate the reviewer's sharp observation. This is a critical point that we will clarify and support with a more comprehensive ablation study in the revised manuscript.

Clarification of Contribution: Our primary contribution is the LABridge framework, which hinges on the Text-Image Alignment Encoder (TIAE) to generate structured, text-conditioned priors $\mu_T(y)$. The main benefit of our method stems from redefining the text-to-image generation task as a bridge from the image manifold to a semantically aligned prior manifold, rather than a generic Gaussian. The choice of the bridge (VP, VE, or OU) is the mechanism to traverse this path.

Theoretical Motivation for OU Bridge: As correctly noted, VP/VE processes can be seen as special cases of a Generalized OU (GOU) process where the mean-reversion parameter $\theta \to 0$. The key theoretical advantage of a general OU bridge ($\theta > 0$) is the explicit mean-reversion drift term, $\theta(\mu_T(y) - x_t)$. This term provides direct and continuous guidance towards the text-conditioned prior throughout the entire reverse sampling process. In contrast, standard VP/VE processes have a drift that pulls towards the origin, relying solely on the learned score network to correct the trajectory. The OU drift offers inherent stability and a stronger semantic pull, especially in the early, noisy stages of sampling where the score estimate can be less reliable. This is detailed in our theoretical analysis in Section D.4 and Theorem D.8.

New Experimental Ablation: To provide the quantitative evidence requested, we have conducted a new, more comprehensive ablation study directly comparing VP, VE, and OU bridges within our LABridge framework. We evaluated this on both the class-conditional ImageNet benchmark and the more complex open-domain COCO-10K benchmark to demonstrate broader applicability. The results, which we will add to the paper, are summarized below:

Table 1: Comprehensive Ablation of Bridge Types within the LABridge Framework

Analysis: These results provide strong empirical validation for our choice of the OU bridge.

1. Consistent Superiority: On both ImageNet and COCO, the OU bridge consistently outperforms the VP and VE variants across all relevant metrics. While the performance of VP is very strong (as noted by the reviewer in Table 1), the OU bridge provides an additional, measurable improvement.
2. Improved Alignment: On COCO-10K, the improvements in alignment-focused metrics like CLIP Score, GenEval, and DPG are particularly noteworthy. This supports our hypothesis that the explicit mean-reverting drift of the OU process enhances the semantic consistency between the text prompt and the generated image.

This evidence demonstrates that the OU bridge is not merely a theoretical construct but a practical and superior choice for implementing the LABridge framework.

2. On Hyperparameter-Sensitivity Analysis for the OU Bridge

To address the reviewer's concern in this rebuttal, we provide a preliminary sensitivity analysis for both the mean-reversion parameter $\theta$ and the volatility parameter $\sigma$. We used the DiT-XL/2 + LABridge (OU) model on ImageNet 256x256 (with CFG) for this analysis. (These parameters and results were obtained from our previous experiments)

Table 2: Sensitivity Analysis of OU Bridge Hyperparameters ($\theta$, $\sigma$) on ImageNet 256

Analysis: This study reveals that the model's performance is robust within a reasonable range for both $\theta$ and $\sigma$.

• For $\theta$, a very small value approaches the behavior of a standard VP/VE bridge, losing some of the guidance benefits. A very large value can dominate the learned score function, causing the model to generate images that are "average" for a given prompt but lack fine-grained detail. Our chosen value of $\theta=1.0$ represents a sweet spot.
• For $\sigma$, the performance also peaks around our chosen value of $1.0$. Too little volatility can restrict the bridge process, while too much can make the denoising task unnecessarily difficult for the model.

This analysis shows that while the parameters are impactful, there is a clear and robust optimal range, making the model practical for new applications without exhaustive tuning.

Conclusion

In summary, the OU bridge is not "novelty for novelty's sake" but a theoretically sound and empirically superior mechanism for realizing the full potential of our core TIAE-based framework. It provides stability and direct semantic guidance that other processes lack. We will add the new, comprehensive ablation studies and sensitivity analyses to the final manuscript to make these advantages clear.

We thank the reviewer again for their valuable feedback, which has significantly helped us strengthen our paper. We hope these clarifications and our plan for revision have adequately addressed the concerns.

We sincerely thank the reviewer for their valuable feedback and insightful questions. Below, we provide a more comprehensive response with expanded data to address each point in detail.

---

1. Resource Consumption of LABridge Alignment vs. Standard CLIP Usage

The core idea of LABridge is to invest a one-time, offline training cost for the Text-Image Alignment Encoder (TIAE) to gain significant, lasting improvements in semantic alignment, sample quality, and inference efficiency.

Detailed Cost-Benefit Analysis:

The TIAE module is a lightweight transformer-based network, inspired by DiT blocks. Its objective is to learn a direct mapping from a text embedding space ($E\_y$) to an image latent space ($\mu\_T(y)$). This is a simpler, more constrained task than training the entire diffusion U-Net, which must learn a complex denoising function across multiple timesteps.

• Upfront Training Cost: The TIAE is trained on a large-scale image-text dataset (e.g., a subset of COYO-700M as mentioned in our appendix). While this is not a trivial cost, it is an order of magnitude smaller than training the base diffusion model from scratch.
• Amortized Benefits: This one-time cost is amortized over the model's lifetime, yielding substantial returns:
  1. Superior Alignment: As shown in our main results (Table 2), LABridge consistently improves scores on alignment-focused benchmarks like GenEval and DPG, leading to images that better reflect the text prompt.
  2. Accelerated Inference: The structured prior and OU bridge dynamics provide a much better starting point and a more direct path for the reverse process. As demonstrated in Table 6, this reduces the required number of function evaluations (NFE) by 50% or more for equivalent image quality, directly cutting inference time and cost.

To make this explicit, we will add a more detailed computational cost analysis to the appendix. The table below provides a realistic estimate based on our experience and publicly available data on training large-scale models.

Proposed Addition (Appendix C.5: Computational Cost Analysis):

Table 1: Estimated Computational Cost Comparison (Based on DiT-XL Architecture)

This detailed breakdown formally illustrates that the small, one-time investment in training the TIAE is a highly effective strategy for unlocking substantial and recurring gains in both model performance and computational efficiency at inference time.

---

2. Scalability and Significance of Improvements

Significance of Improvements:

While some individual metric improvements may appear modest, their consistency across multiple state-of-the-art architectures (SD1.5, SDXL, SD3.5, PixArt) and diverse benchmarks demonstrates the robustness and general applicability of our method. Crucially, LABridge delivers targeted improvements in text-vision alignment (GenEval, DPG) while simultaneously enhancing or maintaining image fidelity (FID, CLIP Score).

Scalability Analysis:

To directly address the reviewer's question, we have expanded our analysis to show how the benefits of LABridge scale with the base model's capacity. The results below, compiled from our main paper and supplemented with additional metrics for a complete picture, show that the performance gains not only persist but are often amplified on larger models.

Proposed Addition (Appendix C.6: Scalability Analysis of LABridge on SD3.5 Models)

Table Y: Performance Gains of LABridge Across Model Scales

This expanded analysis reveals a clear and compelling trend:

1. Consistent Improvement: LABridge delivers substantial performance improvements across both model scales (Medium and Large) and both evaluation datasets (COCO and MJHQ).
2. Strong Scalability: The absolute improvement in FID score is maintained or even enhanced when moving from the 1B to the 2B parameter model (e.g., FID drop of -2.02 on MJHQ-30K for the larger model). This demonstrates that our alignment approach is not a low-level fix but a fundamental enhancement that scales effectively with model capacity.
3. Holistic Gains: The framework improves not just one metric at the expense of another, but provides holistic gains across fidelity (FID), semantic similarity (CLIP), and fine-grained alignment (GenEval, DPG).

This evidence strongly suggests that scaling up does bring greater benefits, and that LABridge is a valuable and scalable framework for enhancing future, even more powerful, text-to-image models.

We hope these expanded explanations and more detailed tables thoroughly address the reviewer's questions.

We sincerely thank the reviewer for the thorough and constructive review. Your insightful comments and suggestions are invaluable for improving the quality and clarity of our work. Below, we address each point in detail.

---

1. Terminology: "Ornstein-Uhlenbeck (OU) Bridge"

We agree that our process does not model a classical "bridge" conditioned on a fixed endpoint $x_T = y$ (a delta distribution). Instead, it connects an initial point $x_0$ to a target distribution $\mathcal{N}(\mu_T(y), \sigma_T^2 I)$.

Our use of "bridge" follows recent literature (e.g., Denoising Diffusion Bridge Models [Zhou et al., 2023]), which also connect distributions rather than fixed points. We frame text-to-image generation as learning a stochastic bridge between the image latent manifold and a text-conditioned prior manifold.

We appreciate your suggestion to view our method as a "shifted" VP SDE. Indeed, our OU process resembles a VP-like process with a learned, text-conditioned mean-reversion point $\mu_T(y)$ rather than the origin, which is our key contribution.

To avoid confusion, we will revise Sections 2.2 and 3.2 to:

• Explicitly clarify that the "bridge" connects an initial point to a target distribution, not a fixed endpoint.
• Introduce the "shifted VP-SDE" interpretation as an intuitive understanding of our model, emphasizing that learning the semantic shift $\mu_T(y)$ is novel.

---

2. Mean-Reverting Property

You are correct that standard VP-SDEs are OU processes mean-reverting to the origin.

Our novelty lies in conditioning the mean-reversion on a semantically meaningful, non-zero mean $\mu_T(y)$ dependent on the text prompt. Standard VP-SDEs revert to zero regardless of conditioning, forcing the score function to steer the trajectory entirely.

Our explicit drift term $\theta(\mu_T(y) - x_t)$ provides stronger, direct guidance, improving stability and accelerating sampling—especially when the target $E[x_0|y]$ is far from the origin.

We will clarify in Sections 4.2 and 4.3 that the innovation is not the mean-reversion itself but the conditioning of the reversion mean, which enhances guidance compared to a fixed origin.

---

3. Theorems 4.1–4.4

We will reclassify Theorems 4.1–4.4 as Propositions and Heuristic Arguments, and refine the statements to specify precise conditions, removing vague terms like "potentially accelerates."

---

4. Experimental Details (Section 5, Appendix C.2, Hyperparameters)

We apologize for the omissions.

• Experiment description: Due to page limits, many details were moved to the appendix. We will add a concise summary of the experimental setup in Section 5.
• Appendix C.2: C.2 Learning from Scratch (ImageNet 512x512) does not actually describe the experimental setup and details. Essentially, it presents the experimental results of Table 5. Due to LaTeX formatting/layout issues, the table is placed at the top of the page, making it appear as if Appendix C.2 is empty.
• Loss weights $w_a, w_s, w_r$: After a brief sweep, we used $w_a=1.0$, $w_s=0.5$, and $w_r=0.2$. The alignment loss $L_{align}$ was prioritized, with semantic and reconstruction losses as regularizers.

We will incorporate these details in the final version and ensure appendices are complete.

---

5. Computational Budget and TIAE Overhead

• TIAE training (Stage 1) is significantly lighter than diffusion model training (Stage 2).
• The TIAE model is smaller than the main U-Net/DiT backbone and converges quickly on a simple regression objective. Empirically, Stage 1 consumes about 15–20% of Stage 2’s computational budget. For example, fine-tuning SDXL took ~ 260 GPU-hours on A100s for TIAE training.
• Inference overhead: Negligible, as TIAE requires only a single forward pass to compute $\mu_T(y)$. Inference time per 1024×1024 image is about 0.3–0.6 seconds.

We will add a paragraph on computational cost in the revised experimental section.

---

6. Code Availability

As noted in the checklist, we will release full source code, pretrained TIAE models, and training configs upon publication.

---

Additional Question: ODE Sampling and Alternative Samplers

We conducted preliminary experiments adapting DPM-Solver++ to our LABridge framework. Results on ImageNet 256×256 (FID↓) are:

These indicate LABridge provides a strong foundation that advanced samplers can further improve.

---

Thank you again for your valuable feedback.

We sincerely thank the reviewer for their valuable feedback and insightful questions. We provide the following clarifications to address the questions raised.

---

1. On the Latent Alignment Loss ($L_{\text{align}}$)

Goal is Alignment: The objective of this loss is to align the text latent space with the image latent space, rather than to minimize the loss as much as possible. Therefore, as long as the alignment is achieved, it is not necessary to optimize the loss to a very low value. Moreover, experiments show that this loss is not difficult to optimize; it decreases quickly and tends to stabilize.

Conditional Mean: If the text-to-image mapping were one-to-one, optimizing $L_{\text{align}}$ to zero would indeed create a direct text-to-image generator. However, the relationship is one-to-many: a single prompt $y$ (e.g., "a dog") corresponds to a vast distribution of possible images, and thus a distribution of image latents $x\_0$.

The loss is defined as an expectation over the data distribution: $L\_{\text{align}} = \mathbb{E}\_{(x\_0, y) \sim q\_{\text{data}}} [||\mu\_T(y) - x\_0||\^2]$

For any given text prompt $y$, the expression that minimizes this loss is the conditional mean of the image latents, i.e., $\mu_T(y) = \mathbb{E}[x_0 \mid y]$. Therefore, our TIAE is trained to map a text prompt to the average latent representation (or "center of mass") of all corresponding images.

Decoding this average latent $\mu_T(y)$ directly would likely produce a generic or blurry image, not a specific, high-fidelity instance. The purpose of this loss is not to be a standalone generator, but to establish a semantically meaningful anchor point in the latent space for the diffusion process to start from. The OU diffusion bridge is then essential for sampling diverse, high-quality instances around this well-positioned mean, which is far more efficient and stable than starting from a generic Gaussian prior.

---

2. On the Semantic Consistency Loss ($L_{\text{sem}}$)

Powerful TIAE Architecture: While the text encoder is frozen, the TIAE architecture (inspired by DiT) is a powerful Transformer-based model. It possesses significant expressive capability to learn a complex, non-linear mapping from the text embedding space to the image latent space. Our goal is not to perfectly replicate the geometry of the text space but to preserve its crucial semantic structure. For instance, if in the text space "king" is closer to "queen" than to "apple," then the corresponding latent priors $\mu_T$ should maintain a similar relative arrangement. The TIAE is flexible enough to learn this.

Crucial Role as a Regularizer: This loss serves as a vital regularizer. While $L_{\text{align}}$ anchors the priors for prompts seen during training, $L_{\text{sem}}$ ensures that the latent space between these anchors is structured in a semantically coherent manner. This is critical for generalizing to new or compositional prompts, as it encourages the learned prior manifold to be smooth and meaningful, preventing it from collapsing or becoming disorganized in unseen regions.

Ablation Study without $L_{\text{sem}}$: We apologize for the omission in the original ablation table. We have conducted the requested experiment and agree it provides valuable insight. Below is an updated version of the ablation results from Table 3, including a new row (in bold) where only $L_{\text{sem}}$ is disabled.

Table 3 (Updated): Ablation on Stable Diffusion XL (COCO-10K)

Stable Diffusion V1.5 Ablation

Stable Diffusion XL Ablation

The results show that removing $L\_{\text{sem}}$ leads to a noticeable drop in performance, particularly on the text-alignment metrics (GenEval, DPG), confirming its importance for preserving semantic structure and improving generalization.

3. Comparison to Denoising Diffusion Bridge Models (DDBM)

Our process does not model a "bridge" in the strictest mathematical sense of conditioning on a specific endpoint $x_T = y$ (a delta function). Instead, it connects an initial point $x_0$ to a target distribution $\mathcal{N}(\mu_T(y), \sigma_T^2 I)$.

Our choice of the term "bridge" was motivated by its broader use in recent literature, such as in Denoising Diffusion Bridge Models (DDPMs) [Zhou et al., 2023], which also focuses on connecting two endpoint distributions rather than two fixed points. Our work frames text-to-image generation as learning a stochastic bridge between the image latent manifold and a manifold of text-conditioned priors.

Our key innovations lie in how we define the two endpoint distributions for the text-to-image task:

1. The "Start" Distribution ($p_0$): We operate in the latent space of a pre-trained VAE, so our starting points $x_0$ are drawn from the manifold of encoded real images.
2. The "End" Distribution ($p_T$): This is our main contribution. Instead of a fixed prior like $\mathcal{N}(0, I)$, we introduce the Text-Image Alignment Encoder (TIAE) to create a structured, text-conditioned prior distribution for the endpoint: $p(x_T \mid y) = \mathcal{N}(\mu_T(y), \sigma_T^2 I)$.

Therefore, DDBM is the foundational tool we build upon. We use its principles to construct a bridge, but our novelty is in creating and aligning the text-conditioned target endpoint $\mu_T(y)$ that makes the bridge semantically direct and efficient. The Ornstein-Uhlenbeck (OU) process is a specific type of bridge we choose for its mean-reverting property, which perfectly complements our goal of steering the generation process towards the text-aligned prior $\mu_T(y)$.

We hope these clarifications have adequately addressed your concerns. We are grateful for your time and constructive feedback, which have helped us improve the clarity of our paper.