FAST: Foreground‑aware Diffusion with Accelerated Sampling Trajectory for Segmentation‑oriented Anomaly Synthesis

We sincerely thank you for your detailed comments and positive feedback, such as "clear theoretical justification" and "complete studies." Below are our responses to your questions.

Q1: Practical benefit of fast sampling in anomaly synthesis.

We thank you for this insightful comment. While synthesis quality is important, we argue that fast sampling is highly beneficial in segmentation-oriented industrial anomaly synthesis (SIAS).

1. In real-world manufacturing, anomaly types evolve quickly, and segmentation models must be updated frequently. FAST enables engineers to generate hundreds of segmentation-aligned anomalies per class in minutes rather than hours or days after training, making rapid retraining feasible.
2. Our proposed AIAS achieves up to 100× speedup by reducing sampling steps from 1000 to 10, while still maintaining competitive segmentation performance (see Table 3 in the paper).
3. We observe that longer sampling may enhance visual realism but introduce overfitting to generative details at the cost of anomaly segmentation consistency. Therefore, in our context, slightly trading off fine-grained fidelity for significant speed gains is not only acceptable but necessary for practical deployment.

---

Q2: Detailed experimental setup.

We appreciate your suggestion and provide further explanation.

In the data synthesis phase, we follow AnomalyDiffusion’s mask generation strategy by employing two complementary methods: (1) augmenting real anomaly masks via geometric and morphological transformations, and (2) training a Latent Diffusion Model to learn the distribution of real masks and generate novel ones. All generated masks are screened to ensure plausibility before being used to synthesize anomaly–mask pairs. Detailed information can be found in the AnomalyDiffusion paper. For each anomaly type within every object class, we generate 500 anomaly–mask pairs to ensure sufficient diversity for segmentation training.

In the downstream segmentation task, we employ SegFormer, BiseNetV2 and STDC as backbone models, as they are widely adopted in industrial scenarios due to their balance of accuracy and efficiency. Besides, we uniformly split the real anomalies from MVTec and BTAD into 2/3 for validation and 1/3 for training. The training set of segmentation models combines this 1/3 subset with the synthetic samples. This design ensures that the segmentation model is evaluated solely on unseen data while benefiting from synthetic samples during training. Additional implementation details, including model settings and training parameters, are available in Supplementary Materials A.5.

---

Q3: The validity of the fixed $\hat{x}_0$ assumption in Lemma 2.

We appreciate your concern and provide the following reasons for the fixed $\hat{x}_0$ assumption.

1. Training Objective Consistency:
   The denoising diffusion model's training objective directly encourages consistency in predicting $\hat{x}_0$ across different timesteps, particularly at lower $t$. This ensures that even when reused in a large segment, a fixed $\hat{x}_0$ is still relatively accurate.

2. AIAS Theoretical support:
   In AIAS, the reverse process is divided into limited segments. Within each segment, the trajectory is approximated by a fixed $\hat{x}_0$ to maintaini the denoising distribution. At the start of each segment, the network re-calibrates $\hat{x}_0$ based on $x_t$ from the previous segment, effectively preventing error accumulation across segments. Previous work such as diffusion-based anomaly detection methods, also demonstrates that diffusion models are robust to small perturbations in $\hat{x}_0$, especially when guided by strong cues like masks or backgrounds. Thus, even if $\hat{x}_0$ from the previous segment is slightly biased, the trajectory remains aligned with the data manifold.

3. Denoising Distribution Dependence:
   The denoising distribution is Gaussian, with its mean given by the posterior mean:

   $\mu_{t-1 \mid t} = A_t x_0 + B_t x_t$

   where the coefficient $A_t$ is defined as

   $A_t = \frac{\sqrt{\bar{\alpha}_{t-1}} \, \beta_t}{1 - \bar{\alpha}_t}$.

   and the coefficient $B_t$ is defined as

   $B_t = \frac{\sqrt{\alpha_t}(1 - \bar{\alpha}_{t-1})}{1 - \bar{\alpha}_t}$.

   As $t$ decreases, $\hat{x}_0$ becomes more reliable, with $A_t$ increasing and $B_t$ decreasing. Therefore, AIAS increasingly relies on the re-calibrated $\hat{x}_0$, which helps mitigate the propagation of error from $x_t$. This also motivates the design of AIAS as a coarse-to-fine aggregation process.

4. Experimental Results:
   Our experiments show that increasing $K$ actually provides noticeable improvements within a certain range. However, with $K=10$, the results are already acceptable.

---

Q4: Theoretical analysis of error accumulation.

Thank you for your suggestion. The sampling approximation error in AIAS originates from two sources:
(1) local segment-wise error, introduced by keeping $\hat{x}_0$ fixed within each segment, and
(2) global cumulative error, resulting from propagation across segments.

Ideally $\hat{x}_0^{(t)}$ should be adaptively estimated at each time step t.

However, AIAS holds $\hat{x}^k_0$ fixed within the $k_{th}$ segment, leading to a local approximation error at the endpoint $t_e$.

$$\delta_{t_e} = \Delta_{t \rightarrow t_e} := \Sigma (\hat{x}_0 - \hat{x}_0^{(t)}) \tag{1}$$

Assuming the full reverse process is divided into $K$ segments with partition points:

$$T = t_0 > t_1 > t_2 > \cdots > t_K = 0,$$

Each segment $(t_k \rightarrow t_{k+1})$ contributes a local error and the global cumulative error can be described recursively. Specifically, the offset at time $t_1$ is computed as:
$\delta_{t_1} = \Sigma_0 (\hat{x}_0^{(0)} - \hat{x}_0^{(t_0)})$.

Then, the state at $t_2$ follows:
$x_{t_2} = \Pi_1 x_{t_1} + \Sigma_1 \hat{x}_0^{(1)}$,

which leads to the offset propagation:
$\delta_{t_2} = \Pi_1 \delta_{t_1} + \Sigma_1 (\hat{x}_0^{(1)} - \hat{x}_0^{(t_1)})$.

By recursion, the total error at the final timestep $t_K$ can be expressed as:

$$\Delta_{\text{total}} = \delta_{t_K} = \sum_{k=0}^{K-1} \left( \prod_{l = k+1}^{K-1} \Pi_l \right) \cdot \Sigma_k \left( \hat{x}_0^{(k)} - \hat{x}_0^{(t_k)} \right). \tag{2}$$

We further assume that the prediction error of $\hat{x}_0$ is uniformly bounded at each segment:

• $\|\hat{x}_0^{(k)} - \hat{x}_0^{(t_k)}\| \leq \epsilon_0$
• $\|\Sigma_k\| \leq C_\Sigma$
• $\|\Pi_k\| \leq \alpha < 1$

Here, $\alpha$ represents the maximum norm of the inter-segment state propagation matrices $\Pi_k$, and $C_\Sigma$ denotes the influence of $\hat{x}_0$ on the subsequent state within a segment. Notably, both are deterministic and computable under known noise schedules, as shown in Lemma 1.

Therefore, the total error admits the following conservative upper bound:

$$\|\Delta_{\text{total}}\| \leq \sum_{k=0}^{K-1} \alpha^{K-1-k} \cdot C_\Sigma \epsilon_0 = C_\Sigma \epsilon_0 \sum_{m=0}^{K-1} \alpha^m = C_\Sigma \epsilon_0 \cdot \frac{1 - \alpha^K}{1 - \alpha}. \tag{3}$$

This bound is highly conservative in practice, $\epsilon_0$ tends to decrease as $K$ increases, since shorter segments reduce approximation error and the overall procedure becomes closer to standard DDPM sampling. Moreover, the assumption that all errors accumulate in the same direction is pessimistic. Due to the inherent robustness of denoising diffusion models, the error introduced in one segment is typically corrected in the next, as $\hat{x}_0$ is re-estimated from the updated $x_t$. This re-calibration helps mitigate the propagation of prior inaccuracies.

These observations further explain why AIAS remains effective even when using only $K = 10$ segments. The combined effect of theoretical error control, diffusion model robustness, and segment-wise re-estimation of $\hat{x}_0$ ensures that the synthetic samples retain acceptable fidelity.

We thank the reviewer for the follow-up comment. We provide response below for the raised questions.

Sampling speed and anomaly quality.

We acknowledge the importance of generation quality, but we want reiterate that FAST is specifically designed for Segmentation-Oriented Industrial Anomaly Synthesis (SIAS), which differs from traditional anomaly detection or generic image generation. In SIAS, the synthesized data is used to adapt segmentation models. Our goal is to generate structurally aligned, mask-consistent anomalies in a timely manner, rather than focusing on visual realism. That is why using too large steps leads to degraded segmentation performance. While increasing the number of steps within a certain range can marginally improve image quality, it comes at the cost of significantly longer inference time, as shown in our ablation study. Besides, in real-world manufacturing, it is common for production lines to undergo frequent reconfiguration (known as production line changeover) to accommodate new products or materials. These changes often introduce new types of anomalies that existing segmentation models cannot handle. Delays in updating segmentation models during such transitions can lead to increased manual inspections, false positives, missed detections, and even line downtime. FAST is particularly suitable for solving this issue. As far as we know, there are also domain adaptation studies that focus on this issue, aiming to reduce the time required for segmentation model adaptation in order to improve production efficiency. FAST aligns with this motivation by enabling rapid synthesis of segmentation-oriented anomalies, thus supporting timely model updates under real-world constraints.

Different metrics across multiple methods.

In our opinion, our proposed SIAS is inherently different from traditional anomaly detection-based anomaly synthesis. That is why we initially focused exclusively on segmentation-based metrics such as mIoU and accuracy. However, after carefully considering your comments and similar concerns raised by other reviewers, we agree that including anomaly detection metrics such as AUROC, PRO, F1 and AP can offer additional insight into FAST’s performance. Therefore, we provide a comparative table below on the MVTec dataset. We hope these results help further demonstrate that FAST not only excels in segmentation-oriented synthesis but is also compatible with detection-based evaluation. Due to character limits, AP and F1 results are included in the comment for Reviewer QVrM. We hope you to check that section.

Table: AUROC comparison across various methods

Table: PRO comparison across various methods

We sincerely thank you for your positive comments about our innovations. Below are our responses to your specific questions.

---

Q1: Comparison to AnoGen.

We thank you for highlighting AnoGen, which is an important and well-recognized contribution to few-shot anomaly synthesis. We will cite AnoGen in the final version and now we provide a direct comparison between FAST and AnoGen. Our preliminary results show that AnoGen achieves impressive performance in most classes, and FAST also demonstrates excellent segmentation performance (e.g., higher mIoU and Acc in some classes), highlighting its effectiveness in segmentation-oriented anomaly synthesis.

Table: Comparison of mIoU and pixel-wise Accuracy (%) across 15 MVTec categories

---

Q2: Evaluation Metrics: IS and IC-LPIPS.

We appreciate your suggestion to evaluate IS and IC-LPIPS, and we report the results in the following table. FAST achieves competitive scores, indicating that our method generates visually realistic and diverse samples, even though it is not explicitly optimized for generic image generation. However, segmentation-oriented industrial anomaly synthesis (SIAS) is a novel direction and different from conventional image generation or anomaly detection. Our objective is not to maximize perceptual quality, but to synthesize structurally coherent and spatially aligned anomalies that benefit downstream segmentation. We agree that IS and IC-LPIPS can still offer helpful auxiliary insights, but pixel-level metrics such as mIoU and Acc on real unseen anomalies are more indicative of a method’s value in SIAS.

Table: Comparison of IS and IC-LPIPS for anomaly synthesis across various methods

---

Q3: On the 500-images-per-class setting.

We sincerely apologize for not clearly describing our data composition. The downstream segmentation models are trained on a combination of 1/3 real anomaly data and synthetic samples, while the remaining 2/3 real anomaly data are used for validation. Importantly, we generate 500 samples for each anomaly type within each class, rather than just 500 per class. Thus, the actual number of synthesized samples is much larger than 500 per class when multiple anomaly types exist. This design ensures sufficient structural diversity while maintaining training efficiency. We also conducted internal ablations showing that 500 per anomaly type strikes a good trade-off between segmentation performance and computational cost. We will clarify this design and rationale in the revised paper.

---

Q4: Mask generation clarification.

We apologize for not making this clearer in the paper. Our mask synthesis strategy adopts the protocol of AnomalyDiffusion, which consists of two complementary components:

1. Geometric augmentation of real anomaly masks through operations such as rotation and flipping.
2. Synthesis of new masks using a Latent Diffusion Model (LDM) trained on these real examples.

All synthesized masks are manually screened to ensure visual realism, diversity, and consistency with industrial abnormal structures. We refer readers to AnomalyDiffusion for further reference.

---

Q5: Training and inference time & value of t for fine-grained sampling.

As reported in Supplementary A.5, FAST is trained for approximately 80k iterations on a single A100 GPU. However, thanks to our training-free AIAS module, inference is significantly accelerated. To illustrate this, we provide the inference time under different numbers of AIAS segments (with batch size set to 8, the image size is 256 × 256), shown in the following table:

Table: Inference time for different sampling step settings (MVTec, batch size = 8)

As noted in our supplementary material, coarse sampling is applied until t = 2, after which we perform standard posterior updates to refine fine-scale textures. This choice of t is empirically determined through extensive analysis, balancing segmentation performance and inference efficiency. Under this setting, FAST maintains high segmentation quality while significantly reducing computational cost.

We sincerely thank the reviewer for the careful observation regarding the AP scores. The discrepancy from the numbers reported in [3] indeed arises from our evaluation protocol. Our work focuses on Segmentation-Oriented Industrial Anomaly Synthesis (SIAS), where the primary goal is to assess how well the synthesized anomalies improve downstream segmentation performance. To ensure fairness across all compared methods, we did not use the full detection pipelines or pretrained detectors from the original papers (including DRAEM, DFMGAN, and RealNet). Instead, we unified the evaluation by:

1. Generating anomalies using only the synthesis method of each model, rather than their complete pipelines with detection branches.

2. Feeding the generated data into the same segmentation backbone such as Segformer, which was trained from scratch.

3. Extracting the anomaly map from the SegFormer logits by taking the channel corresponding to the anomaly. And the anomaly map is then used to compute AP and other evaluation metrics.

We did not adopt the official pretrained weights because doing so would couple the synthesis evaluation with model-specific downstream architectures and training procedures, and thus introduce bias unrelated to synthesis quality. This unified backbone ensures that the AP values reflect only the quality of the synthesized anomalies, rather than differences in downstream detection models. As a result, the AP values may differ from those in [3], but they remain directly comparable within our controlled setting. We will clarify this evaluation protocol in the final version to avoid confusion.

Thank you to the author for the detailed response to my concerns. I will consider increasing the score. However, before doing so, I noticed that the AP metrics provided by the author for the DRAEM methods differ significantly from the AP metrics reported in other papers [3]. Was the DRAEM used in the experiments based on the official weights? Additionally, the AP of RealNet also seems to differ somewhat.

[3] Dual-Interrelated Diffusion Model for Few-Shot Anomaly Image Generation.

We sincerely thank you for the thoughtful and constructive feedback. We address the concerns below:

---

Q1: Lack of description for segmentation backbones, method intuition and limitations.

Thank you for the suggestion. We will add brief descriptions of SegFormer, BiSeNetV2, and STDC in the supplementary material for completeness. These three models are widely adopted in industrial scenarios due to their favorable trade-off between performance and inference speed. Specifically, SegFormer is a transformer-based model known for its lightweight design and robust accuracy; BiSeNetV2 employs a dual-branch structure for real-time segmentation; and STDC leverages short-term dense concatenation for fast and effective feature extraction. Evaluating performance across these architectures can further validate FAST's practical value in boosting segmentation performance under real-time constraints.

For the method intuition, we will revise Section 3 to better explain the underlying intuitions behind AIAS. Specifically, traditional diffusion models rely on dense, step-by-step denoising with predicted noise or clean image at each timestep. In contrast, AIAS accelerates this process by dividing the trajectory into a small number of coarse-to-fine segments. Within each segment, we fix the clean image estimate $\hat{x}_0$ instead of predicting it at each step, and analytically aggregate multiple DDPM transitions into a single closed-form update. By progressing from coarse to fine segments, AIAS compensates for the accumulated error that may arise due to temporal sparsity, enabling fast yet semantically consistent synthesis.

As for limitations, FAST currently relies on binary anomaly masks as input, which may limit applicability in scenarios where such masks are difficult to obtain. Extending it to weakly- or self-supervised mask generation is an exciting direction.

---

Q2: Uses other metrics instead of common ones in anomaly detection.

We appreciate your concern. As stated in Sec.1 and Table 1, our work targets a fundamentally different task from anomaly detection. While anomaly detection focuses on image- or region-level classification, which are often evaluated using AUROC, our work, segmentation-oriented industrial anomaly synthesis (SIAS), is a novel and emerging task that focuses on generating fine-grained plausible anomalies to enhance downstream pixel-level segmentation performance.

Therefore, our evaluation strictly follows segmentation-centric benchmarks. AUROC is unsuitable for evaluating spatial alignment and structural fidelity, and the most appropriate metrics are mIoU and Acc, which are standard choices in segmentation tasks. However, we agree that evaluating generation quality is also reasonable, no matter in anomay detection- or anomaly segmentation-based anomaly synthesis. We additionally provide generation quality assessments such as Inception Score (IS) and Improved Conditional LPIPS (IC-LPIPS) as follows.

Table: Comparison of IS and IC-LPIPS for anomaly synthesis across various methods

---

Q3: Mask generation

We apologize for not clarifying this. Our mask generation strategy follows the setup of AnomalyDiffusion, and employs two approaches:

1. Augmenting real anomaly masks via transformations (e.g., rotation, flipping).
2. Generating novel masks using a Latent Diffusion Model (LDM) trained on a small set of real anomaly masks.

All generated masks are visually screened to ensure structural plausibility and diversity. We will provide further implementation details to the supplementary material and refer readers to the AnomalyDiffusion paper for reproducibility.

---

Q4: Missing studies on other datasets

We thank you for this observation. In response, we have conducted additional experiments on VisA dataset and included the results below. These results reinforce our claim that FAST is a downstream-backbone-agnostic and dataset-agnostic framework for segmentation-oriented industrial anomaly synthesis.

Table: Pixel-level segmentation results (mIoU / Accuracy) on VisA using SegFormer trained on synthetic anomalies

We sincerely thank you for your thoughtful comments, particularly for "solid theoretical grounding" and our proposed "foreground-aware design." Below we address the key concerns.

---

Q1: Limited performance gains.

We respectfully note that FAST consistently outperforms all prior anomaly synthesis methods across multiple segmentation architectures and datasets (see Table 1, Table 2, and Supplementary A.6). In particular, FAST improves the average segmentation accuracy by +9.22, +9.46, and +6.67 points of segmentation models like SegFormer, BiSeNetV2, and STDC on MVtec-AD compared with the second strongest baselines.

Additionally, ablation studies demonstrate substantial gains over the baseline LDM in both quality and controllability of the synthesized anomalies. FAST achieves these improvements with as few as 10 sampling steps, offering over 100× speedup compared to standard DDPM (Table 3). We believe these improvements are non-trivial and practically meaningful for real-world industrial deployment.

---

Q2: CLIP dependency and generalization.

We appreciate your concern regarding CLIP-based saliency guidance. In our setup, instead of directly relying on CLIP embeddings, FARM learns to reconstruct anomaly-aware content in a data-driven manner and is fully decoupled from CLIP during inference, and FAST does not treat CLIP-based saliency as fixed ground truth.

As also demonstrated in the ablation study (Fig. 8), this strategy significantly improves anomaly localization without relying on saliency maps. Specifically, we follow AnomalyDiffusion by employing Bert/CLIP just for textual embeddings initialization, and embeddings are still optimized during training.

To further assess generalization under domain shift, we conduct additional experiments on a grayscale medical dataset (Montgomery X-ray). We extract 1,200 image–mask pairs (256×256) for training and 400 for testing, and augment the training data with 500 synthetic samples generated by FAST.

The results show encouraging improvements in downstream lesion segmentation, suggesting that FAST holds promise in domains with domain shift or limited visual structure, even when saliency may be less reliable.

Table: Segmentation results on the Montgomery County X-ray dataset using Segformer
Original data refers to training using only real samples, while Augmented data includes both real and FAST-synthesized samples.

---

Q3: Inference latency.

Thanks for this important question. As reported in Table 3 and Sec. 4.3, FAST achieves impressive performance with only 10–50 steps, compared to 1000 steps in DDPM.

Furthermore, FAST does not require retraining for different step sizes, making it highly practical for real-time or resource-constrained scenarios. We also provide the time cost for varying the number of AIAS segments (batch size = 8), which illustrates that FAST achieves up to 100× speedup while maintaining high-quality generation, as shown in Table 2.

In terms of inference latency, FAST is also highly efficient. It operates with minimal additional trainable parameters and achieves competitive results. We report runtime benchmarks under a consistent setting (batch size = 1), comparing FAST with other diffusion-based anomaly synthesis methods, as shown in Table 3.

These results confirm the suitability of FAST for downstream real-time segmentation tasks.

Table: Time cost (in seconds) of pixel-level anomaly segmentation with different diffusion steps on the MVTec dataset (batch size = 8)

Table: Inference FLOPs and number of trainable parameters of different models