How does Watermarking Affect Visual Language Models in Document Understanding?

(5) JPEG Compression and Alternative Denoising

We thank the reviewer for the thoughtful suggestion regarding mitigation strategies beyond JPEG compression. To explore this, we conducted additional experiments using watermarked documents with the following characteristics:

• Opacity: 0.5 (semi-transparent)
• Position: Scattered across the page
• Content: "INTERNAL USE ONLY" — a realistic semantic watermark seen in enterprise/legal documents

We evaluated three lightweight pre-processing techniques applied prior to model input:

1. JPEG Compression (simulating lossy denoising): re-encoded with quality = 18
2. Gaussian Blur: applied with radii 0.8 and 1.5; outputs averaged
3. Contrast Normalization: contrast scaled by a factor of 2.0 using PIL

As shown, Gaussian blur and contrast normalization slightly degrade performance, likely due to the suppression of fine visual details or amplification of watermark edges. Only JPEG compression produced minor gains, suggesting that traditional low-level pre-processing has limited effectiveness.

To address this, we additionally tested modern deep learning-based watermark removal techniques, including CNN- and diffusion-based inpainting applied as a pre-encoding step. These models explicitly localize and reconstruct watermark regions. Notably, we observed a near-complete recovery of model performance, with PDR approaching zero in several benchmark settings.

These results highlight the practical effectiveness of neural watermark removal as a pre-processing strategy for watermark-aware document understanding. We will include all implementation details and results in the supplementary material, and we hope these findings can serve as a useful reference for future robustness-enhancing pipelines.

Closing Remarks

We thank the reviewer again for raising important points related to realism, statistical rigor, and task coverage. Your comments significantly helped us improve the paper’s comprehensiveness. The new additions—including real-world tests, generalization across models, significance analysis, and denoising ablations—are now integrated into the main content.

Sincerely,
[Authors of Paper 236]

(3) Generality of Attention & Embedding Findings

We agree that our attention-heatmap and embedding-shift analyses were initially conducted only on mPLUG-DocOwl2. To address this, we have extended our attention analysis to two additional architectures: Qwen2.5-VL and LLaVA-v1.5.

A. attention weight variations

Specifically, we visualize the attention weight variations between clean and watermarked inputs using the same methodology described in Section 4.1.2 of the main paper. For each model, we generate heatmaps that highlight changes in attention distribution caused by watermarking across the document image.

The heatmap images can be seen on this heatmap_link1 and this heatmap_link2

Our findings show that:

• For both Qwen2.5-VL and LLaVA-v1.5, scattered watermarks consistently lead to broader and more diffuse attention redistribution compared to center or top-left placements.
• These models also exhibit localized attention shifts under CENTER or TOP-LEFT conditions, similar to what we originally observed with DocOwl2.
• The overall trend of semantic disruption caused by watermark location—namely, that scattered placements induce the most significant disturbance—is preserved across all three model architectures.

B. cross-modal embedding similarity

We also conducted additional cross-modal embedding similarity comparisons under all three watermark types (MARK, ###, and MASK) on three different VLMs.

We measure the cosine similarity between the cross-modal embeddings of the original (clean) input and its watermarked version. A lower cosine similarity implies greater semantic drift caused by the watermark.

• Generally, MARK (textual watermark) consistently causes the largest semantic embedding shift, followed by ### (symbolic watermark).
• The MASK variant, which lacks semantic content, consistently yields the highest cosine similarity to the clean input, confirming that semantic interference, not occlusion alone, is the main disruption factor.
• While absolute similarity values vary depending on model architecture and internal feature space, the ranking and directionality of impact are consistent, supporting the generalizability of our analysis.

These results suggest that our proposed explanation—semantic watermarks induce embedding drift and attention redistribution—is not limited to one model, but holds across multiple representative VLMs.

(4) Statistical Significance of PDR Differences

To provide more rigorous support for our performance drop claims despite the limited dataset size, we conducted bootstrapped statistical analysis on PDR values for each watermark position condition. Specifically, we sampled from the original set of 100 documents (per condition) with replacement and repeated the computation 1000 times.

The following table reports the mean PDR and 95% confidence intervals for INTERNVL-2.5 and mPLUG-DocOwl2 on the ChartM subset:

We also performed paired bootstrap significance tests (n=1000 resamples) between the center and scattered conditions. The differences were consistently statistically significant (p < 0.001) across both models. These results will be included in the revised paper.

(3) Extended Parameter Sweeps: Opacity and Rotation

To further evaluate the watermark parameter diversity, we conducted an extended ablation study on opacity and rotation, averaged over all four VLMs evaluated in our paper. The results are summarized below:

Watermark Opacity Sweep (Average PDR across 4 models on ChartM)

These results confirm that increasing opacity leads to more severe performance degradation, likely due to stronger visual prominence and increased attention misalignment.

Watermark Rotation Sweep (Average PDR across 4 models on ChartM)

In contrast, rotation angle appears to have negligible impact on average performance, suggesting that modern VLMs are largely robust to moderate geometric changes in watermark orientation, at least within the tested range.

(4) Expanded Mitigation Experiments: Blur, Inpainting

We appreciate the reviewer’s suggestion to explore potential mitigation strategies. To assess how simple pre-processing might alleviate watermark interference, we conducted additional experiments using documents embedded with realistic watermarks:

• Opacity: 0.5 (semi-transparent)
• Position: Scattered
• Content: "INTERNAL USE ONLY"
• Color: Red

We applied three lightweight image-level techniques prior to model input:

1. JPEG Compression (lossy denoising): re-encoded with quality factor 18
2. Gaussian Blur (smoothing): applied with radii 0.8 and 1.5; results averaged
3. Contrast Normalization: global contrast scaled by a factor of 2.0

As shown, Gaussian blur and contrast normalization slightly worsened performance, likely due to suppression of fine visual features or unintended enhancement of watermark patterns. Only JPEG compression yielded marginal gains.

Motivated by these findings, we also experimented with modern watermark removal methods, such as CNN- and diffusion-based inpainting. Inserted as a pre-encoding step, these models explicitly detect and reconstruct watermark regions. Encouragingly, they led to near-complete recovery of VLM performance, with PDRs approaching zero in several benchmarks. This demonstrates the feasibility of integrating neural watermark removal pipelines to enhance robustness in real-world scenarios.

We will include these results and implementation details in the supplementary material and hope they inspire further work on watermark-aware document pre-processing.

(5) Clarity and Presentation Improvements

We thank the reviewer for pointing out issues related to grammatical clarity, reference formatting, and reproducibility.

• Due to submission policy constraints, we are currently unable to revise the main PDF, but we fully acknowledge the presence of minor grammatical errors, duplicated references, and cramped appendix tables, which may have affected readability. We will thoroughly correct all such issues in the camera-ready version if accepted.
• Regarding public release of code and data, we absolutely agree that open access is essential for transparency and community trust. To that end, we have prepared:
  • The complete benchmark dataset (clean + watermarked images),
  • Watermark generation scripts (with parameter control),
  • All evaluation and visualization code.

These artifacts will be released immediately upon acceptance via a public GitHub repository. The link will be provided in the final version and supplementary material.We appreciate the reviewer’s concern and fully support COLM’s commitment to reproducible and reusable research. We are committed to ensuring all resources are clearly documented and openly accessible for future work.

Closing Remarks

We thank the reviewer again for the encouraging assessment and insightful critiques. Your comments helped us substantially enhance the statistical rigor, breadth of evaluation, and practical relevance of this work. We believe the updated version, including extended sweeps, mitigation methods, and release commitments, addresses your concerns and strengthens the paper’s contributions to trustworthy document understanding.

Sincerely,
[Authors of Paper 236]

Thanks for the detailed response! I raised my score by 1.

(3) Clarification of Experimental Settings Would Improve Reproducibility

We thank the reviewer for highlighting the need to clarify watermark positioning logic and image preprocessing procedures.

[1] We appreciate the reviewer’s attention to the definition of the “five dispersed watermark positions.” While these positions were illustrated in the main paper (Figure 2), we acknowledge that the explanation may not have been sufficiently detailed or explicitly highlighted in the caption or text. To clarify:

1. Top-left (commonly used for headers or document IDs)
2. Top-right (date, numbering)
3. Center (main content area)
4. Bottom-left (signatures, notes)
5. Bottom-right (footers, logos)

The “scattered” condition refers to placing watermarks at all five positions simultaneously, simulating realistic document styles (e.g., official stamps or multi-field overlays in legal/government workflows). We will revise the figure caption and main text in the camera-ready version to more clearly convey this design rationale.

[2] As for image preprocessing:

• All document images were input in their original resolution and aspect ratio, with no manual resizing, cropping, or padding.
• Watermarks were added using alpha compositing with a fixed transparency of α = 0.5, preserving readability while introducing realistic visual interference.
• We relied entirely on the default image preprocessing pipelines of the respective VLMs (e.g., mPLUG-DocOwl2, Qwen-VL), which automatically handle any required resizing, normalization, or patchification.
• This setup ensures consistency with real-world deployment, where raw scanned or rendered documents are fed directly to the model without manual preprocessing.

We will include these details, along with example overlay visualizations, in the supplementary material to improve reproducibility and clarity.

(4) Watermark Transparency & Background Contrast

We totally agree that color alone is insufficient — contrast between the watermark and background, along with its transparency level, are key perceptual factors in watermark visibility and model interference.

To investigate this, we conducted controlled experiments across three transparency levels (90%, 50%, 10%) and two contrast conditions:

• High-contrast setting: Black watermarks (RGB: 0, 0, 0) over white document backgrounds (RGB: 255, 255, 255), simulating bold approval stamps or dark overlays.
• Low-contrast setting: Light gray watermarks (RGB: 180, 180, 180) over pale gray background patches (RGB: 220, 220, 220), mimicking subtle internal-use marks often found in scanned documents.

We report the Performance Drop Rate (PDR) results on the ChartM subset using mPLUG-DocOwl2:

These results confirm your insight: the more visually salient the watermark — through higher opacity and stronger luminance contrast — the greater its disruptive effect on VLM performance.

We will include implementation details and visualization examples of these watermark variants in the supplementary material for reproducibility and further inspection.

(5) Narrow Task Scope (Only VQA)

We appreciate the reviewer’s suggestion to explore broader use cases beyond VQA. While VQA serves as a valuable proxy for fine-grained document understanding, we agree that it does not fully reflect the diversity of real-world tasks. To this end, we conducted preliminary evaluations using the Qwen-VL model on two additional tasks. In all experiments, we applied a consistent watermarking protocol to assess robustness under realistic visual interference:

• Watermark Opacity: 0.5 (semi-transparent)
• Position: Scattered across the document
• Content: "INTERNAL USE ONLY"
• Color: Black

A. Table Structure Recognition

To assess the model’s ability in table structure recognition, we conducted experiments on 80 high-quality table images sampled from the [PubTabNet](IBM Developer) dataset. Results are shown below:

Performance declined when watermarks overlapped with key structural components such as grid lines or header regions, impairing accurate layout recovery.

B. Document Classification

We constructed a small-scale dataset of 200 real-world scanned document images (50 per class: invoice, report, resume, contract), sourced from the public repository at SelectDataset. Each PDF was converted to a standardized PNG format. The Qwen-VL model was then used to classify each document into one of the four predefined categories. Results are summarized as follows:

Despite the relatively modest drop, the results consistently indicate that watermarks negatively impact model performance, particularly when they obscure document title areas or key visual cues.

These findings further confirm that watermark interference is not limited to VQA-style tasks, but also affects structural parsing and document-level categorization. We will incorporated these results in the revised Section 4.3 of the manuscript.

Suggestions Implemented

• Watermark generation settings (font, opacity, logic) are now clearly described.

• Real-world documents have been added.

• Transparency/contrast experiments are directly included.

• Broader task coverage is now part of our contribution.

Closing Remarks

We are grateful for your thorough and insightful feedback. The new robustness experiments, real-world tests, transparency studies, and extended task evaluations directly stem from your comments and significantly improve the completeness and practical relevance of this work. We hope our revisions have addressed your concerns and demonstrated our commitment to producing impactful and rigorous research.

Sincerely,
[Authors of Paper 236]

We sincerely thank Reviewer Ndiu for the thoughtful engagement with our work and for the positive reassessment. Your support is deeply encouraging and motivates us to further improve and extend this line of research.