Learning Repetition-Invariant Representations for Polymer Informatics

W1 & Q1: Example from lines 27-33

I don’t quite understand the example from lines 27–33. Could you provide a concrete illustration?

For L27-33, we draw an analogy between repeated phrases in language and repeated monomer units in polymers. For instance, in sentiment analysis, phrases like “very fair”, “very very very fair”, "very fair very fair" typically express the same sentiment. The sentiment model should recognize that the core meaning remains unchanged despite repeated modifiers.

Similarly, a polymer with one repeating unit (1RU) and many RUs should be assigned highly similar feature representations since their chemical identity remains unchanged.

Moreover, do large language models exhibit this problem? If not, perhaps we can handle such invariance naturally via augmentations present in real data.

Real-world augmentations benefit language models but transfer poorly to graph domains.

• In NLP, repetition is ubiquitous and cross-domain transferable. Natural language corpora consistently exhibit repeated structures across domains (e.g., “very very good”) such as news, literature, and scientific writing [3]. Allowing LLMs to learn repeat-invariant patterns through large-scale pretraining without task-specific augmentation.
• In graph domains, repetition patterns are highly domain-specific. As explicitly noted by Chen et al., 2022 [4], repetition in polymer graphs (see Figure 1) is highly domain-specific, distinguishing them from small molecules and social graphs.
• Polymer data scarcity As we discussed in W3, labeled polymer data is limited due to the high cost and complexity of obtaining reliable property labels.

Q2: Figure 2

What is the meaning of Figure 2?

Figure 2 visualizes how repeat invariance improves progressively—from GCN’s repeat-size-based clustering, to GRIN’s repeat-invariant latent space.

It shows a t‑SNE projection of polymer embeddings on the GlassTemp task for four models. We visualize 50 polymers, each represented by 20 embeddings (repeat sizes 1–20), colored ight (1RU) to dark (20RU).

• GCN (leftmost): Embeddings group by repeat size (e.g., all 20RU points, darker one, cluster together), not polymer identity, indicating poor invariance (sim ~ 0.73).
• GCN+RepAug: Light-to-dark stripes emerge—indicating that repeat variants of the same polymer start grouping together, as RepAug promotes consistency across repeat sizes (sim ~ 0.85).
• GRIN-RepAug and GRIN (right panels): Each polymer’s 20 repeat variants collapse into a tight, size-invariant cluster (sim > 0.98).

This trend is also quantified:

To make it clearer, we revised the figure caption as:

• "Figure 2: t‑SNE visualization of polymer embeddings from GCN and GRIN (ours) across repeat sizes (1–20 RUs) on the glass transition task. Points are colored by repeat count (light = 1RU, dark = 20RU). (a) GCN produces inconsistent embeddings for different repeat sizes of the same polymer, clustering by size (same color) rather than identity; our augmentation introduces light-to-dark stripes, indicating improved alignment of repeat variants. (b) GRIN learns repeat‑invariant representations: each polymer’s variants form tight, size‑independent clusters, further improved with augmentation."

How should we interpret the right-hand panel?

The last two panels show GRIN-RepAug and GRIN, where each polymer’s 20 variants (across repeat sizes) form tight, color-mixed clusters. This indicates that the model outputs nearly identical embeddings for different repeat sizes.

Q3: Overall pipeline

I’d like to confirm the overall pipeline:...

The reviewer’s understanding of the pipeline is generally correct for model training. For inference, the trained model can take polymers with arbitrary repeat lengths and produce invariant representations.

Q4: Kernel-based methods

Beyond these geometric deep learning baselines, what about kernel-based methods? I believe repeating patterns could be addressed using domain knowledge—perhaps via motif-inspired embeddings.

We agree that domain-specific methods like graph kernels or motif-counting approaches are promising directions for polymer representation, especially since repeating patterns are core to polymer structure.

Could we design (deep) graph kernels to tackle this problem?

However, designing a polymer-aware kernel or motif library is non-trivial. The key structural features often span both intra- and inter-monomer interactions, are sensitive to anchor positions within the chain, and vary with copolymer patterns (e.g., block, alternating, or random). Additionally, to support size invariance, such motifs must be periodic and invariant to repeat count—a challenge for traditional motif extraction.

In this work, we focused on GNN-based methods to learn such patterns automatically, without requiring an explicit library of motifs.

Q5: Residual Gated Graph ConvNets

How would models like Residual Gated Graph ConvNets, which have an inherent sequential design, perform on this task?

GRIN outperformed ResGatedGCN

We conducted new experiments on ResGatedGCN [5] and a pure sequence baseline, the SMILES-based LSTM [6]. GRIN outperformed ResGatedGCN [5] as follows:

MeltingTemp

Q6: The improvement of GRIN

The improvement of the GRIN variant over the baseline—and of training on G₁+G₃ versus G₁ alone—is not very pronounced given the limited training data. Could you explain this?

Across all tasks, GRIN consistently ranks among the top and remains stable, while baseline models often fluctuate—some peak on specific datasets and degrade significantly at larger repeat sizes.

• Testing on a single repeating unit (Test1), GRIN consistently outperforms BFGNN across homopolymer tasks such as MeltingTemp (~5%) and density (~6–7%), as well as more challenging copolymer tasks such as EA (~5%) and IP (~5%).
• Testing on 60 repeating units (Test60), GRIN’s advantages become more pronounced as it maintains strong performance across all tasks. Five out of eight baselines have negative $R^2$, contrasting GRIN’s generalization to different repeating sizes.

GRIN-RepAug is an ablated variant of GRIN without repetition augmentation. Its strong performance demonstrates the effectiveness of the new neural architecture under limited training data, but it does not imply that the inductive bias becomes unnecessary with larger datasets. From Tables 3, 4, 10 and 11, we found the new architecture remains important even when data augmentation is applied.

Does it imply that encoding such inductive biases as hard constraints is unnecessary when a large amount of training data is available?

As discussed in W3 and Q1, polymer datasets are limited and costly to label. GRIN addresses this by embedding repetition invariance as an architectural prior, enabling generalization under realistic, small-data regimes.

W2: Theory part

The theoretical part is more akin to intuition than rigorous evidence.

The theoretical analysis in Section 3.3 provides an algorithmic insight aiming to explain how specific architectural choices—such as max aggregation and L1 sparsity—promote consistent reasoning across varying graph sizes.

Discussion at L189–L204 show that, under these design choices and augmentation schemes, the model produces stable outputs regardless of repeat length. This aligns with our goal: to show that GRIN encourages repeat-invariant behavior.

W3: Limited dataset scale

GRIN generalizes better on larger unlabeled datasets for repeat-size invariance.

Collecting large-scale labeled polymer datasets is challenging. For example, researchers have spent nearly 70 years compiling only about 600 polymers with experimentally measured oxygen permeability in the Polymer Gas Separation Membrane Database [1].

To address your concern, we used a larger unlabeled polymer dataset from [2] to evaluate the effectiveness of our method. We applied model checkpoints from the GlassTemp task to ~13k unlabeled polymers and measured repeat-size invariance using cosine similarity between embeddings at repeat sizes 1 and 60. The results below indicate that GRIN generalizes better to large repeat sizes than other GNNs on this large-scale dataset.

References

[1] A Thornton, L Robeson, B Freeman, and D Uhlmann. 2012. Polymer Gas Separation Membrane Database.

[2] Otsuka S, Kuwajima I, Hosoya J, Xu Y, Yamazaki M. PoLyInfo: Polymer database for polymeric materials design. International Conference on Emerging Intelligent Data and Web Technologies, 2011.

[3] Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. (ACL 2020)

[4] Graph Representation Learning for Polymer Property Prediction. (NeurIPS 2022)

[5] Residual Gated Graph ConvNets. [6] Chen, G., Tao, L., & Li, Y. (2021). Predicting Polymers' Glass Transition Temperature by a Chemical Language Processing Model. Polymers, 13(11), 1898.

W1.1: Invariance

The paper constantly refers to their embeddings as "repeat-invariant". ...

GRIN achieves a cosine similarity of 0.999 ± 0.000 between repeat sizes 1 and 60 on ~13K unlabeled polymer SMILES.

Thank you for pointing out the distinction between exact mathematical invariance and the practical invariance we observe. We appreciate this suggestion and have revised the phrasing to ensure greater rigor. Our quantitative analysis shows that the embedding similarity between repeat sizes 1 and 60 is very close to 1, indicating strong repeat invariance.

Specifically, we have updated the manuscript as follows:

• L252: “…demonstrating true repetition invariance.” → to: “…demonstrating repetition invariance.”
• We re-scanned the manuscript and found no other instances of absolute phrases such as “true repetition invariance.” To avoid ambiguity, we will add the following clarifying sentence in the Introduction: “Throughout this work, we use ‘repetition-invariant’ to denote practical invariance, defined as a latent representation similarity above 0.99 across repeat sizes.”

To quantify repetition invariance, we evaluated the model checkpoints from the GlassTemp task on ~13K unlabeled polymer SMILES [1]. We computed the cosine similarity between embeddings at repeat sizes 1 and 60. As shown below, GRIN achieved a cosine similarity of 0.999 ± 0.000.

and the embeddings shown in Figure 2 show that GRIN is not actually invariant, just approximately so.

We calculated the mean cosine similarity between embeddings at repeat sizes 1 and 60 on the GlassTemp test set. GRIN achieves 0.998 (± 0.000). (Full table please refer to Minor 2)

W1.2: MST alignment

The paper claims to "align the model with the MST construction process" - very vague terminology

"dynamic programming(DP)-style updates enforce a fixed reasoning procedure" means.

We follow the term "neural algorithmic alignment" [4], aiming to design neural architectures that align structurally with specific DP paradigms.

• Algorithmic paradigm: MST algorithms build a tree by greedily selecting sparse, non-redundant edges.
• Neural architecture: GRIN’s Max aggregation selects the strongest incoming message, while the sparsity penalty suppresses less informative edges. This encourages sparse decision paths and avoids uniform mixing of neighbor information, resulting in a tree-like information flow qualitatively aligned with MST-style connectivity.

GRIN's update scheme mimics DP-style reasoning: a fixed local rule is applied layer-wise, resulting in deterministic update paths. It avoids arbitrary neighbor mixing, as seen in mean or soft attention aggregation.

We clarified the phrasing as:

• L49: "GRIN aligns message passing with the edge-greedy logic" → to "GRIN draws inspiration from the edge-greedy logic"
• L53-54: "aligns it with the MST construction process." → to "heuristically encourages alignment with ..."

the actual "alignment" is not particular to the GRIN model.

Most GNNs adopt sum or mean aggregation [2,3]. The combination of max aggregation with an L1 sparsity penalty is underexplored and, to our knowledge, has not been theoretically or empirically studied for polymer property prediction. This architectural choice, together with repeat-unit augmentation, distinguishes GRIN from prior work.

why the "alignment" with MST is actually useful

L51 "The MST criterion further preserves the most informative backbone of the polymer graph."

The inductive bias of MST is particularly beneficial for repetition invariance. When a polymer graph is expanded with repeated units, many edges are redundant. MST-style selection prevents uniform mixing of the duplicates and concentrates information flow along maximal, consistent paths that capture the core structural features independent of graph size.

W1.3 GRIN phrases

Thank you for the suggestion. The main text reflects this point in Lines 47–49: “In this work, we propose Graph Repetition INvariance (GRIN), a method that combines algorithm-alignment with repeat-unit augmentation to address the challenge of repetition-invariant representation learning.”

W2: Clarity of theory

Thanks for your suggestion. We revised and moved L174: "With repeat-augmented polymers $P_m$ ($m$ ≥ 2), the model can apply same update rule at each layer, yielding a constant output $y^⋆$, irrespective of the repeat size." to the beginning of Section 3.3.2 (after L153).

We revised the sentence on L181: "...is $L$-Lipschitz with $L$<1..." → to "... is assumed to be $L$-Lipschitz, i.e., it satisfies $|f(x)−f(x′)|≤ L|x−x′|$ for all inputs $x$, $x′$ and some constant $L$<1...""

W3: Motivation for max aggregation

In Appendix C.2 (Lines 417–424, Table 9), we already evaluated different aggregation schemes within the GRIN-RepAug (GCN/GIN backbone). Both sum and mean aggregators lead to varying degrees of performance degradation.

W4: Experimental Results

GRIN consistently outperforms baselines across repeat sizes, with especially strong generalization on Test60 where many baselines fail.

Testing on single repeating units (Test1), GRIN consistently outperforms BFGNN across homopolymer tasks such as MeltingTemp (~5%) and density (~6%), and more challenging copolymer tasks EA (~5%) and IP (~5%).

On Test60, GRIN’s advantages become more pronounced as it maintains strong performance across all tasks. While 5/8 baselines have negative $R^2$.

Q2: L78

“Causal information” refers to task-relevant structural features that remain consistent regardless of the repeat count. We acknowledge that “causal” may suggest a formal treatment (e.g., counterfactuals) which we do not pursue here. To avoid confusion, we revised L78: "causal information” → to “key structural information”

Q3: L120 - L121

Thank you for catching this. In our design, skip arises implicitly: max aggregation selects one dominant neighbor per dimension, and L1 sparsity suppresses the rest, making their contributions negligible—functionally approximating an identity mapping for non-winning paths. We revised L121: "With residual connection h, any node not chosen by max-aggregation simply skips the update..." → to "With previous state $h^{l-1}$, any non‑max neighbour contributes zero to $U^l$ leaves corresponding feature dimensions remain unchanged..."

Q4: L149

The guarantees are stated in Section 3.3:

• Proposition 3.2: Training on {$P_1$, $P_n$} with L1 sparsity yields predictions invariant to chain length ($m ≥ 2$) under the hyperchain abstraction, as all node types are exposed and non-dominant paths are suppressed.
• Proposition 3.3: $n = 3$ is the minimal augmentation introducing a degree‑2 node and enabling true two-branch competition; larger $n$ yields diminishing returns under the $L$‑Lipschitz contraction, consistent with observed performance saturation.

Q5: L257

We intended to emphasize that the max-aggregation was inspired by MST, but saying “MST-aligned max aggregation” can be redundant. We simplified this to avoid confusion:

• L257: “... MST-aligned max aggregation ...” → to “... MST-aligned aggregation ...”

Minor 1: Figure 2

Figure 2 is not explained well in the caption.

The caption is revised as: "Figure 2: ... glass transition task. Points are colored by repeat count (light = 1RU, dark = 20RU). (a) GCN produces inconsistent embeddings for different repeat sizes of the same polymer, clustering by size (same color) rather than identity; our augmentation introduces light-to-dark stripes, indicating improved alignment of repeat variants. (b) ..."

Also, the figure doesn't show explicitly which polymers belong to the same class...

Though Figure 2 does not label polymer identities, it clearly visualizes improved repeat-size invariance across models—supported by Minor 2’s scores:

• GCN: Embeddings cluster by repeat count—e.g., all dark points (20RU) group together, showing low invariance (sim ~ 0.73).
• GCN + RepAug: Light-to-dark stripes emerge, showing variants of same polymers begin to align, shifting clustering from repeat size to identity (sim ~ 0.85).
• GRIN w/wo RepAug: Each polymer’s embeddings form tight, size-independent clusters, indicating strong invariance (sim > 0.98).

Minor 2: L41

Thanks for your comment. We computed embedding similarity between repeat sizes 1 and 60 on GlassTemp and found that RepAug improves GCN stability, while GRIN variants achieves consistency.

Minor 3: L104

Thanks for your comment. We added a visualization for this part. According to the conference's protocol, we are not allowed to attach figure or link here.

Minor 4: L127

Thanks for your comment, we defined it at first mention: Revised (L124–L131): “Max aggregation imposes a selection bias toward the strongest neighbor, we add an L1 sparsity penalty on the message and update networks to suppress non‑dominant pathways (Eq. 3), encouraging a sparse, MST‑like backbone.”

Minor 5: L132

We corrected “constrcuted” → “constructed” in the description.

References

[1] PoLyInfo: Polymer database for polymeric materials design. International Conference on Emerging Intelligent Data and Web Technologies, 2011.

[2] How Powerful Are Graph Neural Networks? (ICLR 2019)

[3] Physical Pooling Functions in Graph Neural Networks for Molecular Property Prediction, 2022.

[4] Graph neural networks extrapolate out-of-distribution for shortest paths, 2025.

W1 & L1: Baseline models on multiple RUs

All baseline models are trained only on 1-RU graphs, while GRIN is trained on both 1-RU and 3-RU graphs. This makes it unclear whether GRIN's performance gain stems from model design or richer input structures.

GRIN outperformed the GCN and GIN baselines with 1RU+3RU augmentation

Thank you for your question. As described in Appendix C.2 (Lines 425–432, Tables 10 and 11), repeat-unit augmentation (1RU+3RU) was already applied to both GCN and GIN. While this augmentation improves the performance of all models, GRIN consistently achieves the best performance. A subset of the results on the GlassTemp task is provided below for reference:

Ablation studies showned that both data augmentation and model design improved the performance

Appendix C.2 shows the effectiveness of data augmentation.

In Tables 2–4, GRIN-RepAug represents the variant using only the new model design without data augmentation. This variant still outperformed other baselines. In summary, both the model design and richer input structures contribute to the performance improvement.

W2 & L1 & Q1: Generalization to more complex structures.

The model does not enforce repetition-invariance via explicit constraints or loss terms. Instead, it relies on structural heuristics like max-pooling and augmented training data, which may not generalize to more complex repeat structures.

The method is designed for strictly repetitive polymers, but many real-world polymers exhibit branching or block copolymer patterns that deviate from perfect repetition. Have the authors explored how GRIN behaves on polymers with irregular or partially repeating structures?

GRIN generalizes to complex structures such as copolymers and larger datasets

Thanks for your comments. We have evaluated GRIN on both homopolymers (Tables 2 and 3) and copolymers, including block and alternating structures (Table 4), demonstrating that the approach is not limited to single-unit structures and can handle more complex architectures.

We further evaluated GRIN on a larger dataset by testing model checkpoints from the GlassTemp task on approximately 13K unlabeled polymers from [1]. For each polymer, we computed the cosine similarity between embeddings at repeat sizes 1 and 60. As shown below, GRIN achieved a cosine similarity of 0.999 ± 0.000, demonstrating its ability to generalize to polymers with more repeat patterns.

W3 & L3: Efficiency

Although GRIN generalizes well to long chains, the paper does not evaluate training or inference cost as the RU number increases. This may raise concerns about efficiency in industrial-scale deployment.

GRIN scales efficiently, outperforming several baselines in speed and memory.

We measured training‐cost breakdown using the MeltingTemp task on a single NVIDIA A6000 GPU. All models share the same hyperparameter configuration and GCN backbone for fairness.

Even as the training set doubles, GRIN maintains moderate memory usage (557 MiB) and training time (approximately 764 s), ranking near the middle of the efficiency table, faster than many baselines (e.g., SSR, DIR) and only marginally heavier than GRIN RepAug. We will update Section 4.3 to include the new results.

Q2: Evaluation on multiple RepAug schemes

While the paper claims that three RUs are sufficient, it would be helpful to evaluate whether this number is optimal or if performance significantly changes with more or fewer units.

GRIN with 1RU+3RU augmentation consistently performs best across tasks.

Thank you for the suggestion. This question was already discussed in Section 4.3 (Lines 234–239) and visualized in Figure 3 for the MeltingTemp (homopolymer) and IP (copolymer) tasks. We plotted the figure by varying the number of repeating units and observed a common trend: the {1, 3} scheme outperforms the others. We report the results on the Density task below for reference.

References

[1] Otsuka S, Kuwajima I, Hosoya J, Xu Y, Yamazaki M. PoLyInfo: Polymer database for polymeric materials design. International Conference on Emerging Intelligent Data and Web Technologies, 2011.

W1: Graph transformers

Although authors compare their method to variants of GIN and GCN, but I would like to see how this behaves with Graph Transformers.

GRIN outperformed the graph transformer-based GraphGPS

Thanks for your comments. We conducted new experiments using GAT [1] and GraphGPS [2] on MeltingTemp and O$_2$Perm. The table below showed that GRIN outperformed graph transformer models such as GraphGPS [2].

MeltingTemp

O$_2$

W2: Larger datasets

Experiments can be extended to larger datasets.

GRIN generalizes better on larger unlabeled datasets for repeat-size invariance.

Thank you for the suggestions. Collecting large-scale labeled polymer datasets is challenging. For example, researchers have spent nearly 70 years compiling only about 600 polymers with experimentally measured oxygen permeability in the Polymer Gas Separation Membrane Database [3].

To address your concern, we used a larger unlabeled polymer dataset from [4] to evaluate the effectiveness of our method. Specifically, we applied model checkpoints from the GlassTemp task to 12,764 unlabeled polymers and measured repeat-size invariance using cosine similarity between embeddings at repeat sizes 1 and 60. The results below indicate that GRIN generalizes better to large repeat sizes than other GNNs on this large-scale dataset.

References

[1] Graph Attention Networks, ICLR 2018.

[2] Recipe for a General, Powerful, Scalable Graph Transformer, NeurIPS 2022.

[3] A Thornton, L Robeson, B Freeman, and D Uhlmann. 2012. Polymer Gas Separation Membrane Database.

[4] Otsuka S, Kuwajima I, Hosoya J, Xu Y, Yamazaki M. PoLyInfo: Polymer database for polymeric materials design. International Conference on Emerging Intelligent Data and Web Technologies, 2011.