We thank the reviewer for their acknowledgement of the clarity of the setting and the differentiation to related work

W1,W2: Understanding Model Secrecy

Model secrecy is key in decentralized models as it is the only way to make the economics of large scale decentralised training practical. The fundamental problem we address is that today, there is no economic incentive for participants to collectively donate the millions of dollars worth of compute required to train a very large and expensive model if one of them can easily leak the trained weights (explained on lines 22-27). The solution is for a scheme that allows for such a model to be collaboratively trained and hosted while no participant can gain access to the entire weight set (lines 28-29). This allows schemes for programmatic value flow (for example, assigning revenue to participants) to be implemented. We note that reviewer kk7D praises this setting as "extremely realistic and useful". We will make the setting, its importance, and differences to existing settings such as federated learning more pronounced in the improved version.

W3: Model Secrecy and Number of Transforms

If a simple static model is hosted, which anyone can join and contribute to hosting, an attacker can gain access to the network and collect weights from different stages over time, eventually assembling the full model. At a high level, our approach prevents this by making the model weights incompatible with each other at different time steps. Thus model security is not tied to the number of transforms, but to ensuring that an attacker cannot gain a set of weights that use the same transforms, which requires ensuring that a new transform is applied between when an attacker exits the protocol and enters again. Note that this can be ensured by making participants have to wait for a transform step before entering, so there is a trade-off between the join rate and the overhead of the transforms (as more frequent transforms require more computation). As mentioned on line 270, our practical implementation has a transform step every 30s, meaning that anyone joining the protocol has to wait at least 30s, and has an overhead of 3% to total latency.
We provide a much more detailed write up of the setup and the probability of extraction for the attacker (both with and without UPMs) in Appendix A.

W4: Loss Curves

As mentioned in lines 323-326, for orthogonal transforms the loss curve is exactly the same as the original loss curve. We demonstrate this in Figure 6 in the appendix, where we plot the difference between the original loss curve and our approach over time, showing that for orthogonal transforms with on GPT2 124M, 355M, and 1.5B, the difference between the loss curves at each step is roughly 1e-2 (the final loss is 3.28) over 1.75B tokens. This highlights the importance of our decision to use orthogonal transforms during training; orthogonal transforms make the transformation on gradient space also orthogonal, so the exact same steps are taken (which is mentioned in Section 3.3, proven in Appendix D, and experimentally shown in Figure 6). To highlight the extent to which our approach does not alter training, we include the loss curve as a table below (data from the same source used to plot Figure 6.).

Q1: Double Precision for Computing Transforms

Double precision is needed to properly generate and represent the transforms, as otherwise there is large floating point representation error. Generating random orthogonal matrices as per Eq. 52-53 in Appendix C2 with single precision (compared to double precision) drastically reduces the orthogonality of the resulting transforms (how large $||Q^TQ-I||_2$ is). After generating transforms in double precision, the transforms can be combined with single precision weights and the output stored in single precision (that is, the precision of the actual weight matrices is never altered), and this is sufficient to avoid numerical precision errors as shown in Figure 3.

We greatly appreciate the reviewer's characterization of our method as "simple and effective," and their remark that the manuscript is "well-written and easy to understand".

W1: Attacker Motivation

The final trained model is not intended to be accessible by everyone, even at inference, only participants that hold part of the model can see their portion. As a result, after training the model exists, and yields value, only if enough participants continue hosting model portions to support inference. The model's value would be lost if the model was released after training (like current open weight models), or leaked by attackers. Note that even partially trained versions of the model can still have decent performance, and thus value, so it is important to also protect against weight extraction during training.

W2: Perplexity Increase

These experiments are provided to point out an important consideration when implementing UPMs. During training, gradient descent naturally corrects any floating-point drift introduced by numerical imprecision from the morphing, as demonstrated by the results in Table 1 and Figure 6. For low-precision inference, a workaround would be to actually morph high-precision weights (FP32 or FP64) stored on disk, which are then cast down for a low-precision copy in GPU VRAM used for inference. Note that our results in Figure 3 show that perplexity increase is extremely slow for FP32. Because transforms are applied infrequently, disk-to-VRAM communication is negligible, and the on-device memory holds only the low-precision weights.

W3: Typos

Thanks, we will fix this in the improved version.

Q1: Transforming All Stages vs Some Stages

Yes, it can work for only transforming a few stages, however as the weights are now less "scrambled", this becomes a trade-off with how much compute an attacker needs to pull off a learning attack. It is quite likely that a better trade-off can be gained by transforming more complex stages as you suggest. We leave investigating this trade-off to future work.

Q2: Possibility of Simpler Transforms

Adding random time-dependent noise to a layer's output does not change the weights, it only changes the activations. Thus weights are not inconsistent over time and can easily be pieced together to get the full model.
We specifically targeted complex and random matrix based transforms over simple linear transforms in order to make learning based attacks difficult. It is conceivable that transformations such as simple scaling of the entire weight matrix would be easy to undo by gradient descent.

We are grateful to the reviewer for praising the paper as "clearly written" and recognizing it is a meaningful solution.

W1,W2,Q1-Q4: Understanding the setting, previous work, and other solutions

We wish to clarify that this is a new setting (see line 32), which reviewer kk7D praised as "extremely realistic and useful". Decentralized training is an early but rapidly growing sub-field (see Appendix H for a full discussion of emerging work). The problem our work addresses is that the real world economics of large scale decentralised training are not practical, as there is no incentive for participants to collectively donate the millions of dollars worth of compute required to train a very large and expensive model if one of them can easily leak the trained weights (explained on lines 22-27). The solution is for a scheme that allows for such a model to be collaboratively trained and hosted while no participant can gain access to the entire weight set (lines 28-29). In short, unmaterializability lets models be created, hosted, and used solely within the protocol, enabling incentive mechanisms such as revenue-sharing among participants. If any participant can obtain the full model, these incentives become impossible. We will make the setting and its novelty more clear in the improved version.

As this is a novel setting there is no directly comparable prior work. As discussed in Section 4, the closest related work is black-box extraction attacks, which aim to gain a model with similar capabilities using only API queries. Other potential solutions for our setting, discussed on line 97, either rely on centralization or on expensive cryptographic primitives such as secret sharing or homomorphic encryption. Our method exploits the inherent structure of distributed pipeline parallel training, making it much more efficient. We also discuss the differences and similarities between our setting, Federated Learning and Decentralized Federated Learning in the related work (Section 4).

W3,Q5,Q6: Evaluating latency

There has been several recent works on feasibility of decentralized training [8,36,37] (see Appendix H for a full discussion) which focus on latency and throughput numbers that can be achieved in this setting. Our evaluation in Section 5.2 shows that the overhead of UPMs in this setting is quite low during inference (e.g., 3% impact to overall latency and 0.1% impact to total communication volume), the phase where morphing's impact is most pronounced. During training, the relative impact on all metrics decreases because memory use, communication, and computation grow with the model’s operations.

These figures depend on both the morphing frequency and the communication efficiency of the distributed implementation. Although the frequency can be reduced, the system must prevent participants from rejoining within a morphing interval. We did our communication overhead calculations relative to our centralized implementation, so the overhead of our method in practice could be much lower. Other hyper-parameters may influence communication efficiency, but to a much lesser extent.

We sincerely thank the reviewer for praising our method and acknowledging that it "addresses a significant challenge in collaborative machine learning", characterizing this as a "critical, new, and underexplored area."

W1: Orthogonality under Nonlinear Self-Attention

We do not require the self-attention layer to be linear, we only need it to satisfy the definition of a valid subfunction (Equation 2) or its generalization (lines 150–175), in which the function can be highly non-linear but the inputs interact linearly with the weights so that the transforms cancel out. We explain explicitly on line 161 why self-attention qualifies as a valid sub-function. However, the normalization layer that precedes the self-attention function poses a challenge, as discussed on lines 176–191. A key contribution of our work is to show that a standard Transformer block with RMSNorm layers is a valid generalized subfunction under this definition when orthogonal transforms are used for $T$, and can therefore be morphed by our approach without changing the network's forward function. Concretely, we are able to pass the transform through the input norm, so $Norm(XT^{-1})TU=Norm(X)T^{-1}TU=Norm(X)U$, because its orthogonality ensures that the the normalization factor of RMSNorm does not change.

W2: Clarity of Attack Descriptions

Thank you for the suggestion. We already provide detailed discussions of each attack in the appendices: general stitching attacks with probability analysis in Appendix A, matrix system attacks with more realistic requirements in Appendix E, and learning-based attacks with more results in Appendix G.2. Following your suggestion, we will add a separate section in the appendix concisely summarizing these attacks.

Q1: Varying the number of attackers

For stitching attacks, Appendix A quantifies how varying the number of attackers affects the probability of successful extraction. For example, with 24 stages and 10 replicas, if attackers control 1% of the compute (e.g., 100 compromised nodes out of a total of 10,000 nodes) the probability of extraction within 10,000 steps is $2\times 10^{-7}$ %. For matrix system attacks, Appendix E details the requirements of a more realistic variant, which becomes easier to mount as the number of attackers increases (however the countermeasures are irrespective of the number of attackers). For learning-based attacks, the decisive factor is the attackers' aggregate compute, not their count. Figure 4 demonstrates experimentally how much compute they require relative to the cost of fully training the model (measured in training tokens).

Thank you for the response, which addresses my primary concerns. I will raise my score accordingly to reflect the paper's contribution in posing an important scientific question and its elegant method.

We deeply appreciate the reviewer's endorsement of our setting as "extremely realistic and useful", their description of the method as "an exciting idea that propels collaborative training a step forward", and their assessment of the experiments as "detailed and well-done".

W1: Performance degradation due to precision

UPMs do not degrade model performance during training or inference in practice (we will clarify this). We provide the experiment in Figure 3, which highlights the drift caused by finite precision, to point out an important consideration when implementing UPMs.

During training, gradient descent naturally corrects any floating-point drift introduced by numerical imprecision from the morphing, as demonstrated by the results in Table 1 and Figure 6.

During inference, we have two options. The first option is to run inference in high precision (FP32 or FP64) as Figure 3 shows that performance degrades extremely slowly for FP32, with perplexity increasing by less than $1\times 10^{-4}$ over 10,000 transform steps. The second option is to keep the final weights in high precision on disk while maintaining a low-precision copy in GPU VRAM for inference. Then, at each transform step we apply the high-precision transform to the high-precision weights on disk, which are cast down and loaded into GPU VRAM to replace the copy used for inference. Because transforms are applied infrequently, disk-to-VRAM communication is negligible, and the on-device memory holds only the low-precision weights. Note that our experiment in Figure 3 shows that such a workaround is important for low-precision inference as naive FP16/BF16 inference would slowly cause model weights to drift due to accumulating floating point error.

W2: Performance in resource-constrained settings

As discussed in Section 5.2, UPMs add minimal overhead for both communication and memory (given the inference workaround above). Only the transform must be communicated every few steps, which is smaller than an activation gradient, and once folded into a layer it can be discarded introducing minimal memory overhead. Furthermore, because UPMs operate in a pipeline-parallel setting where the model is split stage-wise across devices, even GPUs with limited VRAM (e.g., T4s) are already supported, as they need to hold only a single stage. For devices that are even more resource-constrained, stages can be further split and still be supported by our framework. For example, splitting a Transformer layer between the self-attention and MLP blocks would still work with our transforms, though we leave concrete experiments to future work.

Q1: Understanding Figure 4

The experiments in Figure 4 are demonstrating learning-based attacks, in which attackers have stitched together trained but incompatible weights (each with different transforms applied because they are from different time steps), rather than results from training models from scratch. Although incompatible weights cannot be combined to recover the original function, they still contain useful information. Consequently, a plausible attack is to fine-tune starting from these weights to determine whether a useful model can be obtained with less computation than training the entire model from scratch. Figure 4 shows that the computation required, measured as the number of tokens used during training, is less than training from scratch, with at most a 36% reduction. Extended results in Figure 5 in the appendix indicate that the reduction is at most 20% for performance matching compute optimal training.

For results on training from scratch with and without UPMs, see Section 5.4 and Table 1. As expected, the number of tokens that UPMs require to match the perplexity of the baseline is the same or more (not less), and it specifically the same for orthogonal transforms. Additional plots are provided in Figure 6 in the appendices.

Q2: Morphing frequency during training

We apply transforms frequently during training (and inference) to prevent an attacker from leaving the protocol and rejoining within a single transform window. Thus, as mentioned on line 270, our standard is to morph every 30, which during training corresponds to a morph every 10-20 forward-backward iterations. If we want to ensure that no participant can exit and re-enter within those 30s, we can require each new participant to wait 30s before joining the protocol.

Q3: Performance degradation during training

As mentioned in the response to W1, during training gradient descent corrects for small precision errors leading to no performance degradation. We experimentally demonstrate this Table 1, training with orthogonal transforms yields no degradation in validation loss after a fixed number of iterations, whereas other transforms do as those transforms alter the loss landscape. We further show in Figure 6 in the appendix that the error in training loss (training drift) is small and constant throughout training with orthogonal transforms.

Q4: Information that can be gleaned by participants

Our method causes not only the weights but also the intermediate activations and gradients to change continually. Because we often use orthogonal transforms, which preserve weight and gradient norms, an adversary could still observe consistent statistics for those norms. However, there is little an adversary can do with that information.