Disturbance-based Discretization, Differentiable IDS Channel, and an IDS-Correcting Code for DNA Storage

We sincerely thank the reviewer for their valuable comments. We hope our rebuttal has adequately addressed the concerns. Minor concerns not mentioned will also be revised.

Q1: The code is empty.

A1: This appears to be a cache issue with the anonymous hosting platform, as several similar cases have also been reported on their GitHub page. It has now been fixed and is accessible at the same link: https://anonymous.4open.science/r/THEACode .

Nonetheless, it’s our fault for not thoroughly verifying the availability of the code.

Q2: How the dataset/profile was constructed and error operation defined?

A2: The DNA sequences are randomly generated with equal probabilities for ATGC, no inherent patterns. The error profiles are constructed according to the respective channel settings. For instance, under the default setting, each profile position undergoes an Ins, Del, or Sub with equal probability (Err_Rate/3), and Ins/Sub are further distributed equally among bases.

For the simulated realistic channel MemSim, we use its official implementation to generate the output sequence $s’$ from the input $s$, then infer an error profile $p(s,s’)$.

We will add an App Sec for the error profile. Here's a brief example: given the sequence ATGGC and an error profile of (Ins C, Ins T, 0, Del, 0,0,0), the resulting sequence would be CTATGGC.

Q3: Comparing with similar NN-based ECC.

A3: We follow the research trend of NN-based ECCs, which focus on linear codes such as LDPC. However, correcting errors in AWGN channels is fundamentally different from handling IDS errors. Ins and Del shift the entire sequence, making them inherently unsuitable for linear codes.

Transplanting these methods to IDS correction would likely face the same challenges addressed in this manuscript. On the other hand, directly applying such approaches to IDS errors would not differ significantly from using conventional linear codes, which has been explored in very early DNA storage research with unsatisfactory results.

Q4: Why conventional approaches do not address varying channel settings?

A4: We believe this is due to two main reasons:

• In conventional ECC research, handling complex AWGN channels has not been a primary focus, as such complexity is less critical than in DNA storage. As evidence, although NN-based ECCs offer advantages in complex channels, most existing works do not emphasize this capability.
• IDS correction follows a different approach from conventional ECCs. Even for the simplest case of correcting a single IDS error, the mathematical foundations remain open, and an optimal code has yet to be established. Designing more advanced IDS codes for complex channels thus remains a challenging open problem.

Q5: Is it possible to process a short codewords with paddings?

A5: Padding is not explicitly described but is actually used throughout the work. This is necessary because synchronization errors (Ins/Del) alter the sequence length.

Applying variable-length sequences is an interesting topic. It raises the question of whether the model would learn individual codes for different lengths or a consistent code that accommodate both short and long sequences.

This model was trained with fixed-length sequences, as sequence length is a key prior knowledge for correcting synchronization errors. For example, without the knowledge of codeword length, a broken codeword of length n could originate from either a length n-1 codeword with an Ins, a length n codeword with Subs, or a length n+1 codeword with a Del. Multiple errors would further complicate this scenario. We infer that variable-length sequences would increase the task's difficulty significantly.

Consider this, it may require extensive research in future work to answer this question.

Q6: More details on Chicken-and-Egg dilemma.

A6: The dilemma arises from the interdependence of the encoder and decoder during training. Specifically, when using discreteness constraints, if the encoder converges prematurely to a local minimum due to these constraints, the entire framework fails to function properly. To mitigate this, we introduce the auxiliary task, which serves as a logical "warm-up" for the encoder. This task is simple yet effective, as in App D.

Q7: Competitive to combinatorial codes in correcting single error?

A7: The accuracy cannot surpass combinatorial codes in this scenario, as they are mathematically guaranteed to correct a single error. However, from our other research efforts, it’s found that a NN-based decoder can 100% decode the combinatorial codewords.

To evaluate whether the end-to-end method competitive to combinatorial code in single IDS channel, we conducted experiments with a code rate aligned to Cai’s code at 34/50 and 133/150. The reported NER is: 1.6%, 2.1%, respectively, inferior to the combinatorial code, although the THEA-Code performs far better in correcting multiple errors.

We sincerely thank the reviewer for their valuable efforts. We will revise the manuscript accordingly. We hope our rebuttal has addressed the concerns.

Q1: Is a more realistic setting than MemSim available?

A1: Firstly, a simulated channel is essential for training such a model, as alternating between training epochs and wet-lab experiments is neither time- nor resource-efficient.

As far as we know, MemSim is currently a most advanced option available. It utilizes base-context-related statistics to simulate the biochemical process. In reality, DNA storage channels vary significantly depending on the specific methods, equipment, etc. As a result, users may prefer to train codes tailored to their own channels rather than relying on a universal simulated channel. This is also why we present our work as a generalizable method rather than just a standalone model.

In future research, a generative model like VAE, in combination with the differentiable IDS channel, could be explored for a NN-based simulation.

Q2: If the goal is to simply produce sharp distributions, will simply adjusting the temperature of the softmax will achieve the same thing?.

A2: This is an insightful thought. Our initial attempt involved adjusting the softmax temperature, but this alone was not the optimal choice. In training phase, we want to progressively sharpen the encoder’s output distribution while ensuring the decoder remains sensitive only to the maximum entry of the distribution.

Applying low-temperature softmax in training may hinder convergence. If the autoencoder has not yet learned meaningful features, an overly sharp distribution can prevent it from reaching a non-trivial solution. Experiments across different settings showed that while convergence is possible, it requires careful tuning of $t$, making training less robust. Some results under the default setting are listed below:

It suggests that adjusting the softmax temperature can help generate low-entropy codewords, but the functionality of the codewords is compromised compared to the proposed disturbance constraints.

Beyond applying fixed low-$t$ softmax, we also explored alternative approaches, including: random $t$ softmax, $\sin t$ softmax, and optimizing an entropy constraint on the distributions. Among these, the entropy constraint also worked, but similar to adjusting the softmax $t$, its weight $\lambda$ required careful tuning to balance distribution sharpening, model convergence, and preventing the decoder from exploiting soft distributions.

Q3: It would be interesting to see how the system performs without the simulated IDS channel, and how accurate the simulated IDS channel is.

A3: We would like to thank the reviewer for this insightful comment.

Without the simulated channel, the entire framework would not function, as the neural-network-based channel is essential for back-propagating gradients to the encoder. If we bypass the IDS channel using a straight-through approach, the task degenerates into a trivial copy-and-paste operation with 100% accuracy. This has been confirmed in unreported experiments by setting the channel error rate to 0.

We evaluated the accuracy of the simulated IDS channel (DIDS) by comparing it to the groundtruth produced by the conventional IDS channel (CIDS). Part of the results are as follows:

We found that the learned channel is reliable for simulating channels with an error rate of less than 20%.

We will append a section in App C to include detailed results from these experiments.

Q4: The simulated IDS channel takes an error profile vector, which is a simple statistics over types of errors encountered. However, it is unclear how a simple vector can summarize more complex errors.

A4: The error profile records the errors that occur in a sequence. For instance, given ATGGC and an error profile of (Ins C, Ins T, 0, Del, 0,0,0), the resulting sequence would be CTATGGC. This strategy covers all possible error types.

We will add an App Sec on how the profile is defined.

In the simulated channel the profile is DNA-depended. The differentiable IDS channel faithfully transforms the sequence according to the error profile. Thus, the channel is fully defined by how the profile vector is generated. Context-free channels, such as C111, generate error profiles based on preset probabilities, while MemSim is sequence-dependent, generating profiles based on DNA sequences by sampling from $P(profile|DNA)$. The right column of Line 371 in the manuscript describes this part in detail.

We sincerely thank the insightful feedback, which is invaluable in improving our manuscript. As this is the only negative score, we genuinely hope our rebuttal has addressed all the concerns and that the reviewer may reconsider the score.

Q1: The code is empty.

A1: This appears to be a cache issue with the anonymous hosting platform, as several similar cases have also been reported on their GitHub page. It has now been fixed and is accessible at the same link: https://anonymous.4open.science/r/THEACode .

Nonetheless, it’s our fault for not thoroughly verifying the availability of the code.

Q2: Any genomic datasets are used? Random AGTC sequences?

A2: We use random AGTC sequences, as DNA molecules serve as a memoryless medium for storing arbitrary information in DNA-based information storage.

The use of genome-style or bio-compatible DNA sequences for in vivo storage has been explored (see [1]). However, in this in vitro storage, where DNAs exist in dry powder form, sequence constraints are relaxed. Typically, only patterns that are difficult to synthesize or sequence should be noticed, which is actually a property of the IDS channel (i.e., patterns introducing higher error rates) and should be in charge of channel modeling.

When storing genomic information in DNA molecules, genomic knowledge is also unnecessary. The data is usually compressed in silico before storage, maximizing entropy and eliminating inherent sequence patterns.

[1] An artificial chromosome for data storage, NSR

Q3: Thm 3.1 needs some work. Proof for general $\tau$ and $n$ logits.

A3: We will revise Thm 3.1 and its proof for clarity. Specifically, $\epsilon_1$ is similarly to the convergence tolerance, while $\epsilon_2$ accounts for the chance that $y_1$, as a sample from a distribution, deviates beyond this tolerance.

Additionally, we will provide a full proof in the Appendix for general $\tau$ and $n$ logits, in about 1.5 extra pages. The proof follows the sketchy proof but includes additional details and a trick involving the mean value theorem for multivariable functions.

Q4: Codeword length, why not fix source length? Still work at 300/600?

A4: In DNA storage, the sweet point of molecule length is around 150 due to biochemical limitations. Shorter lengths require additional indexing resources, while longer lengths are currently neither time- nor cost-effective; excessively long synthesized DNA sequences accumulate high error rates. Therefore, we conducted experiments by fixing the codeword length rather than fixing the source length.

As suggested, we had experiments with both shorter and longer codeword lengths at settings 25/50 and 300/600. Under a 1% error channel, training the code for 25/50 was much easier, while training for length 600 was relatively challenging and resulted in an inferior NER, as in

We acknowledge that the proposed method is not applicable to arbitrarily long sequences, as it relies on a plain Transformer, which is computationally impractical for very long inputs. However, encoding long DNA is not a currently urgent priority. Future work may explore more efficient Transformer variants for this purpose.

Q5: Fairness of comparison. Source/Codeword length settings?

A5: Yes, the comparison is not entirely fair, primarily because the compared methods use discrete settings. The compared source/codeword lengths are presented in Tab 7 App A. The most unfair setting is for Cai’s code, which uses smaller codeword lengths to align the code rate. However, in this case, Cai's accuracy is overrated rather than underestimated. This code is reliable for correcting single error but fails to correct multiple errors. Shorter codeword reduces the likelihood of encountering multiple errors in a channel with fixed error rates.

Q6: Accuracy of the learned IDS channel.

A6: We would like to thank the reviewer for this insightful comment. We evaluated the accuracy of the learned IDS channel (DIDS) by comparing it to the groundtruth produced by the conventional IDS channel (CIDS). Part of the results are:

It suggests that the learned channel is reliable for simulating channels with error rate less than 20%.

We will include the detailed results from these experiments.

Q7: Theorem 3.1 is a bit disconnected to the rest of the paper, as sparsity is not discussed later on in the experiments.

A7: The entropy of codewords, which directly reflects sparsity or discreteness, was recorded in Sec 6.1 and App B to illustrate the effect of Thm 3.1. The reviewer may have missed these parts, which is our fault due to the small text used after zooming out the figures for page limit. We will revise this in the updated version.

All other suggestions will be taken into account.

We sincerely thank the reviewer for their valuable efforts. We will revise the manuscript accordingly. We hope our rebuttal has addressed the concerns.

Q1: The theorem is sometimes hard to follow.

A1: We will revise the main text of Theorem 3.1 for clarity.

Additionally, we will provide a full proof in the Appendix for general $\tau$ and $n$ logits, in about 1.5 extra pages. The full proof follows the structure of the sketchy proof but includes additional details and a trick involving the mean value theorem for multivariable functions.

Q2: Readers unfamiliar with the area might feel there is a slight lack of background on DNA storage.

A2: We agree with the reviewer’s concern. Initially, we had such a paragraph on introducing DNA storage before the second paragraph in the introduction, but it was removed due to page limits. We will reintroduce this paragraph explaining the DNA-based information storage pipeline to provide better background for readers unfamiliar with the area.

Q3: typos such as "distrubance", "apperent", etc.

A3: We apologize for these typos and will correct them in the revised version. Additionally, we will conduct a thorough review of the manuscript.