Major Concern:

• The explanation of how the asymmetric thresholds are optimized using the normal STE (Straight-Through Estimator) is not entirely clear. It would be beneficial for the authors to provide a more detailed derivation of the gradients for $\Delta_l^p$ and $\Delta_l^n$ and elaborate on the formulation of the bi-STE method. Additionally, it would be helpful to know whether the authors utilized stochastic ternarization or deterministic ternary function in their implementation.
• While the normalization of the distribution of Gaussian-like weights and activations using $\mu$ and $\sigma$, and the adjustment of variance with $\gamma$ seems reasonable, the additional multiplication by $Z_l$ appears somewhat unconventional. It would be valuable to understand if this technique was also applied to activation quantization during inference, and if so, it may introduce some additional computational costs.
• The introduction of the uniform distribution raises some questions about its necessity. Although the authors may alter the distribution of the original Gaussian through Equation (9), it should be noted that the product of two Gaussian random variables does not result in a uniform distribution. It may be worth considering modifying the threshold in Equation (8) to achieve the desired "approximately uniform quantization" of the corresponding parameters after regularization.

Minor Issue:

• The authors mentioned the utilization of RPReLU [1] to enhance the results, but it would be beneficial to have an ablation study to evaluate the impact of this setting. Additionally, comparing the results of two-stage training with other low-bit quantization methods may not provide a fair comparison. It is suggested to include an additional column in Table 1 and Table 2 to compare the training costs of different methods.

Reference:

• [1] Reactnet: Towards precise binary neural network with generalized activation functions. ECCV2020

1. The paper needs to provide an efficiency analysis and more concrete description on the ternary system. For example, what will the storage format be? Are quantized weights and activations still stored in 2-bits or a special ternary format? Are matmul accumulations (supposed to be high-precision) stored in 32-bit binary format or a special ternary format? These discussions will affect the memory efficiency and change the view of where TNN is positioned between BNN and 2-bit NNs.

2. Without efficiency analysis, e.g., estimated performance or memory savings coming from the ternary logic gates, it is hard to differentiate between a real “ternary” network and a 2-bit network. The experimental results in the paper show that the accuracy of ternary networks is close to that of 2-bit ones. In some context of symmetric integer quantization, as some of the baselines are, a 2-bit network {-1, 0, +1} is actually ternary. Readers cannot precisely evaluate the benefits of the proposed ternary models over the previous 2-bit models.

1). The primary concern is the extreme lack of novelty. Nearlyall the techniques introduced in the paper have been previously proposed in existing literatures, including the learnable asymmetric threshold, 2-stage KD and entropy-based regularization. In fact, the first reference in the paper introduced the asymmetric threshold and 2-stage KD, but the author only very briefly mentioned in the main context.

2). Evaluation is limited in scope. The evaluation is primarily done on ResNet model, which, while important, is somewhat outdated. It is expected that more state-of-the-art CNN and transformer models should be assessed to demonstrate the efficacy of these techniques.

3). The performance improvement is limited. Even with combined existing techniques and the utilization of optimized process (such as using RPReLU), the performance improvement is limited. Notably, the TNN ResNet shows worse results than 2-bit version, which has similar computational complexity in hardware implementation.

1. The proposed methods proposed an optimization method named bi-STE based on stochastic theory. But in equation 8, the deterministic ternary function in the forward pass set a hard threshold on the probability and now it’s not stochastic. It will be helpful to explain how to select hard thresholds that enable the variables in the model to be correctly quantized.
2. This paper argued that uniform distribution of weights/activations can enhance the information capacity of the model. However, it only used a few experiments in Table 3 to show it, which is not enough to support the conclusion. It would be better if authors could provide more mathematical proof or experiment results to demonstrate it.
3. Some figures in this paper are not clear enough (e.g., figure 2), and hard to find the corresponding line of your approach.
4. In the section 3.3, this paper assumes the weights follow a Gaussian distribution and correspondingly proposes their method. Hence, it will be better if the method can be more general and suitable for non-Gaussian distribution.