Does Vector Quantization Fail in Spatio-Temporal Forecasting? Exploring a Differentiable Sparse Soft-Vector Quantization Approach

Dear Reviewer 1ZsW,

We sincerely thank your thoughtful insights and recognition of our efforts in exploring VQ's deterioration effect in spatio-temporal forecasting and for proposing SVQ. Your support and endorsement of our work truly inspire us to continue pursuing this research direction.

[W1] Visual Layout: Perhaps due to space constraints, the layout of the figures and tables could be more aesthetically pleasing.

We acknowledge that the original arrangement might have felt constrained due to space limitations. In response, we have carefully revised the layout to make it more visually appealing. The revised version of our paper is available for download.

[Q1] The paper emphasizes the advantages of the proposed method in spatio-temporal forecasting, but VQ is also widely used in generative tasks (e.g., VQ-VAE). Could this method be applied to such tasks?

We anticipate that our proposed method will likely improve image generation due to its broad applicability in addressing non-differentiability and representation issues that are generally unrelated to the design of base models. We are eager to test our method in domains such as image generation, multimodal LLMs, and other application scenarios where existing VQ blocks provide a natural substitution. However, many of these tasks are extremely GPU-intensive, and we currently lack the resources to pursue them. Spatio-temporal forecasting, on the other hand, is an application that does not require tens or hundreds of GPUs, making it a viable option for exploration.

The OpenSTL [1] benchmark encompasses three primary categories: weather forecasting, video forecasting, and traffic forecasting. In the realm of video forecasting, there is a strong focus on evaluating the quality of output images, moving beyond traditional metrics like MSE and MAE, which have been the standard measures in both weather and traffic forecasting. Our work demonstrates improvement across all three tasks in terms of MSE/MAE metrics as well as image quality metrics. For instance, using the LPIPS metric, SVQ demonstrates enhancements of 26.2%, 3.6%, and 22.2% on three video forecasting datasets: Human3.6M, KittiCaltech, and KTH. We believe that our proposed method holds great potential for generative tasks.

[1] Tan C, Li S, Gao Z, et al. Openstl: A comprehensive benchmark of spatio-temporal predictive learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 69819-69831.

[Q2] In the experimental section, I noticed that the baseline models employed are relatively outdated. Given the recent advancements in spatio-temporal forecasting, particularly the emergence of various diffusion-based methods that have demonstrated significant improvements in predictive performance, it would be beneficial for the authors to consider incorporating these state-of-the-art models as baselines.

Thank you for your suggestion. We are currently adhering to the OpenSTL benchmark [1], the ongoing benchmark for spatio-temporal forecasting. In light of this, we would like to include three additional baselines from the past two years: MMVP (ICCV 2023) [2], SwinLSTM (ICCV 2023) [3], and WaST (AAAI 2024) [4]. MMVP and WaST are evaluated on WeatherBench-S dataset. Due to SwinLSTM only supports H = W for image input, we evaluate it on TaxiBJ dataset. The results are presented in the table below. SVQ consistently enhances the performance of these baselines. The results are presented in the table below. SVQ consistently enhances the performance of these baselines. Furthermore, given the general nature of our proposed method, which addresses non-differentiability and representation issues typically orthogonal to base model design, we believe it is likely to enhance future methodologies.

Table: Additional baselines on WeatherBench-S dataset

Table: Additional baselines on TaxiBJ dataset

[1] Tan C, Li S, Gao Z, et al. Openstl: A comprehensive benchmark of spatio-temporal predictive learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 69819-69831.

[2] Zhong, Yiqi, et al. "Mmvp: Motion-matrix-based video prediction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[3] Tang, Song, et al. "Swinlstm: Improving spatiotemporal prediction accuracy using swin transformer and lstm." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[4] Nie, X., et al. "Wavelet-Driven Spatiotemporal Predictive Learning: Bridging Frequency and Time Variations." Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(5): 4334-4342.

[Q3] The explanation of the quantization module's implementation lacks depth regarding its integration with the overall spatio-temporal forecasting model. While the authors outline the architecture and components involved, they do not provide sufficient details on how the quantization process interacts with other model elements or influences the final forecasting results. A more thorough exploration of these interactions would enhance the clarity and applicability of their proposed method.

We concur with the review's observation regarding the integration of the baseline method and the VQ method. Our goal is for our method to enhance various baseline methods, and digging into each baseline's interaction may not seem universally applicable. However, we fully understand your inquiry, as we intended to investigate this aspect during the project. Consequently, we explore the positioning issue of VQ in Section 5.4. Our findings indicate that performing quantization before the translator provides a significant optimization advantage, resulting in a smoother learning curve and more accurate outcomes compared to quantization after the translator. This suggests that our SVQ design should be closer to the original input prior to the baseline algorithm's representation learning module, indicating that a weak and pre-processed VQ block is sufficient to enhance performance without strong interaction with the base model.

Dear Reviewer AkZ2,

We sincerely appreciate the time and effort you have taken to provide such an insightful review. Your constructive feedback is invaluable to us. Below, we would like to address each point raised.

[Q1] The method does not adequately demonstrate how the theoretical advantages of sparse regression are translated into tangible improvements in quantization performance. While the authors discuss optimization strategies, they fail to provide a clear connection between these theoretical claims and the practical outcomes. A more comprehensive explanation of how these optimization techniques directly enhance quantization would bolster the credibility of their approach. In Section 4, the mathematical proof lacks clarity in the notation used, which may hinder readers' understanding. For example, what’s the meaning of g' after Eq. (8)? Additionally, the proof does not establish a strong connection to the problem being addressed. I recommend revising this section to improve the clarity of the symbols and to explicitly link the proof to the main objectives of the paper.

Our theoretical analysis shows how the proposed VQ mimics the sparse regression model (i.e. ReLU and two layers in MLP correspond exactly to thresholding and quadratic dependence on fixed dictionary in sparse regression), and thus enjoys the advantage of sparse regression (i.e. error depends on $\log d$ not $d$), making it surpass traditional cluster-based VQ methods. Another key insight is that a frozen codebook can be nearly as effective as a trainable one, as long as it is sufficiently large. Although it may appear counterintuitive, our proof is based on a simple fact that any vector in a simplex can be well approximated by a sparse combination of vectors from a frozen and large dictionary. Thus, when using high-dimensional vectors, a sparse representation can be more expressive than a dense one in lower-dimensional space. Furthermore, for high-dimensional vectors, a basis can often be randomly sampled due to the high probability of orthogonality, eliminating the need for it to be learned. I hope this helps clarify the theoretical analysis, as these concepts are crucial for addressing the underlying 'why' questions.

What we tried to show in the paper is that the first iteration of the sparse regression algorithm can be very implemented by a two layer MLP, where the nonlinear ReLU function in MLP corresponds to the thresholding used in the sparse regression algorithm, and the two layers corresponds to the quadratic dependence on dictionary matrix $Z$ in the algorithm. All the mathematical equivalence is illustrated in Eq. (2) and (3). Finally, it is not difficult to observe that we can simulate running the sparse regression algorithm with more than one step by introducing more layers (of course, have to be even number of layers) in MLP.

g is an s-sparse unit vector, we use it to show the covering number. g' after Eq. (8) is a sample of the vectors from the set that densely covers the unit ball of s-sparse vectors such that any s-sparse vector g can be closely approximated within this covering set. This is relevant to the discussion of vector covering numbers, which aim to cover a space with vectors such that every point in the space is close to at least one of these vectors. To bound $T(\mathcal{B}, \delta)$ for the clustering method, we use the covering number for a unit ball $\mathcal{B}$, requiring at least $1/\delta^d$ code vectors to approximate any vector within error $\delta$.

Dear reviewer AkZ2,

We sincerely appreciate your time and effort in reviewing our work and offering valuable feedback. As the discussion period is nearing its end, we would like to confirm whether our responses have effectively addressed your concerns.

If you require further clarification or have any additional concerns, please do not hesitate to contact us! We are more than willing to continue our communication with you. More discussions and suggestions on further improving the paper are also always welcomed!

Best regards,

The Authors

[Q7] Why are there different types of comparison results between WeatherBench-S and WeatherBench-M in Tab. 1 (Total Cloud Cover in WeatherBench-S and Wind UV in WeatherBench-M)? Why not compare the same subjects?

Our work aligns with the NeurIPS 2023 benchmark paper, OpenSTL [1]. OpenSTL provides both single-channel and multi-channel forecasting options for WeatherBench. WeatherBench-S involves predicting at 1-hour intervals, while WeatherBench-M operates at 6-hour intervals and larger dataset. This setup is defined within OpenSTL to evaluate models across a broader range of temporal and spatial complexities. Therefore, comparing the same subjects across these two datasets would deviate from the benchmark.

[1]Tan C, Li S, Gao Z, et al. Openstl: A comprehensive benchmark of spatio-temporal predictive learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 69819-69831.

[Q8] How are the differences in SVQ performance across different physical quantities and data scales?

We conducted a high-dimensional experiment detailed in Appendix D.8, Table 19, using 110 weather covariates to demonstrate the effect on various physical quantities. Of these, 109 showed positive improvement with the addition of SVQ, with an average improvement of over 8.9%. These findings underscore that despite the complexities introduced by high-dimensional datasets, SVQ effectively adapts to and enhances the predictive capabilities of the backbone model. Notably, we observed significant improvements in channels such as temperature at different heights, geopotential height, and potential vorticity. The improvements were particularly pronounced for high altitude features associated with low pressure height, while surface weather features exhibited comparatively lower improvements.

Since all variables are normalized before being input into the prediction algorithm, we do not observe a clear difference between channels with varying scales. However, we are not entirely sure if the question pertains to channel scale or dataset size. Therefore, we conducted additional few-sample experiments using the Weatherbench-S-temperature dataset, with SimVP as our baseline model:

The results show that SVQ consistently enhances its performance across different percentages in few-sample experiments. When 10% of the data is used, the improvement is about 5%. This enhancement stabilizes at approximately 12% as the data percentage increases from 30% to 90%.

Question about some details

[Q5] What does the "improvement" in Table 1.2.3 refer to, SimVP?

Yes, that's compared to its SimVP backbone without adding SVQ.

Perhaps there may be some misunderstanding regarding our method's performance. For example, as noted in Weakness 1: "slight enhancements ... in the KTH dataset in Table 2," our SVQ method actually leads to significant improvements in the KTH dataset results, with a 9.3% improvement in the MAE metric and a 22.0% improvement in the LPIPS metric. The related results are presented below.

Table 2: Results on KTH Dataset

[Q6] Compared to diffusion methods.

We concur with the knowledgeable reviewer that diffusion models are achieving state-of-the-art results in video forecasting tasks. Our work aligns with the NeurIPS 2023 benchmark paper, OpenSTL [1], and follows its ongoing updates to evaluate spatio-temporal forecasting baselines. This task encompasses three main categories: weather forecasting, video forecasting, and traffic forecasting. Video forecasting often emphasizes the assessment of output image quality, going beyond traditional metrics such as MSE and MAE, which have historically been the primary measures in both weather and traffic forecasting. Therefore, we opted to follow this setting.

But we would like to include three additional baselines from the past two years which are included in the latest OpenSTL benchmark: MMVP (ICCV 2023) [2], SwinLSTM (ICCV 2023) [3], and WaST (AAAI 2024) [4]. MMVP and WaST are evaluated on WeatherBench-S dataset. Due to SwinLSTM only supports H = W for image input, we evaluate it on TaxiBJ dataset. The results are presented in the table below. SVQ consistently enhances the performance of these baselines. Given the general nature of our proposed method, which addresses non-differentiability and representation issues that are largely independent of base model design, we believe it is likely to enhance future methodologies. Further research may be required to explore the application of our proposed method in diffusion models, given the complex interactions between noise and signal inherent in the diffusion process. Our findings in Appendix D.3 suggest that applying quantization before the translator module offers a significant optimization advantage. This approach results in a smoother learning curve and more accurate outcomes compared to quantization applied after the translator module. Consequently, our SVQ design should remain closely aligned with the original input, prior to the baseline algorithm's representation learning phase. This implies that a minimal and pre-processed VQ block is adequate for enhancing performance without extensive interaction with the base model. Therefore, VQ may not naturally align with the diffusion algorithm.

Table: Additional baselines on WeatherBench-S dataset

Table: Additional baselines on TaxiBJ dataset

[1] Tan C, Li S, Gao Z, et al. Openstl: A comprehensive benchmark of spatio-temporal predictive learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 69819-69831.

[2] Zhong, Yiqi, et al. "Mmvp: Motion-matrix-based video prediction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[3] Tang, Song, et al. "Swinlstm: Improving spatiotemporal prediction accuracy using swin transformer and lstm." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[4] Nie, X., et al. "Wavelet-Driven Spatiotemporal Predictive Learning: Bridging Frequency and Time Variations." Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(5): 4334-4342.

[Q2] Discussion on computational cost.

Due to space constraints, the computational cost table has been relocated to Appendix Table 11. The introduction of our SVQ design results in a slight increase—less than 10%—in the number of learnable parameters and a moderate FLOPs increase of 30% to 70%, depending on the task. While this might initially seem significant, our experimental results suggest otherwise. Our fully differentiable SVQ design, implemented with an MLP, extends the per-epoch training time on a V100 GPU by only about 15% compared to the baseline with identical parameter settings but without VQ. Additionally, it offers a speed advantage over other VQ methods in most cases due to its differentiable nature. Consequently, it proves to be a highly efficient add-on block.

We would like to emphasize that the significant improvements observed here are largely attributable to the high efficiency of our chosen baseline model, SimVP. In our proposed method, the increase in learnable parameters and FLOPs is primarily determined by the codebook size and dimension, and is independent of the base model. If a more complex baseline with a larger parameter size and more FLOPs, such as ConvLSTM or PredRNN, or other recurrent-based models, were used, the increase in FLOPs would be much less than 2%. Below we show the parameter size and FLOPs on a more complex baseline: MMVP [1].

Table: Computational cost of SVQ

[1] Zhong, Yiqi, et al. "Mmvp: Motion-matrix-based video prediction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[Q3] Regarding novelty.

Our innovation lies in addressing a challenge that no previous state-of-the-art VQ method has tackled—enhancing spatio-temporal forecasting results. We acknowledge that there are diverse opinions on the novelty of architectural improvements and whether they result in marginal benefits for a given task. However, we aim to address a binary question: does our approach improve or hinder performance? Furthermore, we investigate why such binary outcomes occur, particularly focusing on non-differentiable problems. This is how we define and position our contribution.

[Q4] Significance of theoretical analysis.

Our theoretical analysis shows how the proposed VQ mimics the sparse regression model (i.e. ReLU and two layers in MLP correspond exactly to thresholding and quadratic dependence on fixed dictionary in sparse regression), and thus enjoys the advantage of sparse regression (i.e. error depends on $\log d$ not $d$), making it surpass traditional cluster-based VQ methods. Another key insight is that a frozen codebook can be nearly as effective as a trainable one, as long as it is sufficiently large. Although it may appear counterintuitive, our proof is based on a simple fact that any vector in a simplex can be well approximated by a sparse combination of vectors from a frozen and large dictionary. Thus, when using high-dimensional vectors, a sparse representation can be more expressive than a dense one in lower-dimensional space. Furthermore, for high-dimensional vectors, a basis can often be randomly sampled due to the high probability of orthogonality, eliminating the need for it to be learned. I hope this helps clarify the theoretical analysis, as these concepts are crucial for addressing the underlying 'why' questions.

Dear Reviewer PA7X,

Thank you for your thorough analysis and constructive feedback on our paper. Before addressing the specifics of your questions, we would like to provide some context regarding the weaknesses you highlighted. We are grateful to you for raising these important points, as they were considerations we also had before commencing our project. Allow us to explain them in detail below.

[Q1] Regarding the motivation.

As you insightfully pointed out, it might seem intuitive that using an extensive codebook with numerous parameters would improve results. Initially, we shared this assumption. However, our findings reveal that this belief does not hold true for the majority of state-of-the-art VQ methods, as demonstrated in Table 4 and Figure 5 on page 8 of our paper (we put the table below to highlight as well). We conducted experiments by varying the size of the codebook, from small to large, and found that none led to improved outcomes. Although a larger codebook size results in less deterioration, it does not enhance results. This has led us to conclude that there is an underlying issue that needs attention, and merely increasing the codebook size does not result in improvements compared to the non-VQ counterpart.

Table 4: Comparison of vector quantization methods

You are absolutely correct in noting that VQ compresses redundant information. In fact, we view VQ as a process that eliminates noise and retains the signal. We believe it is a general approach that can be applied to enhance prediction performance by improving the signal-to-noise ratio. Typically, the VQ method is effective in achieving this.

What we found is that the non-differentiability of VQ introduces quantization error, or optimization error, during model training. This creates a trade-off between the deterioration caused by quantization error and the improvement achieved by VQ's ability to compress redundant information. To address this, we designed a differentiable VQ method that minimizes quantization error while focusing on enhancing prediction performance by increasing the signal-to-noise ratio.

Another crucial observation is that the specific parameters, or learnable parameters, aren't as significant as one might expect. We conducted a frozen codebook experiment, which demonstrated that a randomly initialized frozen codebook almost achieved the same level of improvement as a learnable one as shown in Tables 1 and 2 (we put the table below). This finding supports the previous belief that parameters are not the key factor, and that merely increasing the size of the codebook doesn't necessarily enhance performance.

Table 1: WeatherBench-S results

Table 2: Video prediction results

Dear reviewer PA7x,

Thank you very much for taking the time to review our work and for providing valuable feedback. We were wondering if our responses have resolved your concerns.

We will be happy to have further discussions if there are still some remaining questions! More discussions and suggestions on further improving the paper are also always welcomed!

We look forward to hearing from you and remain available to provide any additional details that might assist in resolving your concerns.

Best regards,

The Authors

Dear reviewer PA7x,

Thank you so much for your response. We completely understand and appreciate your decision to maintain the score, and we are grateful for the time you spent reviewing our work and helping us improve it. We're pleased to hear that our additional experiment addressed some of your concerns.

In our experiments, video prediction is indeed not the only task. As our work follows OpenSTL benchmark, which includes three tasks: weather forecasting, traffic forecasting, and video prediction. We agree with you that video prediction, compared to the other two tasks, tends to focus more on the quality of video generation, specifically how realistic the generated images are. On the other hand, the weather and traffic forecasting tasks, which use MSE/MAE metrics, may align more closely with our design approach—denoising and preserving the signal. On this point, we are aligned with your perspective.

Dear reviewer PA7x,

We sincerely appreciate the time and effort you have dedicated to reviewing our work and for raising insightful concerns. We would like to take this opportunity to further clarify some possible misunderstanding.

[1] "In other words, the common and unremarkable approach of enhancing performance through increased parameters and computation applies here as well. "

It might seem intuitive that using a large codebook with numerous parameters would naturally lead to improved results. Initially, we shared this assumption. However, our findings demonstrate that this belief does not hold true for the majority of state-of-the-art VQ methods, as evidenced by Table 4 and Figure 5 on page 8 of our paper (as previously noted in our first rebuttal). We conducted experiments by varying the size of the codebook, from small to large, and found that none led to improved outcomes.

Therefore, the enhancements observed in our method cannot be attributed significantly, if at all, to increased parameters or computational overhead. Instead, the improvements are primarily attributed to two key factors: first, the differentiable SVQ design, which minimizes quantization error, and second, the enhanced representation power achieved through sparse regression-based VQ.

[2] "However, what specific challenges in video prediction necessitate the use of VQ or its differentiable variant, SVQ?"

Our primary motivation lies in leveraging VQ as a process to reduce noise while preserving the signal. We view this as a general approach applicable to a variety of spatio-temporal forecasting tasks to enhance prediction performance by improving the signal-to-noise ratio. While video prediction task may often emphasize the quality of video generation, SVQ is designed with broader applicability in mind. It aims to address challenges across diverse spatio-temporal forecasting tasks. These tasks frequently require a balance between denoising and preserving meaningful details, a need that aligns closely with the core motivation behind SVQ. Our experimental results in weather forecasting and traffic forecasting further validate this point.

If you need further clarification or have any additional questions, please don’t hesitate to contact us. We are more than willing to continue the discussion.

Best regards,

The Authors

[Q3] Will the SVQ module added as a plug-in to the model have similar performance improvements for other tasks (such as image generation or natural language processing)? What are the applicable application scenarios?

We anticipate that our proposed method will likely improve image generation due to its broad applicability in addressing non-differentiability and representation issues that are generally unrelated to the design of base models. While image generation seems promising, the impact on NLP might differ, as most LLMs utilize quantization based on the natural segmentation of words without VQ blocks. We are eager to test our method in domains such as image generation, multimodal LLMs, and other application scenarios where existing VQ blocks provide a natural substitution. However, many of these tasks are extremely GPU-intensive, and we currently lack the resources to pursue them. Spatio-temporal forecasting, on the other hand, is an application that does not require tens or hundreds of GPUs, making it a viable option for exploration.

Dear Reviewer pbDr,

We would like to express our gratitude to you for championing our work and for the thoughtful review, particularly in highlighting the weaknesses and questions section. We are committed to addressing these points and alleviating any concerns.

[Q1] The SVQ method proposed in the manuscript still requires a lot of computing resources, especially in the case of high-dimensional data and large-scale codebooks.

Thank you for taking the time to examine the appendix section and verify the computational resource results. It might be beneficial to include these findings in the main section, as they were overlooked by another reviewer. You are correct in noting that SVQ increases the FLOPs. The introduction of our SVQ design results in a slight increase—less than 10%—in the number of learnable parameters and a moderate increase in FLOPs, ranging from 30% to 70%, depending on the task.

We would like to emphasize that the significant improvements observed here are largely attributable to the high efficiency of our chosen baseline model, SimVP. In our proposed method, the increase in learnable parameters and FLOPs is primarily determined by the codebook size and dimension, and is independent of the base model. If a more complex baseline with a larger parameter size and more FLOPs, such as ConvLSTM, PredRNN, or other recurrent-based models, were used, the increase in FLOPs would be much less than 2%. Below we show the parameter size and FLOPs on a more complex baseline: MMVP [1]. Besides, our ablation experiments have shown that the codebook size can remain relatively small while still yielding significant improvements, so a smaller codebook size can be chosen if that's an issue.

Table: Computational cost of SVQ

[1] Zhong, Yiqi, et al. "Mmvp: Motion-matrix-based video prediction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[Q2] The comparison methods cited by the author in Tables 1 and 2 are only up to date in 2022, and lack comparisons of the latest methods in the past two years.

Thank you for your suggestion. We are currently adhering to the OpenSTL benchmark [1], the ongoing benchmark for spatio-temporal forecasting. In light of this, we would like to include three additional baselines from the past two years: MMVP (ICCV 2023) [2], SwinLSTM (ICCV 2023) [3], and WaST (AAAI 2024) [4]. MMVP and WaST are evaluated on WeatherBench-S dataset. Due to SwinLSTM only supports H = W for image input, we evaluate it on TaxiBJ dataset. The results are presented in the table below. SVQ consistently enhances the performance of these baselines. Furthermore, given the general nature of our proposed method, which addresses non-differentiability and representation issues typically orthogonal to base model design, we believe it is likely to enhance future methodologies.

Table: Additional baselines on WeatherBench-S dataset

Table: Additional baselines on TaxiBJ dataset

[1] Tan C, Li S, Gao Z, et al. Openstl: A comprehensive benchmark of spatio-temporal predictive learning[J]. Advances in Neural Information Processing Systems, 2023, 36: 69819-69831.

[2] Zhong, Yiqi, et al. "Mmvp: Motion-matrix-based video prediction." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[3] Tang, Song, et al. "Swinlstm: Improving spatiotemporal prediction accuracy using swin transformer and lstm." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.

[4] Nie, X., et al. "Wavelet-Driven Spatiotemporal Predictive Learning: Bridging Frequency and Time Variations." Proceedings of the AAAI Conference on Artificial Intelligence, 2024, 38(5): 4334-4342.

Thank you for taking the time to review and respond to our work. It's fine to keep the score. we're glad that our response contributes to a better understanding.