Chinese Inertial GAN for Writing Signal Generation and Recognition

1. For W1: Thank you for your constructive feedback on the visualization and clarity of the diagrams. In response, we have significantly improved Figure 1 to provide a more systematic and intuitive representation of our framework. The updated diagram now explicitly defines the key tasks and symbols, ensuring that the roles of each component, such as CGE, GAN, FOT, and SRA, are visually clear and aligned with their descriptions in the text.  Additionally, we have enhanced the representation of abstract concepts like constraints and regularization techniques by incorporating detailed annotations and visual cues. For example, we illustrate how FOT mitigates mode collapse and mode mixing with specific examples, and the semantic alignment enforced by SRA is clearly depicted to highlight its interaction with other components. We sincerely appreciate your suggestion and kindly invite you to review the revised figures and look forward to your feedback.

2. For W2: Unlike images, where the quality of generation can often be assessed visually, it is challenging to determine whether generated time-series signals are realistic or semantically correct. This necessitates the use of strong constraints like FOT to ensure the quality, diversity, and semantic accuracy of the generated signals. FOT achieves this by forcibly aligning the glyph encoding features, generated signal features, and real signal features using the Wasserstein distance. This alignment ensures both semantic correctness and motion fidelity in the generated signals, effectively mitigating mode collapse and mode mixing.

   To further address your concern, we have added a new section in the appendix titled Mathematical Explanation of FOT for Preventing Mode Collapse. In this section, we provide a rigorous mathematical derivation to demonstrate how FOT mitigates mode collapse. Briefly, FOT preserves the diversity of generated signals by penalizing incomplete mode coverage and mode mixing. We kindly invite you to review this section, which we believe offers a solid theoretical foundation for the effectiveness of FOT in addressing this critical issue.

3. For W3: To address your concern, we conducted additional experiments to thoroughly evaluate the system's robustness under external disturbances, specifically by introducing varying levels of Gaussian noise to the real inertial signals during training. The Gaussian noise was added at proportions of 0.0%, 5.0%, 10.0%, and 20.0% of the original signal's standard deviation to simulate sensor inaccuracies and environmental interference. Under different levels of Gaussian noise added to the real inertial signals, we trained CI-GAN models to generate 15,000 IMU signals for each noise setting. These generated signals were then used to train six classifiers (1DCNN, LSTM, Transformer, RF, XGBoost, and SVM), and their classification accuracy was evaluated using 5-fold cross-validation. The results, presented in the table below, reflect the accuracy of the classifiers under varying noise conditions. |Noise ratio|1DCNN|LSTM|Transformer|RF|XGBoost|SVM|
   |-|-|-|-|-|-|-| |0.0%|95.7%|93.9%|98.4%|83.5%|93.1%|74.6%| |5.0%|95.2%|94.1%|98.0%|82.9%|93.3%|71.8%| |10.0%|94.5%|92.3%|97.1%|81.7%|92.6%|70.7%| |20.0%|93.9%|92.5%|95.9%|79.8%|91.0%|69.4%|

   These results demonstrate that the system maintains high performance even under significant noise levels. While performance slightly decreases with higher noise ratios, the overall degradation is minimal. This robustness is attributed to the combined contributions of Glyph Encoding Regularization (GER), Forced Optimal Transport (FOT), and Semantic Relevance Alignment (SRA). CGE introduces a regularization term based on Rényi entropy, which is the first embedding targeted at the shape of Chinese characters rather than their meanings, providing rich semantic guidance for generating handwriting signals. FOT establishes a triple-consistency constraint between the input prompt, output signal features, and real signal features, ensuring the authenticity and semantic accuracy of the generated signals and preventing mode collapse and mixing. SRA constrains the consistency between the semantic relationships among multiple outputs and the corresponding input prompts, ensuring that similar inputs correspond to similar outputs (and vice versa), significantly alleviating the hallucination problem of generative models. Together, these components ensure the system's resilience to external disturbances and its capacity to generate realistic and accurate signals under challenging scenarios.

We sincerely thank you for your thoughtful and constructive feedback, which has greatly helped us improve the rigor and robustness of our work. Your recognition would mean a great deal to us, and we truly hope that our revisions meet your expectations.

Thank you for your review of our paper. We have distilled your comments into four key points and addressed each one individually.

1. Review Comment: Comparing CI-GAN with other high-performing classification models, such as in reference [1].

   Response: CI-GAN is a generative model, not a classification model. CI-GAN can generate high-quality inertial measurement unit (IMU) signals for Chinese handwriting recognition, but can not classify or recognize handwriting characters. Therefore, comparing a generative model to classification models may be a methodological misunderstanding. We acknowledge that the classification and recognition methods you suggested are excellent, and we will cite these references to highlight their contributions. However, as a generative model, CI-GAN is fundamentally different from classification models, making a direct comparison with them highly impractical.

2. Review Comment: CI-GAN generation effect can be verified on other open source datasets of Chinese character data, such as IAHCC-UCAS2016, CASIA-OLHWDB, and ICDAR 2013.

   Response: The IAHCC-UCAS2016, CASIA-OLHWDB, and ICDAR 2013 datasets are used for handwriting recognition tasks, based on visual or pen-tip trajectory data. CI-GAN is designed for generating IMU handwriting signals rather than recognizing them, so it would be challenging to evaluate a generative model on a classification dataset.

3. Review Comment: The dataset collected is small in scale, with data from only nine individuals and without full coverage of the complete Chinese character set.

   Response: We would like to clarify that IMU signal data collection is inherently challenging and considerably more complex than collecting visual data. Handwriting signals are continuous, and therefore each segment corresponding to a specific character must be extracted from the continuous stream of handwriting signals. For images, videos, or pen trajectories, such segmentation is relatively straightforward due to visual cues. However, for IMU signals, which are time-series waveforms, it is extremely difficult to identify the start and end points of each character segment visually, requiring auxiliary optical equipment for precise annotation.

   We invested significant time and effort to obtain the 4,500 handwriting signals presented in this study, creating the first IMU dataset of Chinese handwriting that covers the official set of commonly used characters in China. This effort underscores the value of our CI-GAN model, which eliminates the need for such labor-intensive annotation by directly generating IMU signals for each Chinese character. This dataset sufficiently supports the training of our generative model for the IMU signal generation task, and our experiments have validated the practical effectiveness of CI-GAN on this scale and quality of data.

4. Review Comment: Comparing the CI-GAN with other generative models.

   Response: It is important to highlight that this study is the first to propose a generative deep learning model for generating IMU handwriting signals. CI-GAN is specifically designed to address the unique challenges of IMU handwriting signal generation, with tailored modules and optimization strategies to suit IMU data. In the field of IMU signal generation research, there is currently no precedent or comparable model, meaning that no readily available generative model exists for direct comparison.

   Therefore, we conducted a thorough comparison of CI-GAN with twelve commonly used data augmentation methods spanning five major categories. This comprehensive evaluation demonstrates the rigor of our approach and the scientific validity of our results. While you suggested that we use Diffusion models or other image-based generative models for comparison, these models were originally designed for image data. Applying them directly to IMU signal generation would involve technical and theoretical misalignment. Adapting such image-based generative models to IMU signal generation would require substantial modifications to both model structure and algorithms, along with re-training and validation on IMU data. This adaptation alone would justify an entirely new research paper, extending far beyond the scope of our current study.

In conclusion, we sincerely hope that our replies address your concerns satisfactorily. Your understanding and recognition of our efforts are of utmost importance to us.

1. For W1: Thank you for your insightful suggestion regarding the introduction of the concept of inertial data earlier in the manuscript. In response, we have revised the paper to include a clear and concise explanation of the advantages of inertial sensors and their applications in IMU-based human-computer interaction systems right at the beginning. This sets a strong foundation for understanding the study's context. We kindly invite you to review the updated manuscript and look forward to your valuable feedback.

2. For W2: We have revised the contributions summary to clarify the scope of our research, ensuring it aligns accurately with the study’s focus and avoids any potential misinterpretation.

3. For W3: Thank you for your feedback. In response to your suggestion, we have streamlined the description of CGE in Section 3.1 to enhance readability and reduce complexity. At the same time, we want to emphasize that CGE is fundamentally different from a standard embedding. Unlike traditional embeddings that primarily encode semantic meanings, CGE captures the glyph-specific features of Chinese characters, such as shape, structure, and writing strokes, by leveraging the inherent relationship between the character glyph and its writing motion recorded by inertial sensor signals. Additionally, the Rényi entropy-based regularization we designed ensures that the encoding vectors are orthogonal and maximally informative, which not only strengthens the quality of glyph representations but also provides a generalizable mechanism that could benefit other representation learning tasks. This innovative approach goes beyond conventional embeddings, making CGE a key contribution of our framework. We kindly invite you to review the revised section.

4. For Q1: In our original ablation experiment, removing CGE meant eliminating both two parts. To address your concern, we have conducted an additional ablation experiment where the first part (converting one-hot encoding into dense features) is retained, while only the second part (GER) is removed. |Ablation Model|1DCNN|LSTM|Transformer|RF|XGBoost|SVM| |-|-|-|-|-|-|-| |No augmentation|0.87%|2.6%|1.7%|4.9%|1.2%|6.7%| |w/o all (Base GAN)|18.5%|14.8%|15.7%|12.4%|20.5%|8.4%| |w/ OT|26.4%|28.6%|27.3%|21.0%|30.9%|20.9%| |w/ FOT|39.9%|38.0%|35.3%|31.9%|46.8%|27.3%| |w/ CGE|54.6%|51.2%|47.9%|38.6%|57.5%|34.1%| |w/ CGE (w/o GER)|35.7%|32.1%|30.9%|33.8%|41.1%|29.0%| |w/ CGE (w/o GER)+SRA|61.4%|58.1%|60.2%|51.0%|59.9%|45.2%| |w/ CGE (w/o GER)+FOT|59.6%|55.2%|54.0%|53.4%|58.3%|47.5%| |w/ CGE+SRA|84.9%|77.4%|86.8%|61.4%|68.9%|56.1%| |w/ CGE+FOT|80.7%|80.5%|80.9%|57.2%|70.4%|59.5%| |w/ CGE+FOT+SRA (CI-GAN)|95.7%|93.9%|98.4%|83.5%|93.1%|74.6%|

   The results, now included in the revised manuscript, demonstrate the significant impact of GER on the performance of the framework. Specifically, we observe that retaining the dense feature transformation without GER still improves performance over the baseline GAN, but the lack of regularization results in noticeably lower effectiveness compared to using CGE with GER fully enabled. This confirms the critical role GER plays in enhancing glyph encoding by ensuring orthogonality and maximizing the information entropy of the encoding vectors.

5. For Q2: The pre-trained VAE and CGE serve fundamentally different roles in the framework and cannot be substituted for one another. The VAE is designed to extract features from inertial sensor signals, focusing on capturing signal-specific characteristics. In contrast, CGE is designed to encode the categorical features of Chinese characters. In essence, the VAE operates on the signal space, learning to represent the temporal and motion characteristics of IMU data, while CGE works in the character space, embedding class-level information that distinguishes one glyph from another.

6. For Q3: $h_T$ represents the real signal feature, $h_G$ denotes the generated signal feature, $e$ is the glyph encoding derived from the CGE module, which encodes glyph-related features. During training, $h_T$ is extracted from real IMU signals in the dataset using the pre-trained VAE, providing the ground-truth feature representation for supervising the generator. The Forced Feature Matching (FFM) loss aligns $h_T$, $h_G$, and $e$, ensuring that the generated signals reflect both the motion dynamics of real IMU data and the glyph-specific semantics of the target character.

Thank you so much for your insightful and constructive feedback. It’s clear that you have a deep understanding of the field, and the detailed suggestions you provided have been incredibly helpful in improving the quality of our work. Your recognition is extremely important to us, and we truly appreciate the thought and effort you’ve put into reviewing our paper.

Thank you for your recognition of our work, especially your positive feedback on the motivation, experiments, and writing of our paper! Regarding the two weaknesses you mentioned, we provide the following responses:

1. Weaknesses: The dataset is relatively small and lacks comprehensive coverage of the Chinese character set.

   Response: We would like to clarify that collecting high-quality inertial sensor (IMU) handwriting signals is inherently challenging due to the nature of IMU data. Unlike visual data, IMU signals are continuous time-series waveforms, making it difficult to segment and label individual characters without auxiliary tools like optical tracking devices. Despite these challenges, we successfully collected a dataset of 4500 IMU signal samples, covering “Commonly Used Chinese Characters List” published by the Chinese government.

   While our dataset does not encompass every Chinese character, our model has learned the structural relationships and shape patterns between different character, as evidenced by the t-SNE visualizations where characters with similar structures and stroke patterns cluster closely together. As most complex Chinese characters are constructed by combining simple elements, our work represents a foundational step from 0 to 1 in this field.

2. Weaknesses: The proposed method should be tested on other public available benchmarks with other SOTA methods, such as IAHCC-UCAS2016 and CASIA-OLHWDB.

   Response: Our CI-GAN is designed to generate handwriting signals captured by inertial sensors. However, the IAHCC-UCAS2016 dataset and the CASIA-OLHWDB dataset were collected by entirely different sensors. The IAHCC-UCAS2016 dataset relies on the Leap Motion, a vision-based optical device that captures hand trajectories in the air, while the CASIA-OLHWDB dataset uses an Anoto digital pen to record pen-tip trajectories data on specialized paper. Given the fundamental differences in data modalities, these datasets are not suitable for evaluating CI-GAN tailored for inertial sensor signal generation.

   In fact, there is currently no publicly available handwriting dataset based on inertial sensors, highlighting a significant gap in this field. Our work addresses this gap by creating CI-GAN to generate an infinite number of inertial-sensor-based handwriting signals.

Thank you for your insightful comment. We hope we have addressed your concerns.

1. For W1: Thank you for pointing out the inconsistencies and omissions in the original Figure 1. We have unreservedly revised Figure 1 according to your suggestions. Specifically, "CGE" is now correctly labeled instead of "GER," the "SRA" module has been visually represented to provide a complete overview of the framework, and all modules now include both their full names and abbreviations. We kindly invite you to review our updated manuscript.

2. For W2&Q2: Thank you for your suggestion. We have provided a rigorous mathematical proof demonstrating how FOT imposes strong constraints in the feature space to effectively mitigate the mode collapse problem in GANs. Due to page limitations in the main text, we included this proof in Appendix D. We kindly invite you to review the supplemental mathematical analysis and sincerely hope that this revision addresses your concerns.

3. For W3&Q3: Thank you for your valuable feedback regarding the ablation studies. In response, we have expanded our experiments to include all possible combinations of CGE, FOT, and SRA, thoroughly exploring their individual and collective contributions. It is worth noting that SRA relies on input semantics and therefore must be used alongside CGE in this framework. As shown in the results, the addition of each module consistently improves performance across all tested base models, regardless of which modules are already included, demonstrating that each module contributes unique and complementary strengths to the framework. |Ablation Model|1DCNN|LSTM|Transformer|RF|XGBoost|SVM| |-|-|-|-|-|-|-| |No augmentation|0.87%|2.6%|1.7%|4.9%|1.2%|6.7%| |w/o all (Base GAN)|18.5%|14.8%|15.7%|12.4%|20.5%|8.4%| |w/ OT|26.4%|28.6%|27.3%|21.0%|30.9%|20.9%| |w/ FOT|39.9%|38.0%|35.3%|31.9%|46.8%|27.3%| |w/ CGE|54.6%|51.2%|47.9%|38.6%|57.5%|34.1%| |w/ CGE (w/o GER)|35.7%|32.1%|30.9%|33.8%|41.1%|29.0%| |w/ CGE (w/o GER)+SRA|61.4%|58.1%|60.2%|51.0%|59.9%|45.2%| |w/ CGE (w/o GER)+FOT|59.6%|55.2%|54.0%|53.4%|58.3%|47.5%| |w/ CGE+SRA|84.9%|77.4%|86.8%|61.4%|68.9%|56.1%| |w/ CGE+FOT|80.7%|80.5%|80.9%|57.2%|70.4%|59.5%| |w/ CGE+FOT+SRA (CI-GAN)|95.7%|93.9%|98.4%|83.5%|93.1%|74.6%|

   We believe these additional experiments provide a clearer understanding of the contributions of CGE, FOT, and SRA. Thank you again for your insightful suggestions. We kindly invite you to review the updated manuscript.

4. For W4&Q1: We appreciate your suggestion and would like to clarify the positioning of our study and its broader applications. Our work primarily focuses on the development of a robust IMU signal generation algorithm, which can produce a large volume of high-quality inertial signals. These generated signals enable IMU-based human-computer interaction systems, offering significant advantages for accessibility, particularly for individuals with visual impairments. For example, by facilitating natural handwriting interactions, our algorithm has already contributed to the production of devices designed specifically for visually impaired users. That said, aiding disabled individuals is just one of many application scenarios for our algorithm. By enabling the generation of diverse and high-quality IMU handwriting signals, it supports the development of IMU-based systems in education, digital handwriting analysis, and personalized training for handwriting recognition algorithms. These applications demonstrate the algorithm’s potential to revolutionize human-computer interaction by providing high-fidelity motion data.  To better reflect this breadth, and considering the comments from reviewer a2VT, we have revised the manuscript to introduce the advantages of inertial sensors and IMU-based human-computer interaction systems at the very beginning. In this context, we present the example of assisting disabled individuals as a representative case, highlighting it as one key motivation but not the sole focus of our study.

We sincerely thank you for your valuable feedback, and we hope that our revisions have adequately addressed your concerns. Your recognition is truly invaluable to us and we look forward to your response and further insights.

I hope this message finds you well. Considering the discussion phase has now been ongoing for 10 days, and I have yet to receive any response. This has left us feeling increasingly anxious about the status of our submission. This paper holds immense importance to us, and we have devoted considerable effort to meticulously address every concern and suggestion raised in the initial reviews. Given the contribution of our work in advancing the fields of AI for sensors, and its potential impact on improving human-computer interaction for individuals with disabilities, we are eager to receive the reviewers' feedback on our responses.

We sincerely believe that the experimental results and the detailed explanations provided in our response have adequately resolved the issues you pointed out. This work is not only a critical part of our research but also has the potential to make a meaningful contribution to the field.

I humbly and earnestly request you to kindly review our responses and share your feedback at your convenience. Your input is invaluable, and I sincerely hope that our diligent efforts will meet your expectations.

Thank you so much for your understanding and for taking the time to support us in this process.