SPECTRUM: Empowering Online Handwriting Verification via Temporal-Frequency Multimodal Representation Learning

1. Some experiments indicated that the performance of SPECTRUM is suboptimal. For instance, in the evaluation on DeepSignDB (Table 3), SPECTRUM did not outperform previous methods, such as Sig2Vec and DsDTW, in certain aspects. The author did not discuss the reasons for SPECTRUM’s weaker performance on this dataset.
2. In the experimental process, the methods used for comparison are primarily from 2021 and 2022, which are somewhat outdated. The paper should discuss recent advancements in OHV methods and include comparisons with more recent approaches.
3. Some parameter choices are unclear, and additional ablation studies are needed. For example, why is $\lambda$ set to 0.1? Are there any ablation studies to support this choice? Similarly, how is the threshold $c$ determined, and are there any justifications or ablation studies provided? Moreover, the loss terms include $L_{intra}$, $L_{tri}$, and $L_{BCE}$; however, the paper does not explain why these particular losses were chosen or if any ablation studies were conducted to validate their selection.

• Scalability and Real-World Feasibility: Although SPECTRUM demonstrates high performance, the paper could further discuss computational efficiency, especially for large-scale OHV tasks or real-time applications. Time would be beneficial if an analysis of computational cost or inference were included.
• Limited Exploration of Dataset Biases: The paper briefly mentions the use of Chinese and Latin signatures but does not explore potential biases that could arise from dataset limitations. A discussion on demographic diversity in handwriting samples would help assess the generalizability of SPECTRUM.
• Impact of Multi-Scale Integration: While the multi-scale interactor is effective, the paper could delve deeper into the effects of different scale configurations, particularly for datasets with variable stroke dynamics, such as cursive signatures or complex character-based languages.

The main concerns, questions, limitations and weaknesses of this paper are as follows:

• In this paper, images and their frequency domain information obtained via DFT are used as input for network training, and this approach is referred to as multimodal features learning by the authors. However, I believe this terminology is inappropriate (although the even and odd time steps are treated separately as temporal and frequency features, respectively). Multimodal typically refers to different types of data sources, such as visual, text, audio, or sensor data. Therefore, the combination of temporal and spatial domains is more accurately described as multi-dimensional or multi-scale features rather than multimodal.

• Could the authors provide a comprehensive analysis of the inference speed during the model testing phase and compare it with previous algorithms?

• More ablation studies should be conducted to validate the effectiveness of the design "$x_{even}$ for temporal and $x_{odd}$ for frequency". Specifically, the experiments that should be supplemented include "$x_{even}$ for frequency and $x_{odd}$ for temporal" and "both $x_{even}$ and $x_{odd}$ for frequency and both $x_{even}$ and $x_{odd}$ for temporal".

• In Tables 1, 2, 3 and 5, the existing algorithms compared by the authors were all proposed at least two years ago. Are there any more recent algorithms available for comparison?

• What is the parameter count for each module in the proposed algorithm? Could the authors provide a comprehensive analysis and compare it with previous algorithms?

• Any ablation studies about the number of multi-scale interactors? Any differences between every interactor (e.g., inputs, structure) except for the "1D learnable complex weights"? If not, why not directly increase the learnable parameters of single-scale interactor and shared other learnable parameters?

• Any visualization results that can validate the effectiveness of the introduced "1D learnable complex weights"?

• How much performance improvement does the introduction of multi-head self-attention bring to the model?

• Any visualization results that can validate the effectiveness of the introduced "self-gated fusion module"?

• One peculiar point is that the authors claim frequency and temporal domains as two modalities (a claim I find unreasonable), yet they perform no alignment during feature fusion (like LMM). Instead, they simply learn a weight to fuse them, which supposedly yields good results.

• Why not Linear + softmax for "self-gated fusion module"?

• The formula representations for sequence images are inconsistent in Section 3.1 and 3.2.

• How to set c in Eq. (7) for practical application?

• Any ablations studies on the loss weight in Eq. (8)?

Overall, this is not a sufficiently strong work that could be accepted by ICLR, as its main contribution—combining temporal and frequency features for online handwriting verification—is already commonplace in the field of deep learning. Additionally, modules like the self-gated fusion module, designed to support the proposed temporal and frequency integration strategy, lack novelty (these modules add a significant number of extra parameters and computational load to achieve the performance improvement, which is meaningless). Finally, many expressions in the paper are also inadequately phrased.

• Limited novelty. Fourier transform has been widely used in OHV before. The submission works on some details on using FT.
• Experiments only conducted on the three datasets and lacking comparison on multi-linguistic cases.