Responses to Reviewer n8KK (1/1)

Dear reviewer n8KK,

Thank you for your valuable feedback on the supplement of relevant literature of the article. Here is the response:

W1. The idea of learning dynamic graphs that model variable relationships is quite common in literatures. It would be better for the authors to cite related works, compare with existing works and make arguments about novelty and effectiveness of the proposed modules.

• "Spatio-temporal meta-graph learning for traffic forecasting." In Proceedings of the AAAI conference on artificial intelligence, vol. 37, no. 7, pp. 8078-8086. 2023.
• "Crossgnn: Confronting noisy multivariate time series via cross interaction refinement." Advances in Neural Information Processing Systems 36 (2023): 46885-46902.
• "Networked time series imputation via position-aware graph enhanced variational autoencoders." In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2256-2268. 2023.
• "Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 8, pp. 9089-9097. 2024.

A1. Thank you for the reminder. In a later version, we will cite these four papers in the section on “Graph Neural Networks for Multivariate Time Series” in the related work. Although modeling dynamic graphs to capture relationships between variables is not new, many dynamic spatiotemporal graph methods are designed for traffic flow prediction tasks, which involve regularly sampled multivariate time series. Applying these methods directly to irregularly sampled medical time series classification tasks may lead to suboptimal performance. Our approach specifically addresses the challenges of ISMTS by using domain knowledge as guidance and adjusting edge weights in the graph based on observation masks and dynamic densities, which is a key distinction from these methods.

W2. There exist some missing references and baselines for irregularly sampled time series classification problems. It would be better to cite related works and compare with them in the experiment sections.

• Li, Zekun, Shiyang Li, and Xifeng Yan. "Time Series as Images: Vision Transformer for Irregularly Sampled Time Series." Advances in Neural Information Processing Systems 36 (2023).
• Labach, Alex, Aslesha Pokhrel, Xiao Shi Huang, Saba Zuberi, Seung Eun Yi, Maksims Volkovs, Tomi Poutanen, and Rahul G. Krishnan. "Duett: Dual event time transformer for electronic health records." In Machine Learning for Healthcare Conference, pp. 403-422. PMLR, 2023.

A2. Thank you for the reminder. In the revised version, we will cite these two papers in the section on “Irregularly Sampled Multivariate Time Series Modeling” in the related work. Additionally, we have conducted comparative experiments, and the results are as follows:

Q1. It seems some of the baselines such as DGM^2-O can reach good performance with relatively less time and space requirements. Can there be any potential improvements to the efficiency of the proposed KEDGN model? It seems BERT is also optimized during training, it would be interesting to see some efficiency experiments and analysis regarding this aspect.

A3. Firstly, it should be clarified that BERT is frozen and does not participate in the optimization of training. We only take the sentence representation output by BERT as the semantic representation of variables, so the training cost is consistent with the variable embedding of random initialization. In addition, due to the sequential nature of RNN computation, there is an inherent bottleneck in the running time of our model. We have optimized the code implementation as much as possible to keep the overall running cost of the model within an acceptable range. We acknowledge this as a limitation of our model and will add it to the limitations section.

Responses to Reviewer 68iN (1/1)

Dear reviewer 68iN,

Thank you for your valuable feedback on the formula details and completeness of the paper. Here is the response:

W1. $Z^{t}$ 's definition is not clear

A1. We apologize for the oversight. If there are no observations at the preceding/following time points, we take the time interval of the following/preceding observation as the density. If neither a preceding nor a following observation exists, it indicates that this observation is the only one for the variable, so we take half of the maximum observation time span across all variables as the density.

The corrected definition is as follows:

Here, $Z^{(t)} = Z_{i, v}$ represents the average density at time $t$ for the $v$-th variable's $i$-th observation.

W2. I am just a little bit confused by the node in the dynamic variable graph. As we know, the patient's EHR data will contain lots of clinical notes and some like treatment, medications, and so on. You have shown the description in your model's framework in Figure 3. So do you consider other variables in patient's EHR data? Besides, the node here denotes a text, not detailed medical terminology, right?

A2. Here, we use general medical semantic text to differentiate between variables and guide the modeling of correlations between these variables in irregular time series. If I understand correctly, the "other variables in patient's EHR data" you mentioned likely refer to patient-specific clinical records, treatment history, medication history, etc. These are personalized medical data and not general medical domain knowledge. They often include multiple modalities, such as text and images, which extend beyond the scope of time series analysis. Our datasets only contain time series data and do not include clinical notes. Thank you for the constructive suggestion; we may consider integrating multimodal medical data in future work.

W3. Could you please add a comparison graph between the original variable representation and the final representations?

A3. Certainly, but due to the limited information that can be uploaded during discussion period, we will include this in a later version.

Responses to Reviewer yFgU (1/1)

Dear reviewer yFgU,

Thank you for your valuable feedback on the formula details of the article. Here is the response:

W1. Details of the Proposed Model could be better explained by giving the intuition behind the equations. (Especially for 4.3.2 and 4.4)

A1. Firstly, we need to correct equation(8) in Section 4.3.2 to: $G^{(t)}_{ij} = A^{(t)}\_{ij} \times (1 - W\_{ij} |D^{(t)}\_{i} - D^{(t)}\_{j}|).$

The intuition behind this equation is as follows: If two variables have similar densities at a given time, their correlation should be higher. Conversely, if the density of one variable increases or decreases significantly at subsequent times, the correlation between these variables will decrease. For example, Figure 6 in Section 5.4.3 shows that NIDiasABP and HR have similar densities at $t=4$, but by $t=15$, the density of NIDiasABP decreases, which reduces its correlation with HR. Therefore, we use the absolute difference in densities between the two variables and add a negative sign so that the density difference is negatively correlated with the correlation.

Regarding the formulas in Section 4.4, they integrate variable-specific parameter spaces and dynamic graph networks into the GCRNN backbone network. Equation (9) represents the standard definition of $1st$-order Chebyshev polynomial expansion approximation for graph convolution. Equations (10-12) define the operations in GCRNN. Equation (13) indicates that only the hidden states of observed variables are updated at each time step, ensuring that the model fully adheres to the original ISMTS pattern.

Q1. Did the authors try to use definitions given by health care professionals to confirm the definition given by the GPT model?

A2. In this paper, what the model required as input are general medical domain knowledge text descriptions of each variable. These domain knowledge is relatively well-established and fixed, and can be confirmed by consulting medical literature. Therefore, we did not seek help from health care professionals.

Q2. $Z^{t}$ 's definition is not clear.

A3. We apologize for the confusion regarding the notation. We have revised the definition of $Z^{(t)}$ in Equation (6) as follows:

Here, $Z^{(t)}=Z_{i, v}$ represents the average density at time stamp $t$ for the $v$-th variable's $i$-th observation point. It is important to note that density is used for dynamically adjusting edge weights, and edges only exist between actual observations. Therefore, when a variable has no observations, there are no edges to adjust, so the density calculation is not needed in such cases. Hence, we did not consider this situation in defining the average density.

Q3. It is not clear why the equation (9) contains a transpose.

A4. As indicated in Equation (4), after computing the cosine similarity between variable node embeddings, we apply the Softmax function to normalize each column to obtain the graph. Since matrix multiplication involves the left matrix’s specific rows and the right matrix’s specific columns, and each column of the right matrix $S \in \mathbb{R}^{V \times I}$ represents the values of all nodes for a specific input channel, we transpose $I_V + G(t)$ in Equation (9). This transposition converts each column into a row, representing the correlations of one variable with all other variables, ensuring that the sum of the output edge weights for each variable node is 1.

Responses to Reviewer vHK1(2/3)

**Q1. The function of activation function in equation (7). **

A1. On one hand, as you mentioned, this activation function reflects the dynamics of variables, such as time decay or exponential increase. On the other hand, this activation function serves a normalization purpose because the edge weights in the knowledge-empowered complete graph $\boldsymbol{A}$ are normalized values. If we directly use the absolute value of the density to adjust the edge weights, the values might become excessively large or small, which would severely disrupt the basic graph structure learned from textual knowledge. Typical activation functions with normalization capabilities include Sigmoid and Tanh. Since Tanh performed better in our experiments, we chose Tanh.

The experimental results for different activation functions are shown in rows W/O $\sigma$ and $\sigma$ Sigmoid in Table 2 of the newly submitted PDF.

Q2. The parameter complexity of the model.

A2. We have calculated the number of model parameters for each dataset, as shown in Table 5 of the newly submitted PDF.

As shown, the parameter count of our model is not particularly high. It is on the same order of magnitude as Warpformer and significantly lower than Raindrop by three orders of magnitude. Although we calculate an independent $W$ for each variable, the total $W \in \mathbb{R}^{V \times I \times O}$ does not equate to the parameter count. $W$ is derived from the multiplication of two matrices: the variable embedding matrix $Q \in \mathbb{R}^{V \times q}$ and the weight matrix $W \in \mathbb{R}^{q \times I \times O}$. The first matrix is computed from textual embeddings, and only the second matrix belongs to the model parameters. The sizes $q$, $I$, and $O$ are hyperparameters, independent of the number of variables, and typically set to be less than 16. Hence, the parameter complexity of our model is not high. We will include this parameter complexity analysis in Appendix F.2 on computational costs.

Q3. The importance of using text information of variables.

A3. First, as analyzed in Section 5.4.1, the relative distribution of variable embeddings obtained from different text sources via PLM is consistent with the relative distribution of the variables' time series patterns. Both of them reflects the relative distribution of the variables' inherent sense. The embedding space of variables obtained from different text sources may vary in absolute distribution, but the relative distribution should be similar. So the performance from different text sources should also be similar, which aligns with our expectations and proves incorporating text information has a universally valid.

Moreover, as shown in table 4 of the newly submitted PDF, the overall results of the ablation experiments indicate that the introduction of text contributes to performance improvement and is not useless. More importantly, as demonstrated in the visualizations in Section 5.4.1, if the variable text embeddings are replaced with learnable embeddings, the resulting learned variable embedding space tends to be nearly uniformly distributed, which lacks interpretability. In contrast, using text embeddings ensures that the variable embedding distribution is consistent with domain knowledge, thereby providing the model with strong interpretability.

Q4. Calculation and symmetry issues of correlation graph

A4. Your observation is correct. Figure 5(a) is computed from $g(E)$. It is true that using the BERT embeddings of these variables can achieve a broadly similar correlation figure, but there are subtle differences induced by $g(\cdot)$. The introduction of $g(\cdot)$ is necessary as it not only performs feature reduction but also avoids using a completely fixed prior graph. This preserves the model’s ability to adaptively optimize the graph structure based on different data distributions and downstream tasks.

To validate the effectiveness of $g(\cdot)$, we provide the ablation experiment results on row W/O $g(\cdot)$ in table 2 of the newly submitted PDF.

Regarding the asymmetry of Figure 5(a): As indicated in Eq.(4), after computing the cosine similarity between variable node embeddings, we apply the Softmax function to normalize each column to obtain the graph. This normalization ensures that the diagonal elements correspond to the maximum values in their respective columns, but not necessarily the maximum values in each row.

We intentionally set the graph to be asymmetric because some variable correlations are directional. For example, a decrease in insulin level may lead to an increase in blood glucose, but an increase in blood glucose does not necessarily lead to a decrease in insulin level. This is why the edge weight from insulin to blood glucose is not equal to the weight from blood glucose to insulin. The same reasoning applies to HR and FiO2.

Q5. The basic time resolution or sampling rate for each dataset.

A5. In ISMTS, there are significant differences in sampling rates among different samples, variables, and periods. As a result, calculating a basic sampling rate for an entire dataset is challenging and sometimes not very informative. Instead, more relevant statistics for ISMTS datasets include metrics such as missing rate and maximum length.

Q6. How are the static features combined with time series.

A6. We apologize for the omission. We follow the approach used in Raindrop to incorporate the static features. Specifically, static features are first mapped to static vectors through a linear layer and then concatenated with the feature vectors of the time series before being input into the classifier.

Responses to Reviewer vHK1(3/3)

**Q7. How are the input length and prediction length determined. **

A7. We apologize for the omission. As you mentioned, for the MIMIC-III, P12, and PhysioNet datasets, we use ICU data from the preceding 48 hours as input to predict the mortality during the hospitalization. For the P19 dataset, we use up to 60 hours of ICU data to predict whether sepsis will occur within the subsequent 6 hours. We will include these details in the "Appendix B: Datasets" section of the revised version.

Q8. Limitations

A8. Although our proposed method effectively guides ISMTS modeling through the domain knowledge from text modality, it has some limitations. The backbone of our model is based on an RNN architecture, which inherently has a sequential computation characteristic that can be a bottleneck in terms of runtime. Additionally, our method is specifically tailored for medical applications, and its performance may be limited in other ISMTS applications, such as human activity recognition, where variables may lack domain knowledge and thus cannot generate high-quality text descriptions. We will include a detailed discussion of these limitations in the revised version of the paper.

Dear Reviewer vHK1,

Hope this message finds you well.

We appreciate the diligent efforts of you in evaluating our paper. We have responded in detail to your questions. As the discussion period will end soon, we would like to kindly ask whether there are any additional concerns or questions we might be able to address.

Once more, we appreciate the time and effort you've dedicated to our paper.

Best regards,

Authors