A Semantic Parsing Framework for End-to-End Time Normalization

Thank you for your positive and constructive review. We greatly appreciate your recognition of our methodological innovation, practical implementation, and LLM-driven data augmentation approach. We address your concerns below:

W1: Dataset and domain limitations

We acknowledge the limitation to TempEval-2013. Unfortunately, this is the only publicly available dataset with SCATE annotations. While medical domain SCATE annotations exist, privacy restrictions prevent access to the underlying text.

However, our data augmentation method is domain-agnostic and can be applied to generate annotations for any domain. In fact, we successfully augmented news data (CC-News) in our experiments. Extending to more domains through automatic annotation is a key direction for future work.

W2: Span detection struggles

While span detection remains challenging, we emphasize that our method achieves state-of-the-art performance on this difficult task. During the rebuttal period, we conducted additional comparisons:

Existing baselines:

• Neural Parser (Xu et al., 2019) - The only publicly available state-of-the-art SCATE system before our work: F1: 0.4317
• SUTime (ISO-TimeML-based): F1: 0.4510

Our method achieves F1: 0.59 - a 36.7% improvement over the previous SCATE state-of-the-art and 30.8% over SUTime.

The span detection challenge is inherent to the complexity of temporal expressions, particularly:

• Compositional expressions like "the past three summers" where understanding the full semantic scope requires compositional reasoning
• Context-dependent boundaries where the same phrase may or may not be temporal (e.g., "spring" as season vs. mechanical spring)
• Multi-span expressions that SCATE uniquely handles but are impossible in simpler frameworks

Our end-to-end approach is the first to achieve robust performance on these complex cases without requiring separate models for recognition, linking, and normalization. While there's room for improvement, we've established a new state-of-the-art baseline. Further improving span detection through larger-scale data generation and better modeling of compositional semantics is a promising direction for future work.

We will add these comparisons and analysis to the revised manuscript.

W3: Public release

We are fully committed to releasing all code, models, and datasets upon publication. This includes:

• Complete SCATE Python library implementation
• Fine-tuned Qwen2.5-0.5B model
• 8,583 augmented training examples
• Training and evaluation scripts

W4: Open-source LLM alternatives

We would like to clarify that training and releasing an open-source small model (Qwen2.5-0.5B) is one of our main contributions. For data augmentation, we did explore open-source LLMs from the Llama family, but their performance was significantly below GPT-4.1 and Claude. However, our method is model-agnostic and will benefit from improvements in open-source LLMs.

Questions:

Q1: Two-stage pipeline for span detection

Our end-to-end approach is motivated by both practical and technical considerations:

Practical motivation: Time normalization is often an upstream task in large-scale applications. For example, when processing millions of documents to build temporal knowledge graphs, a two-stage pipeline significantly increases complexity and reduces efficiency. Single-model inference is crucial for production deployments. Experimental validation: During the rebuttal period, we tested a two-stage approach using the CC-News + Training Set as training set:

• Stage 1: Train BERT-base for time expression extraction
• Stage 2: Train Qwen for SCATE code generation
• Results: F1: 0.2490 (vs. 0.59 for our joint model)
• Critical failure: 251/313 expressions missed (80% miss rate)

While we acknowledge this BERT model here wasn't heavily tuned and more NER-specific data could help, the fundamental issues remain:

• Error propagation: Missed expressions cannot be recovered in stage 2
• Semantic coupling: Expression boundaries often depend on their SCATE interpretation (e.g., "the past three summers" - the full phrase is needed to understand it's a time expression), while separate extraction may miss context
• Deployment complexity: Maintaining two models with different architectures

Our joint approach leverages SCATE semantics during extraction, leading to better recognition and normalization. We agree specialized extraction techniques are promising for future work, particularly continued large-scale data generation to improve end-to-end models in this area. We will add these experiments and discussion to the manuscript.

Q2: Non-English languages

Yes, the framework is language-agnostic. As long as SCATE annotations are designed for the target language, our approach can be applied. The Python library and code generation method work independently of the input language.

Q3: Extension to other tasks

As demonstrated in an exsiting paper "A Dataset and Evaluation Framework for Complex Geographical Description Parsing", similar operators (Union, Intersection) apply to geo-normalization tasks. Our executable code generation approach can be readily extended to any semantic parsing task with compositional semantics, including event extraction, spatial reasoning, and logical form generation.

Q4: Failure analysis on discarded samples

Discarded samples primarily failed due to:

Syntax errors:

• Missing parentheses: Year(2023 instead of Year(2023)
• Undefined variables: Using Months instead of MONTH enum
• Invalid Python syntax: Malformed string literals or incorrect indentation

Semantic errors:

• Invalid operator combinations: Next(shift=Period(...)) when Next requires a Repeating shift
• Incorrect parameter types: Repeating(unit=DAY, value="Monday") instead of numeric value
• Incompatible nesting: This(interval=Period(...)) when This requires an Interval

Runtime errors:

• None value operations: Attempting arithmetic on underspecified expressions
• Invalid date operations: Interval.of(2023, 14, 1) with month 14
• Infinite loops in repeating intervals without proper bounds

These failures demonstrate our validation mechanism's effectiveness: only code that is both syntactically correct and semantically meaningful passes through, ensuring high-quality training data. We will include this analysis in the revised manuscript.

Q5: Biases in LLM-generated data

We tested our augmented dataset using the vector-institute/Llama3.2-NLP-Newsmedia-Bias-Detector model. The analysis found no biases in the generated temporal expressions.

Thank you again for your constructive feedback. We will incorporate these clarifications and experimental results in our revision.

Thank you for your review and constructive feedback. We appreciate your recognition of our novel approach and the potential value of our resources to the community. We address your concerns below:

W1: Limited experimental evaluation and comparison with previous work During the rebuttal period, we conducted additional comparisons.

Existing SCATE-based system:

• Neural Parser (Xu et al., 2019) - The only publicly available SCATE implementation before our work: F1: 0.4317, Precision: 0.5677, Recall: 0.3482

• SUTime (ISO-TimeML-based system): F1: 0.4510, Precision: 0.5838, Recall: 0.3674

Our method:

• Qwen2.5-0.5B + CC-News + Training Set: F1: 0.59, Precision: 0.59, Recall: 0.59
• 36.7% improvement over the previous SCATE state-of-the-art
• 30.8% improvement over SUTime

These results demonstrate that our approach significantly outperforms existing methods in comparable evaluation settings. We will add these comparisons to the revised manuscript.

W2: Data efficiency issues

During the development, we explored several strategies:

1. Different fine-tuning approaches: We experimented with LoRA during development. However, we observed that LoRA produced similar results to full fine-tuning, particularly because our model is already quite small (0.5B parameters). Given the minimal difference and the simplicity of full fine-tuning for such a small model, we opted for full fine-tuning in our final experiments.
2. Model selection rationale: We also explored code-pretrained models during development. However, we found that small code-specific models like Qwen2.5-Coder struggled significantly with temporal expression identification, leading to numerous missed time expressions before even attempting SCATE generation. This resulted in severe performance degradation (more than 0.2 F1 score drop). The issue appears to be that code-pretrained models, while excellent at code generation, lack the natural language understanding capabilities needed for identifying temporal expressions in text. Since our task requires both strong NLP capabilities (for span identification) and code generation abilities, we found that general instruction-tuned models like Qwen2.5-0.5B-Instruct provide the best balance. We acknowledge that exploring larger code-specific models or hybrid approaches could be valuable future work, but for our goal of deployable small models, the general instruction-tuned model proved most effective.

W3: In-depth analysis and examples

We conducted detailed error analysis on our best model's predictions. Here are specific examples illustrating the main failure modes:

1. Missed expressions (70% of errors): These are the most critical failures where the model completely fails to identify annotated time expressions:

• Example: "We estimated we could do it in 100 days, and we got across on the 99th day."

• Time expression: "99th day"

• Gold: Nth(shift=Repeating(unit=DAY), index=99)

• Predicted: Missing

• Analysis: The model struggles with ordinal expressions embedded in narrative contexts.

• Example: "This flu season started in early December..."

• Time expression: "This flu season"

• Gold: This(interval=Interval.of(2013, 3, 22), shift=Repeating(unit=None))

• Predicted: Missing

• Analysis: Abstract temporal concepts like "season" without explicit calendar anchors are challenging.

2. Boundary errors (10% of errors): The model identifies temporal expressions but with incorrect boundaries:

• Example: "He said: 'Lowe was a brilliant, kind fellow who never sought the limelight... and 60 years on from Everest his achievements deserve wider recognition.'"
• Gold annotation: "60 years on from"
• Model prediction: "60 years on" (missing "from")
• Both generate similar SCATE code but the boundary difference affects evaluation
• This suggests the model understands the temporal semantics but struggles with precise span detection

3. Structural errors (10% of errors): Complex nested temporal expressions requiring compositional operators:

• Example: "The season started about a month earlier than usual, sparking concerns it might turn into the worst in a decade"
• Time expression: "month earlier than usual, sparking concerns it might turn into the worst in a decade"
• Gold: Last(interval=Before(interval=Interval(...), shift=Period(unit=MONTH)), shift=Period(unit=DECADE))
• Predicted: Only captures fragments ("a month earlier", "decade") separately
• Analysis: The model cannot recognize that these fragments form a single complex comparative temporal expression

4. Operator confusion (5% of errors): Correct span identification but wrong SCATE operator selection:

• Example: "Mr. Obama said later at a news conference in Amman..."
• Time expression: "later"
• Gold: After(interval=Interval(...), shift=None)
• Predicted: Next(interval=Interval(...), shift=None)
• Analysis: Both operators indicate future time, but "After" is more appropriate for indefinite future references while "Next" implies the immediate next occurrence

5. Granularity errors (5% of errors):

• Example: "Mr Lowe also took part in the trans-Antarctic expedition of 1957-58"
• Time expression: "1957-58"
• Gold: Between(start_interval=Year(1957), end_interval=YearSuffix(interval=Year(1957), digits=58)) → spans 1957-1959
• Predicted: Between(start_interval=Year(1957), end_interval=Year(1958)) → spans only 1958
• Analysis: The model misinterprets abbreviated year ranges, treating "58" as 1958 rather than understanding the convention that "1957-58" means 1957-1958 inclusive

We will add this detailed analysis with examples to the revised manuscript to help readers better understand the model's capabilities and limitations.

Additional clarifications on significance of the problem

While time normalization may appear to be a niche problem, it is actually a fundamental building block that quietly powers many critical real-world systems:

• Clinical decision support: Accurate interpretation of expressions like "take medication three times daily for two weeks" or "symptoms started 3 days post-surgery" can be life-critical
• Legal document analysis: Misinterpreting "within 30 days of receipt" vs "30 days after receipt" in contracts can have million-dollar consequences
• Financial systems: Trading algorithms and regulatory compliance depend on precise temporal reasoning for transaction windows and reporting deadlines
• Question answering: Any system answering "when" questions requires robust time normalization as a core component

The challenge is that these applications often fail silently when time normalization is incorrect: a QA system simply returns wrong dates, a clinical system miscalculates medication schedules. Our work provides the first publicly available, executable implementation of the most expressive temporal framework (SCATE), along with a large-scale dataset and deployable model. This enables researchers and practitioners to address complex temporal expressions that were previously impossible to handle systematically. In essence, time normalization is like parsing in NLP: a seemingly narrow technical problem that underlies countless applications. Our contribution makes this critical component more accessible and capable.

Thank you again for your constructive feedback. We will incorporate these clarifications and experimental results in our revision.

Thank you for your positive review and recognition of our contributions, particularly the value of our SCATE Python library and the solid baseline we establish. We address your concerns below:

W1: Joint extraction vs. separate extraction, Q1: Separate evaluation attempts

Our end-to-end approach is motivated by both practical and technical considerations:

Practical motivation: Time normalization is often an upstream task in large-scale applications. For example, when processing millions of documents to build temporal knowledge graphs, a two-stage pipeline significantly increases complexity and reduces efficiency. Single-model inference is crucial for production deployments. Experimental validation: During the rebuttal period, we tested a two-stage approach using the CC-News + Training Set as training set:

• Stage 1: Train BERT-base for time expression extraction
• Stage 2: Train Qwen for SCATE code generation
• Results: F1: 0.2490 (vs. 0.59 for our joint model)
• Critical failure: 251/313 expressions missed (80% miss rate)

While we acknowledge this BERT model here wasn't heavily tuned and more NER-specific data could help, the fundamental issues remain:

• Error propagation: Missed expressions cannot be recovered in stage 2
• Semantic coupling: Expression boundaries often depend on their SCATE interpretation (e.g., "the past three summers" - the full phrase is needed to understand it's a time expression), while separate extraction may miss context
• Deployment complexity: Maintaining two models with different architectures

Our joint approach leverages SCATE semantics during extraction, leading to better recognition and normalization. We agree specialized extraction techniques are promising for future work, particularly continued large-scale data generation to improve end-to-end models in this area. We will add these experiments and discussion to the manuscript.

Q2: Train vs. test set performance gap

The performance difference is primarily because our prompt development relied on training set experiments to ensure fair comparison across models. The test set remained completely untouched to prevent overfitting. This leads to all LLMs show better training set performance. We will clarify this in the manuscript.

Q3: ISO-TimeML baselines

During the rebuttal, we added comparisons with SUTime (ISO-TimeML-based) and Xu et al. (2019) (the previous best SCATE-based system) to provide a clearer baseline for SCATE performance:

• SUTime (ISO-TimeML-based): F1: 0.4510
• Xu et al. (2019) (SCATE-based): F1: 0.4317

Our method achieves F1: 0.59, demonstrating 30.8% improvement over SUTime and 36.7% over the previous best SCATE system. Our analysis reveals that our method can identify and normalize significantly more compositional time expressions compared to ISO-TimeML-based SUTime, leading to better performance.

Thank you again for your constructive feedback. We will incorporate these clarifications and experimental results in our revision.

Thank you for your review. We appreciate your recognition of our comprehensive Python implementation of SCATE, the manuscript's clarity, and our documentation of framework ambiguities. We address your concerns below:

W1: No benchmarking against existing methods

During rebuttal, We have conducted comprehensive experiments comparing our method against existing approaches:

Neural Parser (Xu et al., 2019) - The only publicly available SCATE-based system and the previous state-of-the-art: F1: 0.4317, Precision: 0.5677, Recall: 0.3482.

This system represents the best existing method for SCATE-based time normalization before our work.

SUTime (Chang & Manning, 2012) - The widely-used timenorm system (for comparison with non-SCATE approaches): F1: 0.4510, Precision: 0.5838, Recall: 0.3674

Our method (Qwen2.5-0.5B fine-tuned on augmented data) achieves: F1: 0.59, Precision: 0.59, Recall: 0.59, representing a 36.7% F1 improvement over the previous best and only available SCATE implementation (Neural Parser; Xu et al., 2019).

This is a significant advancement for SCATE-based time normalization. We emphasize that before our work, researchers had only one option for SCATE-based normalization (Xu et al., 2019), which required complex multi-stage processing. Our end-to-end approach not only outperforms it substantially but also provides a simpler, more deployable solution. We will add these comparisons to the revised manuscript.

W2: Limited experimentation on prompting techniques

We respectfully clarify that our primary contributions are:

1. Novel formulation of time normalization as code generation
2. Complete Python implementation of the SCATE framework - the first publicly available executable SCATE library
3. Large-scale augmented dataset (8,583 validated examples) generated through our approach
4. End-to-end trainable system that achieves state-of-the-art results

All code, data, and models will be made publicly available upon publication.

Prompting serves as a tool for data augmentation and model comparison, not as our core innovation. We have already designed comprehensive and detailed prompts (Appendix B, 23 pages) that effectively capture SCATE's complex semantics, enabling LLMs to generate executable code for a library they have not seen during pre-training.

The effectiveness of our approach is demonstrated by:

• Successfully generating 8,583 valid SCATE code examples from 10k sentences
• Training a 0.5B (BERT size) model that outperforms both its teacher model (Claude 3.7) and all existing baselines

We welcome specific suggestions on additional prompting techniques that would strengthen our evaluation, but emphasize that our contribution lies in the overall framework rather than prompt engineering.

W3: No comparison with BERT baseline

We would like to clarify that BERT is primarily a classification model designed for sequence labeling/classification tasks. For time normalization, BERT would require: (1) a sequence labeling layer for span detection, (2) additional components for linking expressions, and (3) a separate normalization module. This makes direct comparison inappropriate.

Our Qwen2.5-0.5B model (similar size to BERT-large) is a decoder-only transformer specifically suited for generation tasks. It performs end-to-end time normalization in simple forward pass, making it different from BERT's classification paradigm.

Responses to Questions:

Q1: Figure 1 clarity

We clarify that Figure 1 illustrates the SCATE annotation schema. Figure 2 provides the complete system overview with clear input/output examples (bottom of figure). The left side shows input (text + DCT), and the right side shows output (SCATE code + execution results). We will add clearer labels in the revision.

Q2: TempEval-2013 dataset details

As stated in lines 247-254, we use documents from TempEval-2013 with SCATE annotations (not the original ISO-TimeML Tasks A/B/C). We use the SCATE annotations from the official SCATE annotation repository, containing 557 training and 313 test examples with complete SCATE annotations.

Q3: Table 1 clarification

As described in lines 282-284, Table 1 uses our SCATE prompt method. The purpose (line 279) is to evaluate whether state-of-the-art foundation models can perform time normalization using our comprehensive SCATE prompt. We show training set results in Table 1 to compare foundation model capabilities, then evaluate the best model (Claude 3.7 and GPT 4.1 ) on the test set in Table 2 to avoid overfitting. We will clarify this in the revision.

Q4: Runtime advantages

Our method offers significant runtime advantages:

• Neural Parser (Xu et al. 2019): Complex multi-stage pipeline with character-level LSTM (very slow), plus rule-based linker and Scala-based normalizer
• Our method: Simple forward pass through a 0.5B model (BERT-sized)

On long texts, our transformer-based end-to-end model is orders of magnitude faster than character-based LSTMs (well-established in literature). Additionally, our model is 1000x smaller than Claude 3.7, enabling practical local deployment.

Q5: Prompt engineering details

The prompt in Appendix B is the complete markdown prompt used in our experiments.

We performed extensive prompt tuning; shorter prompts (especially with fewer examples) led to performance degradation on the training set.

The prompt was manually designed with assistance from Copilot during development.

We will clarify these details in the revision.

Q6: Comparison with TempEval methods

As mentioned above, we use documents from TempEval-2013 but with SCATE annotations, not the original ISO-TimeML annotations. The original TempEval tasks (A/B/C) target different objectives with ISO-TimeML schema, making direct comparison inappropriate. Our work specifically addresses the more expressive SCATE framework.

We hope these clarifications address your concerns. We are committed to improving the manuscript based on your valuable feedback.

Model size fairness

Qwen2.5-0.5B (500M) is the smallest available decoder model suitable for code generation. To demonstrate that improvements come from our method, not just model size:

• Qwen2.5-0.5B without our augmented data: F1=0.01
• Qwen2.5-0.5B with only augmented data: F1=0.37
• Qwen2.5-0.5B with augmented + original data: F1=0.59

This ablation clearly shows our data augmentation and training strategy drive the improvements.

Contributions

Our contributions remain significant:

1. First end-to-end code generation approach for time normalization (replacing complex pipelines)
2. First publicly available executable SCATE library (enabling future research)
3. 10x larger dataset through validated augmentation (8,583 examples vs 870 original)
4. State-of-the-art results with 31.7x faster inference

The reviewer suggests this is only an extension of Bethard et al. (2016), but Bethard only proposed the annotation schema - they provided no implementation, no executable semantics, and no learning approach. We provide all three, plus demonstrate that code generation fundamentally outperforms classification-based approaches.

Regarding generalization

TempEval-2013 with SCATE annotations is currently the only publicly available dataset for this task. We compensate by generating 8,583 additional validated examples, effectively creating the largest SCATE dataset available.

We hope this clarifies our contributions and addresses the reviewer's concerns.

Thanks for the clarifications- The back and forth here has helped understand the contributions more (which need to be better highlighted in the manuscript)

Runtime comparisons:

• Thanks for adding the numbers. Could you clarify if the times mentioned are per sample or otherwise?

Evaluation

• Although I agree with the challenges on using a BERT model with the current formulation (code generation) of the task, I disagree it being unsuitable. Time normalization, can still be annotated outside of code-generation as a span extraction and concept normalization. From the multi-stage experiments described below (in reviewer's comments) - what was the training set-up, extraction performance for span extraction?

Figure 2

• Shouldn't the output be Repeating(DAY, week, 1) as per author's description in lines 172-178 as opposed to 3 (in the manuscript, considering Monday=0)?

To model the task as span extraction and rule-based normalization using BERT-

Assuming start_day, interval are entities and the relation 'before' connects the two. Wouldn't this along with doc_time as an entity be enough to generate the intervals with rules/post processing? Does the code generation approach accurately generate the dates for hard cases too? I acknowledge reading the error analyses in the rebuttals below- it seems that the LLM-based approach has failure cases similar to a complex example like above.

The seq2seq approach should be able to generate the syntactial expression well if a seq2seq model like T5 is fine-tuned to perform this.

My repeated emphasis on a BERT-based method stems from the fact that span extraction should be extremely easy for the model simply because they are only surface-level descriptions (days of the week + possible abbreviations, months, years, date formats etc. There is limited variability in language for them ). The relations will be a hard task along with normalization to actual dates. This analyses will enrich the paper's contributions if the authors consider adding it.

Implementing a full BERT-based system would constitute a substantial research contribution in itself - requiring architecture design for compositional structure prediction, new training objectives for operator composition, and extensive engineering for SCATE code assembly.

With all due respect, a comprehensive evaluation involves in-depth exploration like this. Again, code generation is a valid approach for Time normalization, however, for a manuscript with comprehensive evaluation, a baseline is a necessity (outside of an LSTM) especially when the contributions involve LLMs. I strongly recommend formulating time normalization outside of code generation and have performance metrics for span, relation extraction and normalization. This makes the claims stronger outside of the arguments of complexity in structures. More below-

• I shared 3 papers related to event extraction- they were meant mainly to be methods (for inspiration) to formulate the task as seq2seq and decompose the complex task into smaller and easier sub-tasks. They were not meant to be plug and play for this work (if that is what the authors expected). While Eberts et al is mainly a BERT-based model for event extraction, it could serve as a method to train a model to extract spans and normalize relations. The other 2 papers were examples of how an event extraction task (span+ relations) can be formulated as seq2seq. I still believe that a simpler baseline using T5 (or a smaller seq2seq model) with the ~8k samples for code generation is a relevant baseline. A (smaller model) baseline helps ground the errors better and validates where an LLM improves or fails. I suggest that the authors consider this for a strong evaluation.

Statistical Significance

• Thank you for pointing me to the Appendix. Please highlight them in the main text.

Error Analyses

I acknowledge reading the errors in the rebuttal (other reviews) and descriptions in lines 321- 338; It will be interesting to compare if the error categories were an issue with other fine-tuned models (like T5) as well.

As suggested by other reviewers as well, the additions on run-time, adding baseline performances can potentially improve the manuscript. I have updated my rating to reflect that possibility and I will engage more if required. Thank you for the hard work in putting this together.

Thank you for your continued engagement and feedback. We appreciate your specific questions and suggestions.

BERT Evaluation Details

Thank you for asking about the training setup:

• Training setup: We used standard BIO tagging with a single label type (B-TIME, I-TIME, O)
• Training data: Original training set + 8,000 sentences from augmented CC-News data
• Label distribution: B-TIME: 6,620, I-TIME: 4,527, O: 167,405

The relatively poor performance (19.8% recall) likely stems from:

1. Label imbalance: O tags vastly outnumber temporal tags (167,405 vs 11,147)
2. Compositional boundaries: Unlike traditional NER, SCATE temporal expressions have boundaries that depend on semantic interpretation (e.g., "three summers" vs "the past three summers")

We will include the two-stage approach (BERT + seq2seq) comparison in our final version as you suggested.

T5 Evaluation

We will add T5 as a baseline in the future version.

Manuscript Improvements

Thank you for your suggestions on the discussion and conclusion sections. We will:

• Add error analysis comparing baselines and our method
• Clarify how code generation helps with compositional structures
• Better frame the open problems in time normalization and our contributions

Thank you again for your feedback.