Interpretable Table Question Answering via Plans of Atomic Table Transformations

The paper's presentation requires improvement, as some tables extend beyond the page margins

Thank you for bringing this to our attention! We will ensure that all tables are properly formatted within the page margins in the next revision.

If there are any further concerns, we would be more than happy to address them and continue the discussion. We are excited to engage in any way!

The XAI comparison is not very convincing. The highlight method is preferable for SQL-based methods such as PoS or Text-to-SQL. However, it is not explicit for other baselines without SQL, such as Dater and CoTable. Did you try to remove them in the experiments and see the results?

We find this comment very thought-provoking, and it indeed aided us in improving our XAI evaluation.

We understand that the highlight-based interpretability method may naturally align better with SQL-based approaches like POS and Text-to-SQL. For non-SQL methods, such as Dater and CoTable, we agree that highlight-based interpretability may be less advantageous compared to SQL-based methods.

Following Reviewer VQhj's suggestion, we removed highlights to assess which explanations perform best without the potential advantage that highlights may give to methods like Text-to-SQL or POS.

We tested explanations with no highlights on three XAI tasks: Preference Ranking, Forward Simulation, and Model Debugging (an additional, more standard XAI evaluation). We repeat the LLM-judge setups as described in our paper for GPT4o-mini and report the data below:

Table: Evaluating Table QA explanations with no highlights (attribution maps).

Our findings are:

• Without highlights, POS still performs best across all three XAI tasks—Preference Ranking, Forward Simulation, and Model Debugging.
• The removal of highlights does not affect the accuracy/ratings for the explanation methods, suggesting that highlight-based explanations may not be important for LLMs perhaps because it is unknown which formats of highlights (image/xml tags/mark-up syntax) should be effective for language models. This insight could guide future developments in XAI methods for Table QA.

References:

• [a] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS
• [b] What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods, NeurIPS

What is the motivation to add highlights in the XAI experiments?

Thank you for this great question!

Highlights (or attribution maps) are widely recognized as a key method for explaining AI decisions to humans across all domains, including images[a], text[b], and time series[c]. In these domains, highlights effectively draw users’ attention to the most relevant parts of the data, helping them understand how the model arrived at its decisions.

• [a] What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods, NeurIPS
• [b] Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?, ACL
• [c] Explainable AI for Time Series Classification, IEEE Access

For Table QA, there has been limited work visualizing model explanations in an interpretable way. To address this gap, we adapted the highlight method from other domains for tabular data to help users understand how the model utilizes specific parts (features) of the input table to arrive at the final answer. By highlighting relevant rows, columns, or cells in the table, we aim to provide a clear visual explanation of the model's reasoning process.

The novelty of the method is limited. Since the step-by-step execution has been already proposed in CoTable, this work seems a simple extension of CoTable with SQL queries and plan generation. And the final performance is even reduced with the added components.

Thank you for your comment! While POS shares similarities with CoTable in its step-by-step approach, we believe our method introduces key innovations beyond CoTable. POS systematically decomposes complex queries into atomic, interpretable sub-steps and SQL-compatible transformations, providing transparency that CoTable and other existing methods lack. Unlike traditional program-based approaches, which often struggle with unexplainable selections or function arguments (see Fig. 1), POS allows each transformation to be fully traceable, making the entire reasoning process interpretable.

We would like to emphasize that our goal is NOT to improve Table QA accuracy but rather to improve the interpretability of Table QA responses so that users and other LLM agents could make better decisions. This is because understanding how a model arrives at a decision in a information-dense format as a table poses a challenge to the cognitive load of users to verify the predicted answer and identify errors in the reasoning chain.

To further support the interpretability frontier of POS, we conducted an additional evaluation where explanation methods (Text-to-SQL, DATER, CoTable, and POS) help us identify whether the predicted answer by LLM is correct (model prediction debugging[c,d,e,f]). Model prediction debugging is the mainstream and established task in XAI and has many real-world applications (e.g. healthcare[g]).

Using both the TabFact and WikiTQ datasets, we evaluated the following question: Given the explanations, can users identify if the predicted LLM answer is correct or wrong? We leverage LLM-as-a-Judge as we did in Forward Simulation and Preference Ranking for this setup.

Here is our prompt to the LLM judges:

The Table Question Answering (Table QA) model is working on a tabular dataset, answering questions based on a given table.

You are given an HTML file containing a Question, Input Table, Prediction, and an Explanation clarifying the Prediction.

Your task is to carefully analyze the explanation and determine whether the Prediction is correct or not.

Explanation Method: [Text2SQL/DATER/CoTable/POS]
Answer with ‘Correct’ or ‘Wrong’ only.

You MUST ignore the order of the options and answer based on the correctness of the Prediction!

We report the result of the evaluation below:

Table: Model prediction debugging accuracy (↑) on TabFact and WikiTQ with LLM judges using different Table QA explanations.

Our findings are:

• POS significantly improves model prediction debugging accuracy compared to other Table QA baselines.
• On the TabFact dataset, POS achieves a debugging accuracy of 76.74% with GPT-4o-mini, 72.85% with GPT-4o, and 72.08% with GPT-4, outperforming other methods by margins of up to 21%.
• On WikiTQ, POS reaches the highest accuracy of 74.45% with GPT4-o.

These results again confirm that POS offers a distinctive interpretability advantage and usefulness compared to existing Table QA models.

References:

• [c] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS

• [d] HIVE: Evaluating the Human Interpretability of Visual Explanations, ECCV.

• [e] What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods, NeurIPS.

• [f] Debugging Tests for Model Explanations, NeurIPS.

• [g] Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation, ICLR.

We sincerely appreciate Reviewer VQhj's thoughtful reviews on our paper!

Your comments, particularly regarding the effect of highlights in XAI comparisons, have provided us with valuable insights to improve our evaluation.

Below, we show how your insights helped us improve evaluation and humbly answer your questions as follows:

The authors argue that it is better for interpretability to merely depend on SQL to process the table and answer the question. However, the necessity of this constraint is still questionable for the table qa in practical scenarios. The tables are of various formats, and the answer usually cannot be produced by a SQL query. Such limitation may explain the lower performance of PoS compared with many other baselines. Are there any cases where the answer cannot be generated by SQL queries?

Thank you for this great comment!

Yes, there are cases where the answer cannot be generated by SQL queries alone, particularly those involving long-form generation—such as text summarization tasks in datasets like QTSumm [a]—often relying on model “creativity” to produce answers. For such datasets, relying solely on SQL is not optimal. We argue that combining the strengths of SQL for structured data handling with the creative reasoning capabilities of LLMs offer a more powerful approach.

We recognize that tables vary significantly in format, which can sometimes make SQL-based processing challenging [b]. However, by preprocessing samples into a SQL-compatible format (i.e., making them "SQLifiable"), we can achieve a significantly better interpretable decision-making process. For instance, the NormTab paper [c] demonstrates effective data cleaning techniques to prepare tables for SQL processing, showing that, after cleaning, applying naive methods like Text-to-SQL can boost accuracy on datasets like TabFact and WikiTQ by 5-10 points. This shows that POS can not only provide great interpretability but also accurate answers.

We also want to note that our primary focus in this work is on interpretability, which is why we emphasize SQL-based methods. For long-form generation, hybrid explanation methods (e.g. SQL + natural-language explanations) would be more promising and we leave this exploration for future work. We hope POS lays the groundwork for interpretability research in Table QA.

• [a] QTSumm: Query-Focused Summarization over Tabular Data, EMNLP
• [b] On the potential of lexico-logical alignments for semantic parsing to sql queries, EMNLP
• [c] NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization, EMNLP

As suggested by Reviewer VQhj, we conducted another ablation study on interpretability .

In our paper, explanations consist of attribution maps (highlights) combined with natural-language steps to (for POS) or functions (for DATER and CoTable) or SQL command (for Text-to-SQL). To assess the impact of attribution maps, we removed highlights in explanations and give to XAI judges.

We tested explanations with no highlights on three XAI tasks: Preference Ranking, Forward Simulation, and Model Debugging (an additional, more standard XAI evaluation). We repeat the LLM-judge setups as described in our paper for GPT4o-mini and report the data below:

Table: Evaluating Table QA explanations with no highlights (attribution maps).

Our findings are:

• Without highlights, POS still performs best across all three XAI tasks—Preference Ranking, Forward Simulation, and Model Debugging.
• The removal of highlights does not affect the accuracy/ratings for the explanation methods, suggesting that highlight-based explanations may not be important for LLMs perhaps because it is unknown which formats of highlights (image/xml tags/mark-up syntax) should be effective for language models. This insight could guide future developments in XAI methods for Table QA.

References:

• [a] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS
• [b] What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods, NeurIPS

2. In Appendix C, the ablation study discusses changes in interpretability after removing different modules, but no quantitative metrics are provided to measure these changes. Could the authors include specific interpretability metrics to quantify the impact of each module?

Thank you for highlighting this point!

We would like to clarify that the goal of the ablation study in Appendix C is to test how each component affects POS QA accuracy, not interpretability.

However, to address interpretability quantitatively, we can use the fallback rate as a proxy metric. In our approach, fallback occurs when an error in the pipeline occurs (e.g. due to an unexecutable SQL action) and we send the entire data to LLMs to get an answer. This approach however lacks interpretability (as shown in Fig. 1). Thus, a lower fallback rate indicates that POS can solve more queries using SQL, providing a fully interpretable decision-making process.

As shown in Table 4, modules like NL Planning and Text-to-SQL significantly contribute to interpretability, with their removal increasing the fallback rate to 40-50%. Removing NL Planning results in SQL-only steps (in lieu of natural-language steps Fig. 7) will diminish interpretability, as shown by lower performance of Text-to-SQL compared to POS in the Forward Simulation experiment (Table 1).

Similarly, without Text-to-SQL, LLM performs table transformations directly in natural language without using SQLs, which is less interpretable due to the inherent LLM black-box reasoning.

When the Atomicity is ablated out, the fallback rate does not change much. To justify the change in interpretability, we visualized the plans generated by NL Planner in Fig.8 and Fig.9 and conducted a qualitative analysis (on 200 samples), which hinted that explanations were indeed less interpretable in this setup due to complex steps (see Fig.8 and 9’s captions).

1. Given POS’s sequential nature and reliance on intermediate tables as inputs for each step, what are the efficiency implications?

Thank you for this valuable question!

To analyze the efficiency of POS, we followed the analysis used in the Chain-of-Table paper (Sec 4.5), using the total number of LLM queries as the measure of efficiency.

When comparing POS to other baselines, including Binder (Cheng et al., 2022), Dater (Ye et al., 2023), and CoTable (Wang et al., 2024), we observe a distinct advantage: POS does not rely on self-consistency (generating multiple candidate answers and selecting the most frequent or consistent one to improve accuracy) during either the planning or SQL generation steps, whereas all three baseline methods use self-consistency to boost accuracy. This helps POS significantly reduce the number of LLM queries needed per sample (to only 4), making POS the most efficient among these baselines. See the Table below.

Table: Efficiency analysis on WikiTQ

Additionally, we propose to compare the efficiency of these methods based on the number of table transformations, as a higher count leads to increased workload on the local database when handling Table QA tasks (i.e., higher number of queries to the table database).

In this regard, POS also demonstrates a clear advantage over the baselines. In particular, while methods like Chain-of-Table rely on a predefined set of 5 operations (e.g., add column, select column/row, group column, sort column) to maintain consistency, POS requires only 2 database queries on average, making it the most efficient approach in terms of database queries. Similarly, Binder and Dater require significantly greater numbers of operations, with Binder doing table database queries 50 times and Dater sending slot-filling queries 20 times.

We sincerely thank the reviewer mQDc for the feedback on our work and for recognizing POS’s contributions to interpretability in Table QA.

We particularly appreciate the suggestion regarding efficiency analysis, which has helped us better showcase the efficiency advantages of our method.

Below, we detail how your insights guided us in conducting this analysis and addressing your questions!

Limited Effectiveness and Scalability: The performance improvements with POS are incremental, indicating only a modest boost in interpretability and accuracy over baseline methods.

In this comment, Reviewer 8Hmg mentioned that: "The performance improvements with POS are incremental, indicating only a modest boost in interpretability and accuracy over baseline methods." However, in the Strength 2nd, the Reviewer mentioned: "It includes a comprehensive evaluation, using both human and LLM judges to assess interpretability and predictive accuracy, showcasing POS’s advantages in clarity, coherence, and overall effectiveness compared to existing methods."

We feel that there is mixed feedback on the interpretability of POS. As demonstrated in all 3 XAI tasks (Forward Simulation, Preference Ranking, and Model Prediction Debugging), POS consistently outperforms existing state-of-the-art Table QA methods in terms of interpretability and effectiveness.

Please let us know if this address the Reviewer's concern about "modest boost in interpretability"!

Limited Novelty: While the paper addresses an important problem in Table QA, the core ideas, such as breaking down queries into atomic steps and using SQL transformations for interpretability, are established in other domains. This reduces the method's originality, as it largely adapts existing decomposition and programmatic interpretability approaches rather than introducing fundamentally new techniques specific to Table QA.

Thank you for this insightful comment!

Reviewer 8Hmg said that: "the core ideas, such as breaking down queries into atomic steps and using SQL transformations for interpretability, are established in other domains."

While it’s true that step-by-step planning is the main idea of almost all the LLM planning papers [a], to the best of our knowledge, there is no such work in literature that do breaking down queries into atomic steps and using SQL transformations explicitly for interpretability. If there are such papers, we would be grateful if the reviewer could point them out so that we can cite and give proper credit.

We also acknowledge that our method leverages concepts from existing decomposition and programmatic interpretability approaches; however, we believe that POS introduces a novel application of these principles specifically tailored to Table QA.

Besides, we would like to emphasize that our goal is not to introduce a new Table QA method that maximizes accuracy but rather to address critical interpretability gaps in current Table QA methods. Interpretability in this domain has been largely overlooked, and POS aims to fill that gap.

To further support the interpretability frontier of POS, we conducted an additional evaluation where explanation methods (Text-to-SQL, DATER, CoTable, and POS) help us identify whether the Table QA model is correct (model prediction debugging[c,d,e,f]). Model prediction debugging is the mainstream and established task in XAI and has many real-world applications (e.g. healthcare[g]).

Using both the TabFact and WikiTQ datasets, we evaluated the following question: Given the explanations, can users identify if the Table QA model is correct or wrong? We leverage LLM-as-a-Judge as we did in Forward Simulation and Preference Ranking for this setup.

Here is our prompt to the LLM judges:

The Table Question Answering (Table QA) model is working on a tabular dataset, answering questions based on a given table.

You are given an HTML file containing a Question, Input Table, Prediction, and an Explanation clarifying the Prediction.

Your task is to carefully analyze the explanation and determine whether the Prediction is correct or not.

Explanation Method: [Text2SQL/DATER/CoTable/POS]
Answer with ‘Correct’ or ‘Wrong’ only.

You MUST ignore the order of the options and answer based on the correctness of the Prediction!

We report the result of the evaluation below:

Table: Model prediction debugging accuracy (↑) on TabFact and WikiTQ with LLM judges using different Table QA explanations.

Our findings are:

• POS significantly improves model prediction debugging accuracy compared to other Table QA baselines.
• On the TabFact dataset, POS achieves a debugging accuracy of 76.74% with GPT-4o-mini, 72.85% with GPT-4o, and 72.08% with GPT-4, outperforming other methods by margins of up to 21%.
• On WikiTQ, POS reaches the highest accuracy of 74.45 with GPT4-o.

These results again confirm that POS offers a distinctive interpretability advantage and usefulness compared to existing Table QA models.

References:

• [a] Understanding the planning of LLM agents: A survey. 2024.
• [b] Watch Your Steps: Observable and Modular Chains of Thought. 2024
• [c] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS
• [d] HIVE: Evaluating the Human Interpretability of Visual Explanations, ECCV.
• [e] What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods, NeurIPS.
• [f] Debugging Tests for Model Explanations, NeurIPS.
• [g] Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation, ICLR.

We sincerely thank the Reviewer 8Hmg once again for the insightful comments!

Dear Reviewer 8Hmg,

We sincerely thank you for the review efforts and your thoughtful feedback on our paper. We deeply appreciate that you recognize the strengths of our work, including the clear motivation for interpretability in Table QA and your acknowledgment of our comprehensive evaluation, which demonstrates POS’s advantages in clarity, coherence, and overall effectiveness compared to existing methods.

For your concerns about the technical contributions and scalability of POS, please find our detailed replies below. We hope these responses address your feedback and clarify the unique contributions and potential of POS!

Thanks for your further clarification. It solves part of my concerns. However, I still do not think the Novelty of this paper reaches the standard of ICLR. As a result, I will keep my rating.

4. Further experimentation across a broader range of datasets (ideally 5–6) would be needed to demonstrate POS’s generalizability and robustness.

Thank you for this suggestion!

We would like to note that TabFact and WikiTQ are also the primary evaluation datasets in existing works, including our main baselines—CoTable, Binder, Dater, and TableSQLify.

Due to budget and time constraints, we are currently unable to extend POS testing to additional datasets as suggested. The cost of running POS on TabFact and WikiTQ is particularly high due to the large input tables and the in-context examples required for each prompt, resulting in a high token usage. For instance, a single evaluation on

TabFact or WikiTQ using step-by-step planning (in Question 1) incurred over $600 in API costs. Given these constraints, we prioritized TabFact and WikiTQ as they offer a robust and feasible evaluation within our available resources.

If there are any additional questions or further concerns, we would be more than happy to address them and continue the discussion. We are eager to engage in any way!

Thank you once again for your time and consideration.

3. The NL Atomic Planner in POS depends on in-context planning examples that are specifically tailored to each dataset. This raises questions about its adaptability to a variety of reasoning types and Table QA problems. It remains unclear whether the current prompting schema can generalize effectively across different Table QA tasks.

Thank you for the insightful comment!

Our in-context planning examples were indeed crafted specifically for each dataset. To explore the generalizability of our prompting schema, we modified the examples to remove any dataset-specific details. These revised, general in-context examples are designed to handle typical Table QA tasks, where the input to the NL Planner is simply the Query/Table and the output is the Plan.

An example of the new generalized in-context prompt is as follows:

We are working with a tabular dataset.
Your task is to develop a step-by-step plan to answer a query on a given table.

### Table: 
/* 
col : id | name | score 
row 1 : 1 | alice | 85 
row 2 : 2 | bob | 90 
row 3 : 3 | charlie | 75 
*/ 
Query: what is the score of alice? 
Plan: 
1. Select rows where the 'name' is 'alice'. 
2. Select the 'score' column.

We ran POS again on both datasets using these generalized in-context examples. The results indicate that this change led to only a slight decrease in accuracy compared to dataset-specific examples, as shown in the table below:

Table: Table QA accuracy (↑) on TabFact and WikiTQ datasets using general in-context examples for planning.

These numbers confirm that POS remains competitive without a strong reliance on dataset-specific examples (1.5% → 3.0% accuracy gaps). Additionally, we observed that the general in-context examples led to low unprocessable rates, with TabFact rates changing from 3.16% → 3.06% and WikiTQ rates from 13.58% → 14.48%.

Finally, it is also worth noting that reliance on tailored, dataset-specific examples is common across other methods, such as CoTable. Typically, general prompts (e.g., Meta-prompting[1]) yield slightly lower accuracy than dataset-specific prompts but improve generalizability, a trend that aligns with our findings.

• [1] Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding

2. The technical contribution of POS is limited, since the whole framework primarily involves prompt engineering. Essentially, compared to traditional program-based methods, POS simply adds an additional layer of prompts to generate natural language “plans” that function as pseudo-comments for SQL statements. This does not introduce substantial technical contribution beyond traditional program-based methods.

Thank you for your comment! While POS does indeed utilize prompt engineering, its design extends beyond simple prompt layering. The key innovation lies in its systematic decomposition of complex queries into atomic, interpretable sub-steps and table transformations—a feature not found in existing approaches. Unlike traditional program-based methods, which often struggle with unexplainable selections or function arguments (Fig. 1), POS offers a fully transparent, step-by-step transformation process.

We would like to emphasize that our goal is not to introduce a new Table QA method that maximizes accuracy but rather to address critical interpretability gaps in current Table QA methods. Interpretability in this domain has been largely overlooked, and POS aims to fill that gap.

To further support the interpretability frontier of POS, we conducted an additional evaluation where explanation methods (Text-to-SQL, DATER, CoTable, and POS) help us identify whether the Table QA model is correct (model prediction debugging[c,d,e,f]). Model prediction debugging is the mainstream and established task in XAI and has many real-world applications (e.g. healthcare[g]).

Using both the TabFact and WikiTQ datasets, we evaluated the following question: Given the explanations, can users identify if the Table QA model is correct or wrong? We leverage LLM-as-a-Judge as we did in Forward Simulation and Preference Ranking for this setup.

Here is our prompt to the LLM judges:

The Table Question Answering (Table QA) model is working on a tabular dataset, answering questions based on a given table.

You are given an HTML file containing a Question, Input Table, Prediction, and an Explanation clarifying the Prediction.

Your task is to carefully analyze the explanation and determine whether the Prediction is correct or not.

Explanation Method: [Text2SQL/DATER/CoTable/POS]
Answer with ‘Correct’ or ‘Wrong’ only.

You MUST ignore the order of the options and answer based on the correctness of the Prediction!

We report the result of the evaluation below:

Table: Model prediction debugging accuracy (↑) on TabFact and WikiTQ with LLM judges using different Table QA explanations.

Our findings are:

• POS significantly improves model prediction debugging accuracy compared to other Table QA baselines.
• On the TabFact dataset, POS achieves a debugging accuracy of 76.74% with GPT-4o-mini, 72.85% with GPT-4o, and 72.08% with GPT-4, outperforming other methods by margins of up to 21%.
• On WikiTQ, POS reaches the highest accuracy of 74.45 with GPT4-o.

These results again confirm that POS offers a distinctive interpretability advantage and usefulness compared to existing Table QA models.

References:

• [c] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS

• [d] HIVE: Evaluating the Human Interpretability of Visual Explanations, ECCV.

• [e] What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods, NeurIPS.

• [f] Debugging Tests for Model Explanations, NeurIPS.

• [g] Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation, ICLR.

We sincerely appreciate Reviewer j2Vr's thoughtful feedback of our paper! Your comments, particularly on the unprocessable rate and the generalizability of in-context planning examples, provided valuable insights. We detail how your insights help us answer to your concerns below!

Thank you for the detailed reply from the authors. Although I still have concerns with regards to the novelty and the evaluation datasets, the new results have resolved some of my concerns regarding the unprocessable rate and the generalizability of in-context examples. So I decide to raise my score from 3 to 5.

Dear Reviewer j2Vr,

As promised, we extended POS (our method) to FeTaQA (Free-form Table Question Answering) dataset. On this dataset, we compared POS with End-to-End QA and Few-Shot QA, as we did with TabFact and WikiTQ. We report the results as follows:

Table: Evaluation on FeTaQA test set with gpt4o-mini.

We find that POS is better than End-to-End or Few-Shot QA in both aspects, accuracy and interpretability:

• POS consistently achieves the best scores across all metrics on the FeTaQA test set, with a BLEU improvement of 0.98 pts and ROUGE-L gain of 1.83 pts over Few-shot QA. The reason for these improvements is that POS retrieves fine-grained and correct information from tables through its step-by-step SQLs, enabling better answer generation. Here we show an example of precise table information retrieval from POS.

Table Page Title: 1948 Oregon gubernatorial special election
Table Section Title: Results
### Table:
/*
col : Party | Party | Candidate | Votes | %
row 1 : - | Republican | Douglas McKay | 271,295 | 53.23
row 2 : - | Democratic | Lew Wallace | 226,958 | 44.53
row 3 : - | Independent | Wendell E. Barnett | 11,380 | 2.23
row 4 : Total votes | Total votes | Total votes | 509,633 | 100
row 5 : - | Republican hold | Republican hold | Republican hold | Republican hold
*/
Question: By what percentage of votes did McKay win the election against Wallace's 44.53% vote?
Plan:
1. Select rows where 'Candidate' is 'McKay' or 'Wallace'.
2. Select 'Candidate' and '%' columns.
3. Ask LLM to generate the final answer based on the current table.

This information retrieval contrasts with End-to-End QA and Few-shot QA, which feed the entire large table into the LLM, making it harder for model to pinpoint relevant information and increasing the likelihood of errors.

• POS provides visual explanations for the generated answers, which End-to-End QA and Few-shot QA lack. We share an explanation from POS on FetaQA here. As POS bases on a concise, final table to generate the final answer, this makes it easier for users to understand why the model generates a particular response. In contrast, End-to-End QA and Few-shot QA require users to refer back to the large, original table, making the reasoning process less understandable and harder to follow.