IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector

Thank you for your valuable feedback and the time dedicated to reviewing our work. We address your concerns and questions as follows.

[W1] The motivation behind using Phi3 medium Instruct as the underlying model for all the components is not provided. Were other models considered and discarded?

• Low Computational Costs compared with other similar-sized models.

  • As discussed in Section 2.3, the inference procedure of our framework is designed to be computationally efficient and accessible:

  • It involves three calls to phi-3-medium-128k-instruct, a lightweight, open-sourced LLM. - It includes one call to GPT-3.5-turbo via API, which introduces fixed latency but no local compute cost.

  • The total computational complexity is bounded by: O(3 · L · n² · d + OpenAIapi) where L = 32 is the number of layers, and d = 3072 is the hidden dimension of phi-3. To support this, we compare the per-layer parameter cost (L × d) of phi-3 with other similarly sized models:

[W2 Q4] The experimental results focus on AUROC & Recall as metrics. To take into account dataset imbalance, F1 scores should also have been included.

• For Tabel 1, we could include the F1 scores since we referred to OUTFOX which does include the F1-score in their results (OUTFOX Table 3).

• For Table 2 and 3, we referred to DetectRL, which does not include F1-score in their results (Table 5). Therefore, we reimplemented the baseline results in Table2 and 3 using the datasets described in the response of reviewer uVgU A2 (500 LGT and 500 HWT under each dataset) :

• For Table 4, we only have LGT for the three structured datasets, therefore we only include the Machine-rec score.

[Q1] Validation set in lines 200-209

• We use the in-distribution 1,000 LGT and 1,000 HWT which are both generated by gpt-3.5-turbo. And we split them into 500 validation set and 500 testing set each. We used the validation set to determine the threshold.

[Q2] In comparing with ChatGPT, how did the authors arrive at the prompt used (Appendix D)?

• The prompt in IPAD's Appendix D is from DPIC's Appendix B Table 5.

[Q2] Were other less convoluted prompts tried as well, especially those that break the task down into steps and involve querying the LLM at each step?

• Yes, we did explore alternative prompts beyond the original DPIC. In particular, we evaluated three prompts: RPE [1], DP1[2], and DP2 [3]. The results show that IPAD's Prompt Inverter has better performance.

• Prompts | Source| Prompt|
  | --- |--| | [1] RPE | You are an expert prompt detective. Given the following LLM response, guess a likely prompt that could have produced it. Return only the prompt. <INPUT> | | [2] DP1 | What question are you asked if you can generate the following answer? <INPUT> | | [2] DP2 | Assume you are the prompt reconstructor who can help the user generate the prompts based on the given answers. Can you generate the original prompts for the following answer? <INPUT> | | DPIC | I want you to play the role of the questioner. I will type an answer in English, and you will ask me a question based on the answer in the same language. Don’t write any explanations or other text, just give me the question. <INPUT> |

• LGT Results | Metric | RPE | DP1 | DP2 | DPIC | Prompt Avg | IPAD | | ---------------- | ------------ | -------- | -------- | -------- | ---------- | ----- | | BART-large-CNN ↓ | -1.98 | -2.43 | -2.18 | -2.12 | -2.1775 | -1.84 | | Sentence-BERT ↑ | 0.58 | 0.41 | 0.43 | 0.46 | 0.47 | 0.69 | | BLEU ↑ | 7.28E-03 | 3.46E-05 | 9.32E-05 | 5.61E-05 | 1.87E-03 | 0.24 | | ROUGE-1 ↑ | 0.17 | 0.09 | 0.08 | 0.04 | 0.095 | 0.51 |

• HWT Results | Metric | RPE | DP1 | DP2 | DPIC | Prompt Avg | IPAD | | ---------------- | ------------ | -------- | -------- | -------- | ---------- | ----- | | BART-large-CNN ↓ | -2.78 | -3.03 | -2.85 | -2.47 | -2.7825 | -2.22 | | Sentence-BERT ↑ | 0.47 | 0.35 | 0.39 | 0.42 | 0.4075 | 0.57 | | BLEU ↑ | 8.42E-04 | 3.65E-06 | 3.25E-05 | 8.75E-06 | 2.22E-04 | 0.13 | | ROUGE-1 ↑ | 0.12 | 0.12 | 0.04 | 0.06 | 0.085 | 0.39 |

[1] Li, H., & Klabjan, D. (2024). Reverse Prompt Engineering. arXiv e-prints, arXiv-2411.

[2] Sha, Z., & Zhang, Y. (2024). Prompt Stealing Attacks Against Large Language Models. CoRR.

[Q5] Was the rating on a 1-5 scale? Further, how were judgements from the 10 participants combined for the final rating?

• Yes it is on a 1-5 scale. The assessment questions are listed as follows: | Aspects | Questions | Rates (1-5) | |-----------------------------------------------------|---------------------------------------------------------------------------------------------------|-------------| | 1. Transparency, Scrutability, and Education | Do you think IPAD provides clearer and more understandable explanations for its decisions?** | | | 2. Trust and Persuasiveness | Do you find IPAD’s outputs more trustworthy and convincing? | | | 3. Satisfaction, Effectiveness, and Efficiency | Do you think IPAD is more effective and efficient in performing its detection tasks? | | | 4. Debugging and Error Handling | Does IPAD allow you to identify and correct its mistakes more easily? | |

[Q5] What was meant by IPAD version 2 in line 219?

• Sorry, this is a typo. It should be just "IPAD". Thank you for pointing it out. We will revise it accordingly.

Thank you for your valuable feedback and the time dedicated to reviewing our work. We address your concerns and questions as follows.

[W1][Q1] The experimental comparison is outdated.

• We have already included two 2024 baselines (Binoculars [1] and Fast-DetectGPT [2]) in Table 2–4,
• We have expanded the comparisons to include five recent baselines from 2024 and 2025.

• These results show that the updated baselines perform reasonably well, but still fall short of IPAD.

[W2][Q2] Sufficient Evidence and Ablation Study Comparing with DPIC.

• Both IPAD and DPIC consist of a Prompt Inverter (PI) and a corresponding classifier, and they have clear differences on both stages.

• PI:

  • DPIC uses a fixed prompt for zero-shot prompt generation without any supervision signal, and we found that zero-shot prompt generation performs significantly worse than IPAD's fine-tuned PI (see Table 5).
  • IPAD fine-tunes the PI using prompt-generated text pairs by using both general and essay-style data, which results in more plausible prompts that benefits the AI detection performance.

• PI-Based AI Detection:

  • DPIC simply adopts a DNN model using text embeddings as input to distinguish the human-written and AI-generated text.
  • IPAD uses two fine-tuned LLMs to perform AI detection, including:
    • (a) Prompt-Text Consistency Verifier (PTCV): measures the alignment between the predicted prompt via IPAD's PI and input raw text, i.e., "Can LLM generate [text] via prompt [IPAD's predicted prompt]?".
    • (b) Regeneration Comparator (RC): measures the alignment between the regenerated text via IPAD's PI and input raw text, i.e., "[Generated text using IPAD's PI as prompt] is LLM-generated, determine whether [text] is also LLM-generated?".
  • By using both PTCV and RC, our IPAD has more superior AI detection performance with promising OOD performance and robustness.

• As suggested by the reviewer, the fine-tuned Prompt Inverter are always expected to outperform the zero-shot one as in DPIC. To show the effectiveness of our AI detection modules PTCV and RC, we compare our methods with the modified DPIC using our fine-tuned PI instead of the zero-shot PI as in the original DPIC.

• From above, IPAD's PTCV and RC each bring significant gains, and their combination as the complete IPAD method achieves the best.

[Q3] Details and analysis on fine-tuning data and OOD data.

• Generalization: To simulate biased training data selection, each training sample $X_i$, which is the embedding of the input text obtained from Qwen1.5-7B-instruct, is assigned a sampling probability $p_i=\sigma(W\cdot X_i)$, where $W \sim \mathcal N(a, 1)$ and $\sigma(\cdot)$ denotes the sigmoid function.

  • We randomly sample 5,000 training samples according to $p_i$, and use a fixed set of 1,000 test samples for evaluation on both training PTCV and RC.

• We additionally introduce label noise by randomly flipping 5% and 15% of the training labels within the above sampled training sets to evaluate the robustness of our method.

• The results indicate that our method generalizes well even under biased sampling strategies, which induce distributional shifts. Moreover, the model remains robust under label noise, with only moderate degradation at 5% and 15% noise levels.

[W3] Poor writing quality Thanks for pointing this issue. We will integrate the main results into a clear table at the beginning of Section 3.

[W1] Related Work

Thank you for your kind suggestion! We will add the following related work in our revised version. The state-of-the-art AI detection methods can be broadly divided into four categories:

• Probability‑based approaches treat a language model’s output distribution as a signal. DetectGPT [8] and Fast‑DetectGPT [2] notes that LGT tends to lie in regions of negative curvature of a language model’s log‑probability function. Lastde [9] mines token probability sequences using time‑series analysis to capture temporal dynamics. Glimpse [7] extends white-box detectors to proprietary models by predicting full distributions from partial observations.

• Regeneration‑based methods generate alternative versions of the text and compare them with the original. RAIDAR [4] and MAGRET [10] observe that LLMs modify human‑written text more than AI‑generated text when tasked with rewriting. DNA‑GPT [3] generates the remaining part of a truncated text and compares n‑gram differences. TOCSIN [11] measures semantic difference after random deletion of tokens and combining this signal with an existing zero‑shot detector.

• Syntax‑based methods leverage structural linguistic cues rather than probability estimates. PRDetect [12] notes that HWT and LGT differ in their syntactic structures, so it extracts syntax‑tree features to achieve high accuracy and robustness. Text Fluoroscopy [13] extracts the layer with the largest distribution difference between first and last layers to detect synthetic texts across domains.

• Contrastive‑learning approaches train detectors to learn writing styles or domain‑invariant features. FAID [5] collects a multilingual, multi‑domain dataset and introduces a fine‑grained classification framework. DeTeCtive [14] argues that detection should distinguish writing styles rather than simply classify LGT versus HWT. Advacheck’s multi‑task system [6] shares a transformer encoder among several heads, with one binary head for classification and auxiliary heads for domain classification.

---

References

[1] Hans et al. Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. ICML 2024.

[2] Bao et al. Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ICLR 2024.

[3] Yang et al. DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. ICLR 2024.

[4] Mao et al. Raidar: generative ai detection via rewriting. ICLR 2024.

[5] Ta et al. FAID: Fine-grained AI-generated Text Detection using Multi-task Auxiliary and Multi-level Contrastive Learning. arxiv 2025.

[6] Gritsai et al. Advacheck at GenAI Detection Task 1: AI Detection Powered by Domain-Aware Multi-Tasking. 2025.

[7] Bao et al. Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection. ICLR 2025.

[8] Mitchell et al. DetectGPT: zero-shot machine-generated text detection using probability curvature. ICML 2023.

[9] Xu et al. Training-free LLM-generated Text Detection by Mining Token Probability Sequences. ICLR 2025.

[10] Huang et al. MAGRET:Machine-generated Text Detection with Rewritten Texts. ACL 2025.

[11] Ma et al. Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness.EMNLP 2024.

[12] Li et al. PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree. NAACL 2025.

[13] Yu et al. Text fluoroscopy: Detecting LLM-generated text through intrinsic features. EMNLP 2024.

[14] Guo et al. Detective: Detecting ai-generated text via multi-level contrastive learning. NeurIPS 2024.

We supplemented the updated Pompr Inverter Methods

• Optimization-Based Methods aim to reconstruct prompts by directly optimizing input tokens or embeddings to minimize the difference between generated outputs and the target. Techniques such as Prompt Inversion Attack (PIA) [3] and SODA [4] use gradient-based optimization or discrete search to reconstruct prompts from internal model representations or output probabilities.
• Embedding-Matching & Generative Methods utilize learned mappings from LLM outputs, hidden states, or embeddings to prompt text. Methods like Logit2Prompt [1], Output2Prompt [2], PILS [5], and generative embedding inversion [8] train encoders or decoders to predict prompts from distributions or embeddings.
• Contrastive Decoding Methods analyze how slight variations in prompts affect outputs to infer salient prompt features. ROSE [10] uses contrastive prompts to identify and suppress harmful behaviors, while prompt classification methods like [7] extract prompt styles by contrasting against known patterns. Though typically not reconstructing verbatim prompts, these methods are valuable for recovering structural or intent-level information, often in low-resource or stealthy scenarios.
• Diffusion-Based Approaches remain largely conceptual in textual prompt inversion. While diffusion methods are well-established in image prompt recovery, they have yet to gain traction for discrete language tasks. Nonetheless, some conceptual work (e.g., ROSE [10]) hints at iterative refinement or contrastive correction mechanisms that could serve as proto-diffusion analogues for text, suggesting an interesting future direction.

We also explored three different prompts in the performance of prompt inverter.

Prompts

LGT Results

HWT Results

[1] Morris et al. Language Model Inversion. ICLR 2024.

[2] Zhang et al. Extracting Prompts by Inverting LLM Outputs. EMNLP 2024.

[3] Qu et al. Prompt Inversion Attack against Collaborative Inference of Large Language Models. SP 2025.

[4] Skapars et al. GPT, But Backwards: Exactly Inverting Language Model Outputs. In The 2nd Workshop on Reliable and Responsible Foundation Models at ICML 2025.

[5] Nazir et al. Better Language Model Inversion by Compactly Representing Next-Token Distributions. arXiv:2506.17090.

[6] Gao et al. DORY: Deliberative Prompt Recovery for LLM. Findings of ACL 2024.

[7] Sha et al. Prompt Stealing Attacks Against Large Language Models. CoRR 2024.

[8] Li et al. Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence. Findings of ACL 2023.

[9] Huang et al. InversionView: A General-Purpose Method for Reading Information from Neural Activations. NeurIPS 2024.

[10] Zhong et al. ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding. Findings of ACL 2024.

[11] Li, H., & Klabjan, D. (2024). Reverse Prompt Engineering. arXiv e-prints, arXiv-2411.

Dear reviewer,

I would really benefit from an updated response and engagement with the authors rebuttal. In their response, they seem to have addressed several of your concerns, but I would really like to have your input if there are still important missing data or insights that you would like expanded in the current reviewer-author discussion phase.

The discussion period has been extended 2 days, but it would be great to see some engagement today.

Thank you, Your AC.

Dear Reviewer LAhb,

Thank you for your feedback and we are happy that most of your concerns are addressed.

Specifically, I don't get the motivation behind using two finetuned LLMs instead of using a DNN model.

We respectfully believe that we have thoroughly discussed the motivation of using two mutually-beneficial fine-tuned LLMs, but we still add more experiments to fully address your concern.

• Unlike DPIC simply adopts a DNN model on the text embeddings, IPAD uses two fine-tuned LLMs to perform AI detection, most importantly, they are mutually-beneficial (we also validate this via extensive experiments):

  • (a) Prompt-Text Consistency Verifier (PTCV): measures the alignment between the predicted prompt via IPAD's PI and input raw text, i.e., "Can LLM generate [text] via prompt [IPAD's predicted prompt]?".
  • (b) Regeneration Comparator (RC): measures the alignment between the regenerated text via IPAD's PI and input raw text, i.e., "[Generated text using IPAD's PI as prompt] is LLM-generated, determine whether [text] is also LLM-generated?".

• Importantly, the 2 LLMs performs AI detection from different prospectives. Specifically, LLM 1 (PTCV) focuses on "whether a prompt can generated a given text" and LLM 2 (RC) focus on "whether the generated text is same as the raw text", making they mutually-beneficial.

• The average performance of DNN is only 54.29%, while using LLM 1 and LLM 2 separately can achieve 94.24% and 93.76%, respectively, and using 2 LLMs simultaneously can achieve 97.47% - these results show that using LLMs for detecting LLMs (that is, AI for AI) brings very strong performance gain!

---

• To fully support our claim that LLMs significantly outperform DNN, we add experiments on training varying layers DNNs using the same training data.

• For a fair comparison, we train all DNNs and LLMs with the same encoder (gte-Qwen1.5-7B-instruct).

• As shown, even the best-performing DNN falls short across in-distribution, OOD, and attacked settings.

Thank you for your valuable feedback and the time dedicated to reviewing our work. We address your concerns and questions as follows.

[W] Cost of Fine-tuning & Multi-Call Overhead

• Single-time training: We only need to train our model once, and it can generalize across various LLMs. For instance, in addition to the four common LLMs shown in Table 1, we also include results from two more models used in [1] to support this argument:

• Efficient runtime: Our model achieves high runtime efficiency compared to similarly sized models.

  • As discussed in Section 2.3, the inference procedure of our framework is designed to be computationally efficient and accessible:

  • It involves three calls to phi-3-medium-128k-instruct, a lightweight, open-sourced LLM.

  • It includes one call to GPT-3.5-turbo via API, which introduces fixed latency but no local compute cost.

  • The total computational complexity is bounded by: O(3 · L · n² · d + OpenAIapi) where L = 32 is the number of layers, and d = 3072 is the hidden dimension of phi-3. To support this, we compare the per-layer parameter cost (L × d) of phi-3 with other similarly sized models:

• Better than zero-shot methods:

On average, our fine-tuned approach consistently outperforms zero-shot baselines.

[Q1] Does the OOD Distribution Only Contain HWT Samples?

• The OOD Distribution in Table 2 and 3 contians both HWT and LGT, while Table 4 only contains the LGT. Below we detail the dataset composition for OOD evaluation:

[Q2] Figure 3: Why Does Adversarial Attack Sometimes Improve Performance?

• This is not an evaluation error. We observed that Prompt Attack and Paraphrase Attack datasets yield slightly higher AUROC than the non-attacked Yelp dataset under the FT Prompt setting.

• The arxiv, xsum, writing, and review datasets represent standard OOD.
• The prompt, paraphrase, and perturbation datasets are attacked OOD.
• These categories are orthogonal, so only averages across all settings are comparable.

Most importantly, the average AUROC without attacks is higher than with attacks, which aligns with expected adversarial degradation.