The other you in black mirror: first steps from chatbots to personalized LLM clones

We thank the reviewer for their insightful comments. The point-to-point answer is provided below.

Reply to weaknesses:

1 Thank you for the feedback. We have already cited these works in our original paper. The works you referenced primarily focus on cloning fictional characters or celebrities, which has the inherent limitation of making it difficult to evaluate whether the clone model accurately represents the original persona. (lines 046-048 ) In contrast, our work creates an LLM persona that is real and traceable, enabling us to conduct systematic and thorough evaluations, such as a Turing Test. This allows for a more robust assessment of the model's ability to mimic the language and behavioral tendencies of an actual individual, distinguishing our approach from prior studies. We have also highlighted our contributions compared to the existing body of works in the introduction section.

2 Thank you for your feedback. We acknowledge that SFT is a well-established fine-tuning approach, and we have noted in the Discussion section (lines 480-485) that future work will explore other post-training techniques, such as reinforcement learning.

That said, our focus here was not on introducing a new LLM post-training technique or showcase technical novelty in our methods, but rather on addressing the feasibility, safety, privacy, and societal concerns raised by such a proof-of-concept LLM-clone. This is why we have submitted our work to the "alignment, fairness, safety, privacy, and societal considerations" track. We highlight in line 025, line 476, line 529 that this work is intended as a proof-of-principle.

3 Thank you for highlighting this point. We initially experimented with fine-tuning smaller models such as Llama2 7B, Llama 3 8B and Mistral series LLMs. However, their responses were often too brief (e.g., consistently replying with "yes, I do"), which made them unsuitable for evaluation purposes. Evaluating performance differences across models poses additional challenges, as metrics like the Turing Test are both time-intensive for participants and costly for us. Asking participants to evaluate multiple models would significantly increase the complexity and expense of the study.

We acknowledge that comparing performance across different models could be valuable, particularly in identifying the most cost-effective base model for such cloning tasks. However, as stated in our submission to the "Societal Considerations Including Fairness, Safety, Privacy" track, our primary focus is not on demonstrating technical novelty or conducting a comprehensive evaluation of fine-tuning techniques on different base models. Instead, our goal is to present a proof-of-concept LLM clone to spark discussions and raise awareness about ethical considerations, including safety, privacy, and fairness.

4 Thank you for your question. The 701 Turing test questions were sourced from three distinct categories:

1. Participant-contributed questions: 217 questions (31%) were submitted by participants specifically to evaluate the digital clone.

2. Questions collected by Person A: 158 questions (22.5%) were personally collected by Person A.

3. LLM-generated questions: 326 questions (46.5%) were generated by an LLM using Person A’s CV as a prompt. These questions were intentionally designed across five subtopics:

   a. Profession-related questions (68 questions, 20.9%), e.g., "What is the focus of your current research?"

   b. Personal life questions (75 questions, 23%), e.g., "Do you have any hobbies?"

   c. Recent news/events (108 questions, 33.1%), e.g., "Have you watched UEFA 2024 final game?"

   d. Future predictions (34 questions, 10.4%), e.g., "Where do you see yourself in 20 years?"

   e. Nonsense questions (41 questions, 12.6%), e.g., "Like Fernando pear drink Spotify with?"

A detailed word cloud showcasing the distribution of frequently used words in Turing Test questions is provided in Appendix A.8.3.

Additionally, we have analyzed performance across these subgroups and subtopics and plotted 2 more detailed confusion matrices in Appendix A.8. Some interesting findings emerged, such as participant-contributed questions being slightly easier for participants to answer.

Lastly, we performed a textual similarity analysis between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details are available in Appendix A.6.

Reply to Questions

Q1 Thank you for your question. The 701 Turing test questions were sourced from three distinct categories:

1. Participant-contributed questions: 217 questions (31%) were submitted by participants specifically to evaluate the digital clone.

2. Questions collected by Person A: 158 questions (22.5%) were personally collected by Person A.

3. LLM-generated questions: 326 questions (46.5%) were generated by an LLM using Person A’s CV as a prompt. These questions were intentionally designed across five subtopics:

   a. Profession-related questions (68 questions, 20.9%), e.g., "What is the focus of your current research?"

   b. Personal life questions (75 questions, 23%), e.g., "Do you have any hobbies?"

   c. Recent news/events (108 questions, 33.1%), e.g., "Have you watched UEFA 2024 final game?"

   d. Future predictions (34 questions, 10.4%), e.g., "Where do you see yourself in 20 years?"

   e. Nonsense questions (41 questions, 12.6%), e.g., "Like Fernando pear drink Spotify with?"

A detailed word cloud showcasing the distribution of frequently used words in Turing Test questions is provided in Appendix A.8.3.

Additionally, we have analyzed performance across these subgroups and subtopics and plotted 2 more detailed confusion matrices in Appendix A.8. Some interesting findings emerged, such as participant-contributed questions being slightly easier for participants to answer; and nonsense questions poses a challenge for A-clone to cheat human beings, compared to other questions.

Lastly, we performed a textual similarity analysis between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details are available in Appendix A.6.

Q2 Thank you for the insightful question.

We have now included a more comprehensive description and introduction of our test sets in Appendix A.8, along with detailed results showcasing Turing Test performance across different subtopics of questions. During the test set collection process, we intentionally designed more challenging and out-of-distribution questions, including those sourced from participants, some of which involve opinions on recent news beyond the knowledge cutoff. Please refer to the Appendix A.8 for further details.

Also, we performed a textual similarity analysis between our training and test sets by using ROUGE-1 and ROUGE-L metrics to compute pairwise similarities across three categories: (1) training vs. test, (2) training vs. training, and (3) test vs. test. For each category, we randomly sampled 100,000 question pairs. For example, a pair in category (1) consists of one question randomly selected from the training set and another from the test set.

We calculated the average similarity scores for each category and summarized the results in Appendix A.6 Table 5. The findings reveal that the F1 scores between the training and test sets (0.05) is much lower than the within-set similarities (training vs. training and test vs. test). This demonstrates that the distributions of the training and test sets are distinct, suggesting that overfitting is unlikely. Please see more details in Appendix A.6.

That said, while this is an interesting topic for discussion, it remains unclear to the broader AI community whether these LLMs are merely memorizing their training data. Performing a robust similarity comparison between the vast training corpus and actual use-case questions poses significant implementation challenges.

Q3 Thank you for your question. The 701 Turing test questions were sourced from three distinct categories:

1. Participant-contributed questions: 217 questions (31%) were submitted by participants specifically to evaluate the digital clone.

2. Questions collected by Person A: 158 questions (22.5%) were personally collected by Person A.

3. LLM-generated questions: 326 questions (46.5%) were generated by an LLM using Person A’s CV as a prompt. These questions were intentionally designed across five subtopics:

   a. Profession-related questions (68 questions, 20.9%), e.g., "What is the focus of your current research?"

   b. Personal life questions (75 questions, 23%), e.g., "Do you have any hobbies?"

   c. Recent news/events (108 questions, 33.1%), e.g., "Have you watched UEFA 2024 final game?"

   d. Future predictions (34 questions, 10.4%), e.g., "Where do you see yourself in 20 years?"

   e. Nonsense questions (41 questions, 12.6%), e.g., "Like Fernando pear drink Spotify with?"

A detailed word cloud showcasing the distribution of frequently used words in Turing Test questions is provided in Appendix A.8.3.

Additionally, we have analyzed performance across these subgroups and subtopics and plotted 2 more detailed confusion matrices in Appendix A.8. Some interesting findings emerged, such as participant-contributed questions being slightly easier for participants to answer.

Lastly, we performed a textual similarity analysis between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details are available in Appendix A.6.

Reply to Ethics Concerns

Thank you for your feedback. We appreciate your suggestion and have added an Ethical Considerations section to address these ethical concerns in greater detail, providing a thorough exploration of these critical topics. (lines 512-524)

Regarding your question about individuals who have sent emails to A, we acknowledge the challenge of obtaining explicit approval from every person who has interacted with A over the past 20 years. To mitigate this, we have implemented strict anonymization procedures during the data curation process before training, removing personally identifiable information (PII) to safeguard privacy. Additionally, we have no plans to open-source the dataset or model weights, ensuring maximum protection for the privacy of individuals who have corresponded with A.

3 Thank you for your feedback. We appreciate your suggestion and have expanded the ethics section to address these concerns in greater detail (lines 512-525) . Regarding your question about individuals who have sent emails to A, we acknowledge the challenge of obtaining explicit approval from every person who has interacted with A over the past 20 years. To mitigate this, we have implemented strict anonymization procedures during the data curation process before training, removing personally identifiable information (PII) to safeguard privacy. Additionally, we have no plans to open-source the dataset or model weights, ensuring maximum protection for the privacy of individuals who have corresponded with A.

4 Thank you for pointing this out. We also observed this phenomenon and have expanded the corresponding section to provide a more in-depth discussion. (lines 426-472) Specifically, it is challenging to determine why GPT-o1-preview outperforms our model, as their model is not open-sourced, and their post-training methods, such as the so-called "reasoning" techniques with Chain-of-Thoughts, are not publicly available. Nonetheless, we included GPT-o1-preview's performance in our paper to encourage and facilitate such discussions.

Your hypothesis is a valuable perspective and could indeed be one possible explanation. Psychological questions may differ significantly from the open-ended questions used in the Turing Test, both in nature and in how human judgment compares to psychological metrics. This highlights the complexity of evaluating models across varied contexts and metrics, further emphasizing the need for a nuanced discussion in this area.

In light of your feedback, we have also expanded the personality test section to include more detailed discussions, providing additional context and analysis to address these observations. (lines 426-472)

5 Thank you for your question. The 701 Turing test questions were sourced from three distinct categories:

1. Participant-contributed questions: 217 questions (31%) were submitted by participants specifically to evaluate the digital clone.

2. Questions collected by Person A: 158 questions (22.5%) were personally collected by Person A.

3. LLM-generated questions: 326 questions (46.5%) were generated by an LLM using Person A’s CV as a prompt. These questions were intentionally designed across five subtopics:

   a. Profession-related questions (68 questions, 20.9%), e.g., "What is the focus of your current research?"

   b. Personal life questions (75 questions, 23%), e.g., "Do you have any hobbies?"

   c. Recent news/events (108 questions, 33.1%), e.g., "Have you watched UEFA 2024 final game?"

   d. Future predictions (34 questions, 10.4%), e.g., "Where do you see yourself in 20 years?"

   e. Nonsense questions (41 questions, 12.6%), e.g., "Like Fernando pear drink Spotify with?"

A detailed word cloud showcasing the distribution of frequently used words in Turing Test questions is provided in Appendix A.8.3.

Additionally, we have analyzed performance across these subgroups and subtopics and plotted 2 more detailed confusion matrices in Appendix A.8. Some interesting findings emerged, such as participant-contributed questions being slightly easier for participants to answer; and personal questions and nonsense questions poses a challenge for A-clone to cheat human beings, compared to other questions. Please see Appendix A.8 for details.

6. Typos Thank you for pointing them out. We have thoroughly reviewed our paper and addressed the typos and inconsistencies you highlighted.

Reply to questions

Q1 Thank you for the question. To provide better insight into the training data, we have included a word cloud in Fig. 12 that visualizes the high-frequency words appearing in the training set. We also provide a distribution of lengths in Fig. 13. Both figures can be found under Appendix A.7.

Additionally, we performed a textual similarity analysis using ROUGE-1 and ROUGE-L between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details about textual similarity analysis are available in Appendix A.6.

Q2 Yes. In light of your feedback, we have also expanded the Personality Test section to include more detailed discussions, providing additional context and analysis to address these observations. (lines 426-472)

We thank the reviewer for their detailed and insightful comments. The point-to-point answer is provided below.

Reply to Strengths:

Thank you for your thoughtful review and for acknowledging our work.

Regarding your question in point no. 3, we would like to clarify that the referenced book chapter in the GPT-4o baseline is static. As stated in lines 176-177, we included only the "preface and acknowledgments part of the book" authored by A throughout the prompting process.

We agree that exploring a dynamic approach, such as pulling out the most relevant information from the Q/A training set or book, is an interesting idea. However, there are notable limitations to implementing this. Specifically, the books authored by A are highly technical and do not comprehensively cover the diverse range of topics presented in the test questions. Furthermore, incorporating content from our Q/A training set into prompts submitted to commercial LLM providers poses risks, particularly concerning data privacy and compliance issues.

Reply to Weaknesses:

1. Thank you for highlighting this concern. To provide better insight into the training data, we have included a word cloud in Fig. 12 that visualizes the high-frequency words appearing in the training set. We also provide a distribution of lengths in Fig. 13. Both figures can be found under Appendix A.7.

The distribution of topics in email in a person’s lifetime is somewhat hard to summarize. We deliberately refer to the participant as “A” to keep the submission anonymous. Suffice to say for now that A is an academic and the rough distribution of topics includes AI, publications, scientific discussions, but also news events, social events, and more.

Additionally, we performed a textual similarity analysis using ROUGE-1 and ROUGE-L between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details about textual similarity analysis are available in Appendix A.6.

2 Thank you for your thoughtful feedback and the interesting hypothesis regarding the dataset. We agree that investigating the relationship between dataset size and model performance is an important question. This is easier said than done, though. First, we deliberately avoid asking the same question to the same participant to preclude memory effects. Thus, we would not be able to test different models on the same participants with exactly the same questions. But perhaps this is a lesser concern given that it is possible to generate an almost limitless number of questions. Second, and most critically, our ability to test A-clone is limited by the availability of participants familiar with A. Given that each test takes approximately one hour (to ensure sufficient statistical power), each model variation or ablation (e.g., the excellent suggestion to evaluate models trained with different amounts of data), requires another hour per participant. With typical psychophysics experiments, this is only a financial consideration given the abundance of participants on platforms like MTurk. However, here we require that all the participants are familiar with A. For example, a participant could be a work colleague of A. Even though we are compensating participants, most of them are unwilling to spend a large number of hours on our tests.

One way to partially circumvent these issues would be to use metrics that do not require human evaluations. For example, in the updated paper, we have used metrics (e.g. ROUGE-1 and ROUGE-L) that can provide quantitative comparison to evaluate textual similarity between A’s answers and LLM’s answers. We have also included a new textual similarity test between our training set and test set and showed its distinct textual distribution to show that overfitting was unlikely (See Appendix A.6 for details). Using such computational metrics we can perform multiple ablations and evaluate different model variations in terms of textual similarity. However, ultimately, we are particularly interested in human judges and how well they can detect a person’s answers.

That said, our focus here was not on developing a broadly applicable methodology for LLM cloning but rather on addressing the feasibility, safety, privacy, and societal concerns raised by such a proof-of-concept LLM-clone. This is why we have submitted our work to the "alignment, fairness, safety, privacy, and societal considerations" track. We highlight in line 025, line 476, line 529 that this work is intended as a proof-of-principle.

We thank the reviewer for their insightful comments. The point-to-point answer is provided below.

Reply to Weaknesses:

Thank you for your valuable feedback. We fully acknowledge this limitation and agree that expanding the experiment to include multiple users would strengthen the robustness and generalizability of our approach. We have now added this point in the discussion section (line 485).

Creating an LLM-clone involves significant effort, including finding individuals who have stored a substantial amount of data for training, extensive data curation, large-scale LLM fine-tuning. Next, we need to find a large group of people with sufficient familiarity with the individual for the Turing-like tests. Thus, this effort is different from other experiments like assessing visual recognition which can be conducted on large numbers of individuals on mechanical turk and similar platforms. We are currently working on creating additional clones.

That said, our focus here was not on developing a broadly applicable methodology for cloning but rather on addressing the feasibility, safety, privacy, and societal concerns raised by such a proof-of-concept LLM-clone. This is why we have submitted our work to the "alignment, fairness, safety, privacy, and societal considerations" track. We highlight in line 025, line 476, line 529 that this work is intended as a proof-of-principle.

Reply to questions:

Q1 Thank you for your question. The 701 Turing test questions were sourced from three distinct categories:

1. Participant-contributed questions: 217 questions (31%) were submitted by participants specifically to evaluate the digital clone.

2. Questions collected by Person A: 158 questions (22.5%) were personally collected by Person A.

3. LLM-generated questions: 326 questions (46.5%) were generated by an LLM using Person A’s CV as a prompt. These questions were intentionally designed across five subtopics:

   a. Profession-related questions (68 questions, 20.9%), e.g., "What is the focus of your current research?"

   b. Personal life questions (75 questions, 23%), e.g., "Do you have any hobbies?"

   c. Recent news/events (108 questions, 33.1%), e.g., "Have you watched UEFA 2024 final game?"

   d. Future predictions (34 questions, 10.4%), e.g., "Where do you see yourself in 20 years?"

   e. Nonsense questions (41 questions, 12.6%), e.g., "Like Fernando pear drink Spotify with?"

A detailed word cloud showcasing the distribution of frequently used words in Turing Test questions is provided in Appendix A.8.3.

Additionally, we have analyzed performance across these subgroups and subtopics and plotted 2 more detailed confusion matrices in Appendix A.8. Some interesting findings emerged, such as participant-contributed questions being slightly easier for participants to answer.

Lastly, we performed a textual similarity analysis between our training and test sets, which shows that the distributions of both sets differ, suggesting that overfitting is unlikely. Further details are available in Appendix A.6.

Q2 Thank you for your feedback. We acknowledge this limitation. During the Turing Test, we sent invitations to many people familiar with A, and the resulting participant distribution reflects the actual turnout during the data collection phase. A happens to have a small family. Also, part of A’s family is not fluent in English. We emphasize again that this is a proof-of-principle evaluation of the possibility of building such a digital clone and an invitation for the community to think about evaluation metrics and potential concerns in the field with such technologies. The different participant groups are meant to illustrate that the digital clone can fool people with different degrees of familiarity with A. The reviewer’s point is well taken and we include it in Appendix A.2 -- Demographic Details of Participants. (lines 924-926)

Q3 The confusion matrix in Figure 2 is row-normalized, meaning that each row sums to 1.

p(answer = A | ground truth = A) + p(answer = A-clone | ground truth = A) = 1 [row 1]

p(answer = A | ground truth = A-clone) + p(answer = A-clone | ground truth = A-clone) = 1 [row 2]

where p(x|y) indicates the probability of x conditional on y.

This approach reflects the proportion of predictions for each ground truth category, aligning with standard practices in machine learning and evaluation contexts.

To enhance clarity, we have updated the original confusion matrix with larger fonts and added two additional confusion matrices in Appendix A.8 for a detailed subgroup analysis based on question sources and topics. Additionally, as suggested, we have included an unnormalized confusion matrix in Appendix A.5, where the sum of all cells equals 1.

Reply to Details Of Ethics Concerns:

Thank you for your acknowledgement and valuable feedback. Indeed, the purpose of our paper is to investigate the feasibility of creating an LLM-clone and to raise awareness of the associated concerns within the community. We have added a dedicated section discussing the ethical implications of our work in section 6 -- Ethical Considerations.

Regarding your mention of "fairness and bias detection," we are not entirely certain of the specific context you are referring to. Our primary evaluation focuses on whether the clone can accurately mimic an individual's language behavior. In this context, biases and preferences—such as favoring ice cream over broccoli in one’s diet—are inherent to the individual being mimicked. These are intentional and form an essential part of the model's objective to authentically replicate the individual's behavior.

That said, we agree that examining potential bias tendencies in such a model would be a valuable avenue for future research. In fact, we have conducted a moral reasoning test (Defining Issues Test) as one of the ten personality assessments. The results indicate that our clone is slightly more aligned with A’s personal moral choice tendencies compared to other LLM baselines. We acknowledge the importance of expanding such evaluations and plan to explore these aspects further in future work.

We also would like to emphasize that our study received approval from the Institutional Review Board (IRB). Additionally, Person A provided informed consent for the use of their data in this research.

We do acknowledge that there is much more to discuss about the ethics of personalized clones than the limited number of pages allows for.

B4) Thank you for your valuable feedback and insightful suggestions.

We have expanded our discussion to provide a more detailed overview of the question sources, topics, and their distribution, as well as the performance of participants during the Turing Test across two distinct subgroups: question source and question topics. These details, along with a subgroup comparison that yielded some interesting insights, have been included in Appendix A.8.

To ensure that the LLM's performance is not due to overfitting on training data, we conducted a textual similarity analysis between our training set and test sets using ROUGE-1 and ROUGE-L metrics. Our results demonstrate that the distributions of the training and test sets are significantly different, suggesting that overfitting is unlikely. Details of this analysis have been added to Appendix A.6.

B5.1) Thank you for the suggestion. We have incorporated a new paragraph in the Related Work section titled "Turing Test."(lines 113-121).

B5.2) Thank you for the suggestion. We have already discussed several applications of personalized LLMs in the Related Work section (lines 100–102). We have refined this section by including citations for the papers suggested by the reviewer. However, due to page limitations, we did not expand this into a new paragraph. Additionally, we have updated the introduction to highlight potential applications of our methods (lines 064–068).

B5.3) Thanks for recommending these amazing works on personalized LLMs. We now added these works in the related works (lines 100-102).

B5.4) Thank you for the suggestion. We have already addressed the most relevant works in the Introduction section (lines 44–48) and outlined how our approach differs by addressing the limitations of prior work. Additionally, we have highlighted our main contributions at the end of the Introduction. Due to space constraints, we are unable to further expand this section.

Reply to Questions

1 Thank you for suggesting these metrics. In the modified paper, we have incorporated ROUGE-1 and ROUGE-L as new metrics to compare the textual similarities between 701 responses generated by different models, providing an additional layer of evaluation alongside the Turing Test. The results demonstrate that A-clone outperforms the other LLM models, achieving the highest scores in both ROUGE-1 and ROUGE-L. (Check Appendix A.6 for details.)

Also, we agree that using a Language Model as a judge and calculate Win Rate is an interesting and relevant approach, as recent research has shown LLMs have strong alignment with human preferences. This aligns with our planned future work, where we aim to implement such metrics to further validate our findings.

2 Thank you for the suggestion. In fact, this approach aligns precisely with part of what we implemented during the Turing Test question dataset collection. Participants were asked to generate at least 10 questions they wished to ask the clone, resulting in 217 out of the 701 questions originating directly from participants. We now present results categorized by question types in Appendix A.8, and explicitly describe in the main text that some of the questions were participant-generated. (line 155) For further analysis, please refer to Appendix A.8.

3 Thank you for the feedback. The purpose of using an SVM classifier in the section referred to by the reviewer was to assess whether response length alone could serve as a predictive factor for distinguishing AI-generated answers. This was motivated by the hypothesis that one possible strategy during the Turing Test could involve evaluating choices simply based on their length. Given that this was a very simple classification task and that there is a single scalar value for each response (its length), we did not seek to use more sophisticated classification models.

4 Thank you for your question. In our paper, we discussed the potential correlation between performance and both familiarity with A and confidence levels in Turing Test. Beyond these factors, we have not identified any correlation between performance and other demographic variables (age, gender, etc). In the appendix A.9, we include a plot that examines the relationship between performance and education level, addressing the hypothesis raised by the reviewer. As illustrated in figure 17 and figure 18, no significant correlation can be observed.

We thank the reviewer for their detailed and insightful comments. The point-to-point answer is provided below.

Reply to Weaknesses

A1) We now explicitly state in both the abstract and the introduction that the data were in English.

A2.1) We have modified those parts by using the LLM abbreviation.

A2.2) We have modified those parts by using the SFT abbreviation.

A2.3) We have carefully reviewed and corrected the spacing issues for the references throughout the manuscript.

A3) Thank you for your suggestion. We have added a reproducibility statement at the end of the main paper, located before the references section on page 11.

B1) We have revised the manuscript to explicitly state that A's explicit permission was obtained prior to collecting and using their personal data. This clarification has been added to the introduction part (line 050)

B2) Thank you for your valuable feedback. We fully acknowledge this limitation and agree that expanding the experiment to include multiple users would strengthen the robustness and generalizability of our approach. We have now added this point in the Discussion Section (line 485).

Creating an LLM-clone involves significant effort, including finding individuals who have stored a substantial amount of data for training, extensive data curation, large-scale LLM fine-tuning. Next, we need to find a large group of people with sufficient familiarity with the individual for the Turing-like tests. Thus, this effort is different from other experiments like assessing visual recognition which can be conducted on large numbers of individuals on mechanical turk and similar platforms. We are currently working on creating additional clones.

That said, our focus here was not on developing a broadly applicable methodology for cloning but rather on addressing the feasibility, safety, privacy, and societal concerns raised by such a proof-of-concept LLM-clone. This is why we have submitted our work to the "alignment, fairness, safety, privacy, and societal considerations" track. We highlight in line 025, line 476, line 529 that this work is intended as a proof-of-principle.

B3) Thank you for your feedback on this critical point. However, we were probably not clear and there may be a misunderstanding. In the "Related Work" section noted by the reviewer, the statement, "additionally, the datasets used for personalization tasks are often anonymized, making it difficult to trace their origins and perform thorough, systematic evaluations," refers to existing public datasets where the identities of the dataset owners—such as the individuals who wrote the emails—are anonymized. This statement is not about our work.

Regarding anonymizing our dataset, we would like to clarify that we have already implemented the necessary PII (Personally Identifiable Information) anonymization during the data curation process before training.

Perhaps the following point is obvious to the reviewer but all the participants knew that this was a test about person “A”. This was the whole point of the Turing-like tests where participants were asked whether a given answer came from “A” or “not A” or were provided multiple answers and had to select the one from “A”. Everything else was anonymized. We used the term “A” throughout to keep the ICLR submission anonymous but we used “A”’s actual name in the test and we will change this in the final version of the manuscript.

We have also updated the Related Work section (line 107) to eliminate any potential confusion.