• The methods studied don't address the core challenges the authors themselves present. While the authors highlight issues that most training data used for this task occurs in a particular thematic framework as a limitation, the data augmentations they use, which are all methods which have been carefully studied in existing work but are not attributed, only perform local word and structure level permutations on data.
• The paper, as written now, spends much more time explaining (somewhat tangential) background than seems necessary. The concrete experimental setup doesn't begin until almost halfway through the work.
  
  For example, the section on Unsloth in section 2.4 and the inclusion of external model evaluations on benchmarks that are unrelated to personalization in both Figure and Table 1 have little to do with the core research questions. These sections distract from the narrative of the paper and should be moved to the appendix to make room for more substantive experiments on the primary topic.
• The work primarily presents a study on just two characters from role-playing that the paper both introduces itself and compares minimally to any external baselines. It is poorly contextualized in the field without evaluations on datasets from the broader community or comparisons to even simple baselines like prompting the model to roleplay.
• There are many signs that the work was minimally refined before submission. Despite having only 11 citations , in these minimal citations there are obvious errors such as listing "Journal Name" as the publication venue and "Unknown Author et al" as the authors (the work in reference has both publicly known authors and a venue of publication: https://openreview.net/forum?id=rc2h1h89aDi). The figures have mislabeled legends (in figure 3 BT Loss is repeated twice) and there are typos where content is duplicated at the start of paragraphs (Line 79: "Wei and Zou (2019) Wei & Zou (2019) introduced")

• Overall lack of clarity. It is unclear to me what is the major claim of the paper. The paper mention several techniques on data augmentation methods, but none of the it's explained in details nor formally. The results are also mysterious, leaving out most of the details.
• The results on the paper are non-sensical, the plots are meaningless, showing no particular trends.
• The paper contains screenshots from other papers (llama3), without a clear citations.

Regarding the proposed dataset:

• It is not clear where the training data comes from and how it was curated
• It is not clear how the proposed dataset relates to existing work, and which specific problems it aims at solving
• There is no measurement of training data leakage. Given their popularity, it seems likely that these TV shows are quite prevalent in LLM training data

Regarding the fine-tuning results:

• There is no comparison with a baseline that does not use data augmentation
• There is no comparison to any few-shot prompting scenario
• It is not clear what the fine-tuning for a specific character consists of: how is the autoregressive loss computed when multiple characters are talking?

Regarding the paper presentation:

• The paper spends several pages presenting topics that are not directly relevant for the subject at hand. While understanding low-rank fine-tuning is a topic of importance, it is not particularly relevant to spend 1 page on it, at the expense of experimental details and results.
• The abstract mentions 3 different datasets, but the results cover only two of them.

1. Incremental Contributions: While the topic of data augmentation in NLP is valuable, the paper’s contributions appear largely incremental. Many of the techniques explored (e.g., synonym replacement, backtranslation) are well-known and widely used. The paper could benefit from exploring more novel augmentation techniques tailored to character-based dialogue data's nuances. Additionally, about half of the paper is describing past techniques like EDA, Paraphrasing, LoRA, etc (specifically, Section 2, and 3), and there is nearly not enough content for this paper to be considered an impactful contribution (figure 3 has too little information to be taking a whole page and Table 1 font seems overtly big as if just to fill some space).
2. Limited Dataset Scope: Only two datasets are used, both of which are highly specific in context (e.g., Paimon from Genshin Impact). This narrow focus raises questions about the generalizability of the findings to other personalized dialogue systems or broader NLP applications. Including a wider variety of datasets or character types would add robustness to the conclusions.
3. Paraphrasing Overfitting Concerns: The paper notes overfitting issues with the paraphrasing technique but does not fully explore ways to mitigate this. Although the authors acknowledge the limitations of paraphrasing with large models in domain-specific contexts, a more thorough investigation into alternative paraphrasing techniques or domain adaptation methods would strengthen the study.
4. Issues With the References: a) There are some invalid entries in the references, such as, "Unknown Author" and b) The reference list is too short. I suggest the authors to conduct a more thorough review of the current literature.

Suggestions

1. Broaden Dataset Variety: I think the authors are already aware of this problem, but including datasets that represent a wider range of conversational styles or character personalities would improve the generalizability of the results. Alternatively, conducting a detailed analysis of why certain augmentation techniques are particularly suited or unsuited to the specific characteristics of these datasets would be more interesting.
2. Investigate Advanced Augmentation Techniques: Consider exploring novel or advanced data augmentation techniques that could better align with the semantic and contextual requirements of character dialogue, such as sentiment-aware or context-sensitive paraphrasing. Additionally, using domain-adaptive techniques for paraphrasing may mitigate overfitting while still generating diverse examples. There is also a huge body of work on using off-the-shelf LLMs for data augmentation that the authors should explore.
3. Refine Analysis of Computational Constraints: Given the choice to work with the smaller LLaMA3-8B model, it would be beneficial to address the specific trade-offs involved more explicitly. This discussion could include any adjustments made to balance memory efficiency with model performance and how this may limit model generalization capabilities.
4. Improve Paper Organization: We don't need to describe past methods like EDA, LoRA, etc. in so much detail. Instead, I would suggest focusing on including more strong baselines (like LLM-based data augmentation).