Energy-Oriented Alignment for Large Language Models

1. While the title mentions "alignment," much of the paper focuses on the dataset's effectiveness and relevance, leading to a disconnect in the writing.
2. Some of the open-source models such as the Llama2 series utilized in the paper are outdated, which may affect the representativeness of the results.
3. The paper does not provide sufficient insights into why the dataset is effective, nor does it delve deeply into why these specific points were chosen for dataset construction.
4. The presentation of results is simplistic, lacking more complex analyses or visualizations to support the conclusions.
5. The paper does not clearly describe the specific training settings used during the alignment process, making it difficult for readers to replicate or fully understand this aspect.

The reviewers' concerns are primarily on the feasibility and legitimacy/utility of the proposed energy-oriented malicious input: whether these malicious inputs are truly meaningful and necessary in invading and exhausting the computational resources of service providers.

1. Service providers, such as OpenAI, assign token limits for each user, for instance, initial accounts often have restricted token consumption per minute. This measure should be sufficient to counter such attacks.

2. While it could be argued that attackers might create numerous accounts to carry out energy-draining activities, this scenario falls under the domain of DoS (Denial of Service) defense and is not directly related to LLMs.

3. Most importantly, if the goal is simply to occupy a service provider's energy, attackers could achieve this by just sending normal requests. As long as the provider is required to generate tokens, their resources will be consumed. Malicious inputs like repetitive or random content are unnecessary for this purpose.

Thus, In the reviewer’s view, the proposed concern is not practically feasible in real-world scenarios.

Another concern arises from the distinction between energy-oriented malicious input and LLM instruction-following capabilities. Table 1 outlines various scenarios categorized as malicious input, but many of these are fundamental to LLM instruction-following applications, such as nested IF/ELSE statements, which have proven useful in contexts like verbalized ML [1]. The designed cases show significant overlap with legitimate instruction-following functions. Therefore, the reviewer is concerned about the potential for excessive refusal of valid input.

Building on the previous concern, the authors did not demonstrate the utility of LLMs after applying EnergyAlign. Specifically, there is no information on whether EnergyAlign causes LLMs to become overly cautious, leading to refusal of legitimate tasks, or how fine-tuning impacts the performance of benign tasks. Additionally, regarding defense mechanisms, did the authors experiment with simple system prompts to mitigate these "malicious" inputs? How effective were these prompts?

Reference:

[1] Xiao, Tim Z., et al. "Verbalized Machine Learning: Revisiting Machine Learning with Language Models." arXiv preprint arXiv:2406.04344 (2024).

w1: The paper seems to lack a strong technical contribution. It primarily relies on manually designed prompts to induce LLMs to produce longer outputs, which are then combined with existing jailbreak methods. However, there may be more effective ways to achieve the stated motivation. For instance, adversarial optimization could be used to make the LLM recursively generate the same content, or minimizing stop tokens (e.g., <\s> in Llama2) could prevent the LLM from stopping.

w2: In the discussion of combining EnergyAlign with jailbreak methods, the authors mainly justify the use of LongGPT and Sample-aware prefix. Could you provide more reasoning regarding the use of these jailbreak techniques?

w3: Potential adaptive defense methods are not considered. For example, powerful models could have a system prompt stating "Determine if the user's input is intended to generate large amounts of meaningless content; if so, refuse to respond," or use robust training techniques.

w4: It seems challenging to distinguish between the proposed energy-oriented threats and legitimate requests in some situations. For instance, developers may need LLMs to generate test samples following specific patterns, which is similar to the "Random" type in EnergyAlign. Could the use of EnergyAlign for robust training potentially harm the model's utility?

w5: Compared to inference time, a more common scenario involves using LLM service provider APIs, which typically calculate costs based on token usage. The price difference between input and output tokens may not be significant. In this case, would using large amounts of random text as input be more effective? Additionally, this approach could be harder to defend against.

• The paper heavily relies on output length as the primary metric for attack success (ASR), which may oversimplify the nature of energy-based vulnerabilities. Factors such as model-specific efficiency in token generation, token density, and variability in response quality are not deeply explored. Alternative or compound metrics that capture computational efficiency, memory load, or latency impacts could offer a more nuanced understanding of the resource burden and add value to the dataset.
• The paper evaluates primarily static defences, such as input filtering and pre-trained refusal criteria, without addressing adaptive, real-time detection techniques that could adjust responses based on dynamic monitoring of output behaviour. For example, adaptive rate-limiting or real-time token stream analysis could help identify and counteract excessively lengthy responses, offering a more resilient model defence approach that extends beyond pre-aligned responses.
• Although EnergyAlign is carefully crafted, it remains synthetic and may not fully capture the complexity of real-world energy-oriented attacks. Synthetic data may lack the subtle contextual variability seen in genuine user interactions, potentially limiting the dataset's ability to generalize effectively in production settings where adversaries employ context-aware or multi-layered input manipulations. Real-world data validation or augmentation could enhance the dataset’s realism and impact.
• Although various jailbreak techniques are tested, the paper’s approach may not exhaustively address all potential vectors for bypassing model restrictions. Current jailbreak techniques, such as ASCII and prefix injection, may be less representative of evolving adversarial approaches that use multi-stage prompts, obfuscation, or subtle contextual shifts. Incorporating more complex, adversarially crafted input sequences would improve robustness evaluations and provide a clearer picture of LLM susceptibility to jailbreaks.