Adversarial Instance Attacks for Interactions between Human and Object

Lack of Clarity: The paper is challenging to follow due to its complex technical language and insufficient contextual explanations. It assumes a high level of familiarity with the subject matter, making it less accessible to a broad audience.

Ambiguity in Terminology: Some terms and acronyms are not adequately defined or clarified, such as "Adversarial Instance Attacks," "Adversarial Interaction Attack," and "HOTR."

Missing Visual Aids: Given the complexity of the proposed framework, the paper would benefit from visual aids, diagrams, or flowcharts to help readers understand the different modules and their interactions.

Evaluation Metrics: The paper could benefit from a more detailed discussion of the evaluation metrics used and how they relate to the effectiveness of the attacks.

Deficient Writing Quality: The paper exhibits evident signs of hasty preparation, accompanied by numerous errors. To illustrate a few examples, some of which are highlighted in bold text:

• The reviewer could not understand “on the interaction between scene contents and realize’ based on”

• The math notations of {$h_i^b, o_i^b, o_i^l, a_i$} are incorrect and confusing. It does not make sense to use a set { } for both coordinates and categories. And defining $a_i$ as the categories of the object is wrong.

• directly adding noise to the union area not only drop the precise of interaction perception but also bring ....

The motivation is unclear. The arguments within the sentences lack proper support. For instance, the sentence "a direct strategy to disturb the interaction is adding noises on the union area of human and object, following the feature extraction procedure in HOI. However, directly adding noise to the union area not only drop the precise of interaction perception but also bring interference in locating and classifying human and objects" lacks clarity and substantiation.

Understanding the paper's concept is challenging, primarily due to its poor writing quality. For example, the exact process for generating "N^q candidate interaction areas" is not explained and appears disconnected from the subsequent content. Upon reviewing Section 3.3, the reviewer found it to be incomplete, offering only a feature extraction process without any content related to sampling. The method for obtaining the extracted candidate predicate areas $A$ is also unclear.

The rationale behind distance calculation remains unclear, as does the definition of the midpoint function $f_{mid}()$.

"To address this issue, we employ random assignment to connect perplexing object categories to candidate predicate areas in order to generate the attack instances." However, the explanation for this random assignment is unclear.

In summary, while the paper may address an important problem of black-box adversarial attacks in scene interactions, its writing quality falls significantly below the standards of ICLR. Significant revisions are required. Therefore, the reviewer recommends clear rejection."

The equation (6) seems to contradict to the original claim of the paper "our goal is to bring confusion on the interaction of a pair of human and an object, without contaminating their detection results including the human bounding box, object bounding box, and object category", as the object bounding boxes seem to be changed. Please provide more clarifications to it.

1. For the experiments, the comparisons between I-FGSM and PGD on detectors and the proposed method are unfair. The authors use the transfer results by I-FGSM and PGD to attack HOI, where the adversarial examples based on the architecture of detectors (e.g., Faster RCNN and YOLO ) are generated. Thus, these methods have a weak transferability across the different tasks. However, the proposed method has more knowledge about the attacked task instead of only detectors.
2. The tasks of HOI highly rely on the results of detections. The existing attacks for detection can be directly employed in this task, leading to awful results. From the proposed method, the attacks aim to fool the interactions between the human and object, instead of the human and the object itself. If so, wrong detection results and boxes easily lead to wrong interactions. Thus, those attacks against object detection, especially for fooling the detection boxes should be involved in comparisons. Besides, why only attacking the interaction in HOI is worthy of investigating?
3. Expect for the illustration in Figure 1, more visualizations with various methods (e.g., PGD and I-FGSM) should be involved to demonstrate its imperceptibility and effectiveness.
4. There are some minor problems.

• In the caption of Figure 1, ‘remaining0’ seems a typo.
• Some formats of references are wrong and inconsistent.

• My main concern is that the proposed attack is somewhat an untargeted attack for an HOI recognition system, where we cannot choose the target HOI category, instead, the target HOI category is determined by the method via finding the most confusing interaction. I am not sure the whether such attack will happen in real applications or not.
• The proposed method is straightforward, which is combination of existing techniques such as Targeted adversarial Objectness Gradient attacks (TOG) in Adversarial Instance Generation module.