Less is More: Exploiting Feature Density for Enhanced Membership Inference Attacks

Q3: This paper claims that the proposed method is resource-efficient but they are not very obvious in the paper presentation. Using around 10k for training and testing and rest for MIA is a common thing in the literature. I do not see why this experiment setting is resource-efficient. For the number of shadow models, it may be required a lot for LiRA attack, but it is not that big for other attacks. Especially, for population attacks in [1], you do not need to train the shadow model. BTW, figure 4 is too small to distinguish the difference.

Thank you for highlighting this point. To clarify this, we provide explicit comparisons to baseline methods in terms of resource efficiency. The table below compares the computational resource requirements (number of trained models and training time) for our method and state-of-the-art methods, which demonstrates that our method could achieve better computational efficiency compared to previous works. Although population attacks do not require shadow model training, they necessitate a large number of queries to the target model to determine the threshold, which also incurs high resource costs. Moreover, our attack substantially outperforms population attacks. We implemented a population attack under the same settings on ResNet-18 trained with CIFAR-100, achieving a TPR of only 0.4% at 0.1% FPR with an AUC of 0.771, compared to our method’s TPR of 4.6% and AUC of 0.946.

Furthermore, beyond computational resources, our method is also resource-efficient in terms of auxiliary data requirements. Many existing methods (e.g., Liu et al., 2022; Shi et al., 2024) require large auxiliary datasets with the same distribution as the target model's training data, in addition to shadow datasets. This requirement is often impractical in real-world scenarios. In contrast, our method does not rely on such large auxiliary datasets. We have revised Appendix E to include a comparison of attack costs.

Additionally, we acknowledge your feedback regarding Figure 4. We have enlarged the figure and included the following table to better clarify our results. We have revised Section 4.2 and Figure 7 to make it clearer.

Q4: I also wonder if the defender uses a pre-trained feature extractor during target model training, will the proposed attack still work? Or in other words, is your proposed method limited to a ResNet-like model and training from scratch setting?

Our method is not limited to ResNet-like models. In fact, we have demonstrated the effectiveness of our attacks even on diffusion models, which are generative and structurally distinct from ResNet-based classifiers. To address the reviewer’s question about the use of a pre-trained feature extractor, we conducted an additional experiment using a pre-trained DenseNet-161 model on ImageNet. We applied transfer learning to adapt this model on CIFAR-100 and then evaluated our random-based attack against the baseline methods in standard adversarial settings. The results, presented in the following tables, clearly show that our attack remains effective even when the defender employs a pre-trained feature extractor during target model training. We have revised Appendix G to include these results.

Thank you for your constructive feedback. We provide detailed responses to your comments.

Q1: I think the proposed method should be compared to well-known attacks such as [1] and current SOTA such as [2] to make the claim this work is SOTA. What will be the results when compared to [1] and [2] proposed methods? Especially, [2] also claims that their proposed method can be low cost making it a good baseline to this work.

Thank you for your valuable feedback. We have conducted additional experiments to compare our method with the newer, stronger attacks mentioned in [1] and [2]. Specifically, we reproduced these attacks using their official implementations on WideResNet trained on CIFAR-10 and compared them with our attack in the same settings. For the attack in [1], we selected Attack R, which achieves the highest TPR at low FPR. We trained 16 shadow models, and the experimental results with different their attack strategies are as follows:

For RMIA [2], which focuses on improving membership inference performance in low-cost settings, we followed their experimental setup by training 1, 2, and 4 shadow models. The results are as follows:

As the tables indicate, our method outperforms both attacks from [1] and [2]. For the attack in [1], we improved the TPR at 0.1% FPR from 1.6% to 2.6%. Compared to SOTA attack RMIA [2], our method achieves higher TPR and AUC, especially when the number of shadow models is low. We have revised Section 4.2 to include these results.

Q2: I am curious that why the balanced accuracy is approximately 0.7, while the AUC reaches around 0.9. Although theoretically possible, I haven't encountered such a significant discrepancy in practical applications, where attack accuracy generally aligns closely with attack AUC. Could the author clarify this anomaly or consider releasing the source code to provide further insights?

Thanks for pointing this out. To maintain consistency with previous evaluations (Shokri et al., Salem et al., Liu et al.), we calculated the balanced accuracy using a fixed threshold of 0.5 on the softmax output of the attack model to determine membership status. In contrast, the ROC curve and the corresponding AUC are obtained by varying this threshold across all possible values between 0 and 1, which is aligned with [1] [2]. This process allows us to compute the TPR and FPR at different thresholds, capturing the model's performance across all operating points. We have revised Section 4.1 to make it clearer.

Our attack model achieves a high TPR at low FPR values, resulting in a steep initial slope in the ROC curve. This steep rise means the ROC curve quickly diverges from the diagonal line representing random guessing and moves toward the top-right corner. This characteristic significantly contributes to a higher AUC. When calculating balanced accuracy at the fixed threshold of 0.5, we may not be operating at the model's optimal point on the ROC curve. If the optimal threshold that maximizes balanced accuracy differs from 0.5, using this fixed threshold can lead to a lower observed balanced accuracy. This discrepancy is also observed in existing studies, such as Carlini et al. (2022), where an AUC of 0.925 is reported, yet the accuracy is only 0.826. We will publicly release our code to facilitate reproducibility.

Dear Reviewer dYkK,

Thank you once again for your thoughtful feedback. We have provided a detailed response to your questions and hope it sufficiently addresses your concerns. With the interactive discussion period closing soon, please don't hesitate to reach out if you need any additional information or clarification.

Q5: Lower AUC Despite High TPR at 0.1% FPR

We would like to clarify that Yeom et al., Shokri et al., Salem et al., Song & Mittal, Ours. Random. and Ours. Guided., were evaluated under the same standard adversarial settings. In contrast, Shi et al., Liu et al., and our method using loss trajectory (Ours w/ loss traj.) were tested in adversarial settings with access to a larger auxiliary dataset. Our method can outperform baselines in both settings in AUC, while Liu et al. and Shi et al. achieved a slightly higher AUC than Ours. Random and Ours. Guided, this is due to their use of an auxiliary dataset, which enhances their attack capability.

We have included the mean and standard deviation values in the following table for ResNet-18 trained on CIFAR-100. These results demonstrate that our method consistently outperforms the baselines, and the differences are statistically significant.

Q6: Clarity in Figures

Sorry for the confusion. We have provided the TPR at 0.1% FPR and the AUC in the table below, showing that our method consistently outperforms LiRA. We have revised Section 4.2 to make it clearer.

Q7: Absence of Requirement for Shadow Models

Thank you for your insightful and constructive feedback. We agree that a significant strength of our attack is its ability to be executed on pre-trained models without requiring the training of shadow models. To validate this point, we conducted preliminary experiments following the same settings as the population attacks in [3], without training shadow models. We implemented both attacks on ResNet-18 trained with CIFAR-100. The population attack achieved a TPR of only 0.4% at a 0.1% FPR with an AUC of 0.771. In contrast, our method achieved a TPR of 1.9% and an AUC of 0.916 under the same settings, significantly improving upon the population attack. This demonstrates the advantage of our approach, which does not require shadow models and is particularly beneficial for large-scale models, thereby significantly reducing computational costs. We have revised Section 1 and put a detailed discussion in Appendix F to emphasize this point.

Reference:

[r1] Sundararajan, Mukund, Ankur Taly, and Qiqi Yan. "Axiomatic attribution for deep networks." International conference on machine learning. PMLR, 2017.

[r2] Selvaraju, Ramprasaath R., et al. "Grad-cam: Visual explanations from deep networks via gradient-based localization." Proceedings of the IEEE international conference on computer vision. 2017.

[r3] Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.

[r4] Lundberg, Scott M., and Su-In Lee. "A Unified Approach to Interpreting Model Predictions." Advances in Neural Information Processing Systems 30 (2017).

Q3: Potential Overuse of CW for Identifying Vulnerable Features

Thank you for pointing this out. We have compared with different super-pixel methods, and the results are discussed in Appendix C - Different Removal Strategy and Table 8. Our method significantly outperforms super-pixel methods by a large margin. Regarding saliency map methods, since we assume a black-box threat model for the target model, we cannot utilize methods that require white-box knowledge, such as Integrated Gradients [r1] and GradCAM [r2]. Therefore, we selected two widely used saliency methods that work in black-box settings: LIME [r3] and SHAP [r4]. The results are as follows:

While these methods only perform slightly lower than our approach, we want to highlight that these black-box saliency methods are generally very computationally expensive. They require a large number of queries to approximate feature importances—typically several hundred to a thousand queries per sample to derive a saliency map. In contrast, our method is very efficient: training a mask prediction model takes less than 2 minutes on an Nvidia 4090 GPU, and it does not require access to the target model to derive the mask.

Q4: Lack of Comparison with Stronger, Recent Attacks

Thank you for your valuable feedback. We have conducted additional experiments to compare our method with the newer, stronger attacks mentioned in [2] and [3]. Specifically, we reproduced these attacks using their official implementations on WideResNet trained on CIFAR-10 and compared them with our attack in the same settings. For the attack in [3], we selected Attack R, which achieves the highest TPR at low FPR. We trained 16 shadow models, and the experimental results with different their attack strategies are as follows:

For RMIA [2], which focuses on improving membership inference performance in low-cost settings, we followed their experimental setup by training 1, 2, and 4 shadow models. The results are as follows:

As the tables indicate, our method outperforms both attacks from [2] and [3]. For the attack in [3], we improved the TPR at 0.1% FPR from 1.6% to 2.6%. Compared to SOTA attack RMIA [2], our method achieves higher TPR and AUC, especially when the number of shadow models is low. We have revised Section 4.2 to include these results.

Thank you for your detailed feedback, we greatly appreciate your positive remarks and constructive suggestions. Below, we address each of your concerns point by point.

Q1: Reframing of Existing Attack by Adding Noise to Input

Thank you for your insightful comments. While both our work and [1] utilize feature perturbation in the context of membership inference attacks, there are several key differences that set our approach apart:

Targeted Feature Perturbation vs. Indiscriminate Noise Addition: Our approach emphasizes the importance of individual features and their specific contributions to the model's decision-making process. Not all regions of an image equally influence the model's output, as extensively studied in explainable machine learning research [r1, r2]. For example, saliency regions containing distinctive object parts or textures may be more critical for the model’s predictions. In contrast, [1] adds random noise indiscriminately across the entire input, which may not fully exploit the feature discrepancies between member and non-member samples. We, on the other hand, progressively perturb individual features, and we train a model to predict which features should be perturbed first. This results in a more fine-grained and targeted strategy that better leverages the differences in feature sensitivities.

Attributing Prediction Change to Feature Perturbation: The success of a perturbation-based attack heavily relies on ensuring that the perturbation operations accurately reflect the importance of the features being perturbed, so that the changes in the model's predictions can be directly attributable to the perturbation of those features. The approach in [1], which involves adding random noise, can introduce distribution shifts (Hooker et al., 2019) and adversarial artifacts (Fong & Vedaldi, 2017). These unintended changes can obscure whether the output variations are due to feature perturbation or other factors. In our work, we designed specific perturbation operations to minimize such impacts, ensuring that the perturbations more accurately represent feature importance.

To empirically demonstrate the effectiveness of our approach compared to [1], we implemented the Merlin attack from [1] under the same experimental settings as our method. Our results show that our method achieves a TPR of 4.6% at a 0.1% FPR on ResNet-18 trained on CIFAR-100, whereas the Merlin attack achieves a TPR of only 1.3% under the same conditions. We have revised Section 2 to clarify the distinctions between our method and that of [1].

Q2: Insufficient Information for Reproducing Results

Due to space limitations, we have provided detailed experimental settings in Appendix A. We apologize for any confusion and will make these details clearer in the manuscript. We will also publicly release our code to facilitate the reproduction of our results. Here, we address your questions point by point:

• The shadow model is trained on a randomly selected subset from the same distribution as the target model but disjoint from its training data. It is trained for 100 epochs with an initial learning rate of 0.1. We utilize a cosine annealing schedule to gradually reduce the learning rate, employ standard data augmentations, and apply a weight decay rate of 0.0001.

• In the standard adversarial setting, we use one shadow model and one target model. For settings with large supplementary datasets, we follow the approach of Liu et al. (2022), applying knowledge distillation for both shadow and target models and storing intermediate model checkpoints. For settings with a large number of shadow models, we follow the strategy of Carlini et al. (2022) and employ 8, 16, and 64 shadow models alongside a single target model.

• We have repeated the experiments 10 times with different random samplings of the datasets, except for the experiments based on Carlini et al. (2022) due to the large computational costs associated with training shadow models.

• The training and testing accuracies of all models are provided in Table 7.

• We have adopted the similar strategy discussed in [1-3], we use the softmax output of the attack model as membership scores, which indicates the attack model’s prediction confidence. We vary the threshold between 0 and 1 to calculate the TPR at each swept FPR, simulating the adversary’s power and error based on these membership scores.

We have revised Section 4.1 and Appendix A.2 to clarify the unclear details.

Thank you for your constructive feedback. We provide detailed responses to your comments.

Q1: It is not clear what the connection is between the intuition and the exact method. In section 3.2, the design intuition is described by the feature embedding and the density. However, it is not mentioned anymore in the later method design.

Thank you for the feedback. Our method is directly informed by the design intuition described in Section 3.2, where we observed that member samples tend to reside in high-density regions of the learned feature space. This suggests that the model is likely to exhibit higher confidence in these samples, even when some features are removed. Building on this insight, we propose a novel membership inference attack centered around the feature removal scheme described in Section 3.4. The feature removal operations, strategies, and model behavior selection are all closely aligned with this intuition. For instance, the design of the feature removal operation aims to accurately capture the importance of the removed features, ensuring that the changes in the model's predictions can be directly attributed to the removal of those features. We have revised Section 3.3 and Section 3.4 to clarify this connection.

Q2: The method is not described very clear.

• Clarification of the mask: The mask has the same shape as the input image, with each element taking a continuous value between 0 and 1. This design choice ensures that the optimization regarding the mask is end-to-end differentiable throughout the optimization process.

• Clarification of the removal process: We apologize for the confusion. We have described the removal process in the implementation details. After training the mask prediction model, we input the original sample $x$ and obtain the predicted mask. We then rank the predicted mask values from lowest to highest, then remove features of the sample $x$ whose corresponding mask value percentiles fall below the specified removal ratio. The resulting sample $x'$ is then fed into the target model. We compute the model’s output behavior (e.g., confidence score for classification models or posterior estimation error for diffusion models). The sequence of output behaviors across different removal ratios forms the "removal-based membership features."

• Clarification of the removal-based membership features: As explained earlier, the "removal-based membership features" are a sequence of model output behaviors (e.g., confidence scores) across different removal ratios. Therefore, they are no longer in the pixel space but instead represent model responses. This allows us to effectively train an MLP to distinguish between the output behaviors of members and non-members.

We have revised Sections 3.3 and 3.4 to make the description clearer.

Q3: The variants of methods in the experiment, Ours. random, Ours Guided, Ours w/ loss traj. are not well-defined. For example, the loss trajectory is firstly mentioned in the background section, and then appears in the experiment result description paragraph.

Thank you for the feedback. We clarify the definitions as follows: Ours. random refers to our random-based removal strategy, where a specified ratio of pixels from the input is randomly removed. Ours. guided denotes our guided-based removal strategy, which ranks the predicted mask values from the mask prediction model in ascending order and removes the features of $x$ whose corresponding mask value percentiles fall below the specified removal ratio. Ours w/ loss traj. indicates that we implemented our attack in the same adversarial setting as Liu et al. (2022) and then concatenated our random removal-based attack features with the loss trajectories obtained using the method from Liu et al. (2022). We include the following table to better illustrate these variants:

Dear Reviewer 9GYb,

Thank you once again for your thoughtful feedback. We have provided a detailed response to your questions and hope it sufficiently addresses your concerns. With the interactive discussion period closing soon, please don't hesitate to reach out if you need any additional information or clarification.

Thank you for your insightful feedback. We provide detailed responses to your additional questions.

Answer to Q1:

This is a good question. The preference for learning the U-Net model to predict the mask stems from several considerations. First, under the black-box threat model assumed in this work, the adversary does not have access to gradient information from the target model. As a result, directly optimizing the mask $m$ using the proposed loss function is not feasible because gradient-based updates cannot be performed. Additionally, while it is possible to use a shadow model to estimate the mask, optimizing $m$ for each sample individually would be computationally expensive. This process involves multiple components of the proposed loss function and requires numerous backpropagation steps for every sample. By contrast, training a U-Net model allows for efficient mask generation. After training, the U-Net produces the mask in a single forward pass, significantly improving the computational efficiency of the membership inference attack. Also, the U-Net model is particularly well-suited for this task due to its established effectiveness in image generation, e.g., in DDPM (Ho et al., 2020). U-Net utilizes feature maps at multiple resolutions to generate sharp and precise outputs.

As for the second part of the question, our approach follows the threat models commonly adopted in prior works (Song & Mittal, 2021; Carlini et al., 2022; Liu et al., 2024). These threat models assume that the adversary is aware of the target model’s architecture and has access to an auxiliary dataset that shares the same distribution as the target model’s training data. Under this assumption, the shadow model, trained on the auxiliary dataset, is designed to closely replicate the behavior of the target model. The mask prediction model, trained using the outputs of the shadow model for both member and non-member samples, leverages this similarity. As the shadow model and target model exhibit comparable decision-making patterns, the mask prediction model is expected to generalize effectively to predict masks for samples from the target model.

We have revised Appendix B to make the rationale clearer.

Answer to Q2:

Thank you for highlighting this concern. The distinction between feature removal and pixel removal arises due to the constraints imposed by the threat model. In our approach, pixel removal serves as a proxy to achieve the goal of feature removal.

A neural network embeds an input sample into an intermediate feature representation, which is then used for subsequent classification. However, since we operate under a black-box threat model, the adversary cannot directly manipulate the intermediate feature representation. Instead, the adversary, having control over the input, can leverage pixel removal to approximate the feature removal process. For example, consider a neural network designed for image classification. The network embeds input samples into high-level feature representations, such as edges, textures, and shapes. When pixels are removed from the input (e.g., obscuring part of an object in an image), the corresponding high-level features, such as the edge structure, are also affected within the feature space. By simulating feature removal in this way, the adversary effectively influences the learned feature space, enabling the attack within the constraints of the threat model.

We have revised Section 3.2 to make it clearer.

Answer to Q3:

The dimensionality of the final membership features depends on the dataset and the specific adversarial knowledge included. The confidence score of the pixel-removed image has a fixed dimensionality of 50, corresponding to 50 steps of pixel removal ratios (with fewer steps also explored in the ablation study). The one-hot label contributes 100 dimensions for CIFAR-100 and 10 dimensions for CIFAR-10 and CINIC-10. Additionally, when incorporating additional adversarial knowledge, such as methods from Liu et al. (2022a), loss trajectories contribute another 100 dimensions. The combination of these components and losses determines the size of the final membership features, which can become quite large depending on the configuration.

Please let us know if you have any further questions.

Dear authors,

Thanks for your response and revised text for the method description. Your response addressed some of my clarity issues. With my (relatively) deeper understanding, I have further questions/concerns:

1. Why is it preferred to learn the U-net for predicting the mask? An alternative way is to optimize $m$ for every $x$ w.r.t. the proposed loss function. Moreover, the mask $m$ is learned w.r.t the shadow model from my understanding, what's the intuition that it will still have the expected performance when it is applying with the samples in MIA test?
2. I am still not very convinced that the intuition is aligned with the method design. The intuition is discussing the learned feature space, where "learned" is in the sense of the feature representation (embedding space) learned with the training data. However the "feature removal" in the method section is more about the "pixel removal" and it seems there is nothing related to the learned feature space. Also this introduces some confusions that "feature" in the intuition section means the embedding in the feature space, but later on "feature removal" seems to be the "pixel removal".
3. Clarity question: "final membership features" mentioned at line 220 of the revised pdf only has very few dimensions, which one dimensional confidence score of pixel removed image, the loss of original image, and the one-hot label + (potentially other signals from the literature)?

W3: Unclear Definition.

We appreciate your feedback. The random-based removal strategy refers to randomly removing a specified ratio of pixels from the input. The guided-based removal strategy ranks the predicted mask values from the mask prediction model in ascending order and removes features of input whose mask values fall below a given percentile threshold, based on the specified removal ratio. We have revised Section 3.3 to clarify these definitions.

W4: Confusing Terms.

Thank you for pointing this out. As you noted, our approach involves proposing a new membership inference signal that more effectively captures the existing gap between members and non-members, rather than increasing that gap. We’ve adjusted our claim in Section 3.2 accordingly to reflect this more accurately.

W5: Presentation Issues.

Thank you for pointing this out. (1) We have enlarged Figure 2 to ensure that each step is distinguishable. (2) The term "the system" refers to the linear equation system. Specifically, the noisy linear imputation method approximates the values of removed pixels using a weighted mean of their neighboring pixels. When multiple pixels are removed, this process forms a system of equations where known pixel values are used directly, while removed pixels are treated as unknown variables, resulting in a linear equation system. We have revised Section 3.4 to make it clearer.

Thank you for your constructive feedback. We provide detailed responses to your comments.

W1: Unclear Intuition.

Regarding Hypothesis 1 (H1):

Hypothesis 1 (member samples exhibit higher density values in the feature space) is inspired by feature learning theory (Cao et al., 2022; Kou et al., 2023). This theory suggests that a model's generalization capability is directly related to the training sample size and the signal-to-noise ratio of the features in the data. Extending this to membership inference, we introduce the concept of feature density, defined as the proximity of a sample's feature embedding to its nearest neighbors in the feature space. We posit that member samples have higher density values in the feature space. To empirically validate this, we conducted experiments by extracting intermediate features from the layer immediately before the final classification layer for both member and non-member samples, then calculated the average L2 distance of each sample to its top 5 nearest neighbors in the feature space. The experiments were repeated across different models and datasets, and the mean and standard deviation of L2 distance is as follows:

• ResNet-18 trained on CIFAR-100: Members: 71.11±1.08, non-members: 78.86±0.97

• ResNet-18 trained on CIFAR-10: Members: 13.42±0.22, non-members: 15.52±0.34

• WideResNet trained on CIFAR-100: Members: 54.34±0.78, non-members: 60.65±0.66

• WideResNet trained on CIFAR-10: Members: 2.99±0.04, non-members: 3.22±0.03

These results consistently indicate that member samples have lower mean L2 distances to their nearest neighbors compared to non-member samples, reflecting a higher feature density.

Regarding Hypothesis 2 (H2):

Confidence resilience naturally arises from increased feature density, as redundant representations in high-density regions enable member samples to maintain higher confidence during the early stages of feature removal. Figure 1 shows a gradual decline in confidence scores for members compared to non-members in the early stage. We acknowledge that architectural differences (e.g., ResNet vs. WideResNet) influence the rate of these changes, however, despite variations in the rate of change, the overall trend remains consistent across architectures: a gradual decline in confidence scores for members, followed by a sharp drop in later stages. The sharp drop observed in the latter stage arises from significant feature removal, which shifts member embeddings from high-density regions to sparser areas in the feature space. This increased distance from the original embeddings causes a substantial drop in confidence scores. In contrast, non-member samples, which typically begin in lower-density regions with more uniformly distributed distances, experience a relatively steady decline in confidence throughout the feature removal process.

We have clarified this design intuition in the revised manuscript, as reflected in Section 3.2 and Appendix D.

W2: Incomplete Experiments.

Thank you for highlighting this point. Resource efficiency is a key advantage of our approach, and we provide explicit comparisons to baseline methods to clarify this.

Comparison of Computational Costs: The table below compares the computational resource requirements (number of trained models and training time) for our method and state-of-the-art methods, which demonstrates that our method could achieve better computational efficiency compared to previous works.

Resource Costs for Mask Prediction Model: Regarding the computational costs of the mask prediction model, it introduces minimal additional overhead. Training the mask prediction model takes less than 2 minutes on an Nvidia 4090 GPU, as it only requires 5 epochs to achieve high prediction performance.

Furthermore, beyond computational resources, our method is also resource-efficient in terms of auxiliary data requirements. Many existing methods (e.g., Liu et al., 2022; Shi et al., 2024) require large auxiliary datasets with the same distribution as the target model's training data, in addition to shadow datasets. This requirement is often impractical in real-world scenarios. In contrast, our method does not rely on such large auxiliary datasets. We have revised Appendix E to include a comparison of attack costs.

Dear Reviewer iNv6,

Thank you once again for your thoughtful feedback. We have provided a detailed response to your questions and hope it sufficiently addresses your concerns. With the interactive discussion period closing soon, please don't hesitate to reach out if you need any additional information or clarification.