Strategic Filtering for Content Moderation: Free Speech or Free of Distortion?

[Definition of social distortion] :

First, we would like to highlight that the example provided by the reviewer stems from a misunderstanding of the social distortion mitigation metric defined in Eq. (5). We clarify this misunderstanding below. This confusion is likely due to a typo in our paper, for which we sincerely apologize. What we intended to define in Eq. (5) is as follows:

$h(f;\mathbf{e},\mathbf{x},c)=D(\perp;(\mathbf{x},c),\mathbf{e})\cdot \mathbb{I}[f(x)\leq 0]-D(f;(\mathbf{x},c),\mathbf{e}),$

and not

$h(f;\mathbf{e},\mathbf{x},c)=D(\perp;(\mathbf{x},c),\mathbf{e})-D(f;(\mathbf{x},c),\mathbf{e}).$

In other words, the metric considers the social distortion mitigation for any non-benign content as zero. This choice is consistent with our justification in Lines 234–236: for content that will either be filtered out or "distorted" towards a desirable direction, there is no need to measure how much distortion could be saved by the moderator. We hope this clarification resolves the reviewer’s concerns about our model.

Additionally, we would like to emphasize that all results presented in our current draft are based on the corrected definition $h(f; \mathbf{e},\mathbf{x},c)=D(\perp;(\mathbf{x},c),\mathbf{e})\cdot \mathbb{I}[f(x)\leq 0]-D(f;(\mathbf{x},c),\mathbf{e})$. Thus, this change is merely a correction of the typo and does not introduce any significant modifications that could affect the validity or conclusions of our results.

About our convex benign region assumption :

We would like to clarify that the convexity of the benign region should not be understood as an inherent property of the semantic embedding space. Instead, it represents a regularity constraint on the class of moderator functions. In industry-level applications, due to practical considerations such as interpretability and the transparency of policies to platform users, platforms are often limited to employing simpler moderator functions, which typically induce convex decision boundaries—for example, linear functions. We acknowledge that our current justification may be overly strong, and we will revise it to adopt this more practical perspective in revision.

[Q1: Why SD=0 when $f(x)<0$] :

We would like to clarify a potential misunderstanding in the reviewer’s interpretation of our notation. In our setting, $f(x)>0$ indicates that the content is marked as problematic by the moderator $f$. Thus, our argument is that the SD mitigation equals zero when the true content is problematic, and not benign. This is, in fact, the opposite of the reviewer’s understanding. The justification for this definition is provided in Lines 234–236, where we explain that for problematic content that is either filtered out or "distorted" towards a desirable direction, there is no need to measure how much distortion is induced or can be mitigated by the moderator. We hope this clarification resolves the reviewer’s concern.

[Q2: Technical contribution of Theorem 1] :

Our main contribution is to show that for many natural function classes $\mathcal{F}$ with finite VCDim, its induced DM function class $\mathcal{H}_{\mathcal{F}}$ also has bounded PDim, which is highlighted in our Proposition 3. Putting together with Theorem 1, we derive sample complexity bounds in terms of the PDim of the social distortion class and the VC dimension of the function class.

Thank you for the clarification on the definition.

I think the new version of the definition is a lot more intuitive for the problem. This is a technical paper, and I did not have sufficient time to verify that all proofs hold for the new definition. Given that, I raised my rating to a 5, and reduced my confidence by 1.

[Concern-1: Main takeaway] :

In addition to our technical contributions, the key takeaway for the platform is that minimizing the overall engagement with harmful social trends cannot be simply achieved by deploying a content moderator that broadly impacts most content, because such an approach inherently risks significant infringement on free expression. Our work formally characterizes this dilemma faced by the platform and provides a novel approach to addressing the trade-off between minimizing social distortion and preserving free expression in a manner that is both sample-efficient and computationally efficient. For a comprehensive summary of the contributions of our paper, please refer to our common response. We will highlight these points clearer in our next version.

[Concern-2: Key features that separate our model from previous work] :

We thank the reviewer for raising this issue. We believe the uniqueness of our problem setting is reflected in the following three aspects, which we will highlight in our revision:

1. Curbing harmful trends as opposed to maximizing accuracy: In strategic classification, the goal of the learner is to maximize accuracy in the presence of strategic behavior. Usually, it is assumed that agents engage in different kinds of activities, either gaming or improvement actions, in order to receive a desirable (positive) label, and the goal of the mechanism designer is to achieve high accuracy. However, in our framework, we assume there are some harmful trends, and the mechanism designer’s goal is to prohibit users from engaging with these trends as much as possible. At the same time, the mechanism designer’s goal is to protect the freedom of speech of the users as much as possible. The latter is specific to social media platforms. We formulate this problem as minimizing social distortion subject to freedom of speech constraints.

2. Free speech constraint: We introduce a practical free speech constraint and demonstrate that achieving the goal of curbing harmful trends under this constraint is inherently challenging. To the best of our knowledge, this perspective—balancing harm mitigation with free speech considerations—has not been explored in related work, making it a novel and practical contribution to the study of online content moderation.

[Concern-3: Relation to adversarial information retrieval] :

We sincerely appreciate the reviewer for bringing this relevant line of work to our attention, and we will add discussions in our next version.

[Concern-4: Clarification on free speech] :

First, we would like to clarify that the precise definition of freedom of speech encompasses multiple aspects. In our model, we focus on a particularly apparent and compelling definition: the inability of a user to express their intended message due to censorship by an authority. In contrast, in the case (b) you proposed, the user still has the opportunity to express themselves but must do so carefully to adhere to specific guidelines. This represents a more nuanced aspect of freedom of speech, likely one of the reasons it is formally addressed in legal frameworks. In fact, we account for this situation in our model under the concept of social distortion.

Second, we would like to address a misunderstanding by the reviewer regarding our definition of free speech violation, specifically in relation to (a) (i.e., whether a user chooses to leave the platform). To clarify, content moderation in our model targets the content produced by users, not the users themselves. From a user’s perspective, their freedom of speech is violated when they intend to publish content that is subsequently restricted by the platform. When faced with such restrictions, a typical user has two choices: (1) modify their content to conform to the guidelines so it can be published, or (2) refrain from expressing the intended content altogether. We consider the first outcome as the social distortion, while the second as the violation of freedom of speech.

[Our main contribution] : Please refer to the common response.

[Relaxation may be impractical] :

To make OP (13) tractable, we introduce two relaxations and here we justify why they are practical.

1. First, to make the objective total loss function differentiable, we introduce a smooth approximation of a single loss on the benign side (see the left two panels of Figure 3, where the smoothed proxy loss function round at y=0). A similar technique is also widely used to make a strategic empirical risk minimization problem practical (see section 3.2 in [Levanon, Sagi, 2021])

2. Second, we relax the hard constraint in OP (13) by introducing a soft penalty term with a controllable regularization parameter, $\lambda$, which is a common technique in machine learning and optimization. In practice, the platform can solve this relaxed problem by experimenting with different values of $\lambda$ on offline data. Specifically, if the solution for the current $\lambda$ satisfies the hard constraint, the platform can reduce $\lambda$ to potentially achieve a better objective value under a tighter constraint. Conversely, if the solution for the chosen $\lambda$ violates the hard constraint, the platform should resolve the relaxed problem with a larger $\lambda$ to prioritize constraint satisfaction.

This relaxation provides a practical framework to balance two competing objectives: minimizing social distortion and avoiding violations of free speech. The trade-off between these objectives, as influenced by the choice of $\lambda$, is illustrated in the rightmost panel of Figure 3.

We will expand on this discussion and clarify our justification in the revised version of the paper.

[1] Levanon, Sagi, and Nir Rosenfeld. "Strategic classification made practical." International Conference on Machine Learning. PMLR, 2021.

[W1/Q1: How to handle legal constraints]:

For content that violates legal standards or crosses established red lines, the platform can incorporate a pre-processing step to automatically remove such content before applying the content moderation function $f$ to the remaining material.

To further clarify, our problem is motivated by the challenge after applying such a hard pre-processing step: the role of the content moderator in our problem is to address harmful yet not necessarily illegal social trends, such as hashtags that might incite hate speech, inappropriate content, or controversial discourse. Unlike the outright removal of content that breaches legal or non-negotiable boundaries, moderating such content involves a degree of discretion on the part of the platform. This discretion allows the platform to establish specific filtering rules or thresholds, which provides the flexibility to design an effective moderation function $f$ tailored to the problem at hand.

[W2/Q2: Single social trend] :

First, we argue that in many practical scenarios, strategic manipulation can often be captured by a single dominant social trend, such as the most popular Twitter hashtag at a given time, which can be identified with relative ease. Our framework allows the platform to handle the potentially harmful and drifting dominant social trend over time, by simply adapting the moderator according to the current trend. Additionally, our framework can be naturally extended to account for multiple social trends, as long as each content $x$ is associated with a trend $e_x$​, and the joint distribution of $(x, e_x)$ in the offline dataset aligns with the online environment. In this generalized setting, the desirable manipulation direction for each content creator follows a fixed distribution that is independent of the deployed content moderator. However, in situations where content creators pursue multiple social trends and their choice of which trend to follow depends on the content moderator, it is unclear whether the same guarantees would hold. We plan to extend our framework in future work to address this more complex and dynamic scenario.

[W3: Motivation of our model choice] :

In the strategic classification literature, usually, the cost of manipulation is captured using the $\ell_2$ distance of the manipulated distance (see [1] among others.). Similarly, we measure the cost of manipulation using the $\ell_2^2$ function. Furthermore, since the inherent goal of the agent is to align with a social trend as much as possible, we measure that using the inner product of the manipulated state and the social trend.

[Q3: Compare with the optimal filter] :

First, as stated in Theorem 2, computing the globally optimal filter $f$ is computationally intractable. This implies that even for our synthetic dataset, we must rely on heuristics or brute-force approaches to approximate a near-optimal solution, which is both cumbersome and time-consuming. More importantly, the primary goal of our experiments is not to evaluate how closely our relaxation approximates the optimal solution, but to illustrate the trade-off between social distortion and freedom of speech and to propose a practical method for platforms to balance these objectives effectively. Consequently, we believe that a direct comparison to the optimal solution is not particularly relevant to the central message of our work.

[1] Ahmadi, S., Beyhaghi, H., Blum, A. and Naggita, K., 2021, July. The strategic perceptron. In Proceedings of the 22nd ACM Conference on Economics and Computation (pp. 6-25)