DeepHalo: A Neural Choice Model with Controllable Context Effects

We thank Reviewer D2h9 for their positive recognition of our work. In the revision, we will include a model architecture diagram and correct the noted typos. Below, we provide point-by-point responses to the reviewer’s comments.

Responses to Weakness

Q: In experiments, I think it is essential to provide the number of parameters of each benchmark model to make sure the better performance of DeepHalo is from the design instead of the size.

A: We thank the reviewer for raising the question regarding the model scale. To address this, we computed the number of trainable parameters under the Expedia experiment setting, where the input item feature dimension is set to 10 across all models. The results are summarized below:

Q: While many benchmark models are compared in datasets with features (Table 2), while in featureless datasets (Table 1), only a few models are tested. However, I think the models like FateNet, TCNet can also be applied to featureless cases.

A: We thank the reviewer for their suggestion. In the featureless setting, we have added results for three representative baselines: FATE, TCNet, and Mixed MNL. To ensure fairness and focus on architectural differences, we controlled the parameter size of all neural network–based models to be approximately 1K parameters. The results are shown below:

Responses to Questions

Q: The formulation of higher-order interactions in Equations (4) and (5) appears conceptually similar to the transformer architecture, with the key difference being that the attention mechanism is replaced by a linear transformation followed by an activation function (just like assigning equal attention across elements). Could you elaborate on the differences between your approach and transformers, both in terms of formulation and implications for model expressiveness or computational efficiency?

A: This is a highly insightful observation. Indeed, the connection to Transformer-style architectures is deeply embedded in the design of our approach, and we appreciate the opportunity to elaborate on this.

In summary, there are two differences between DeepHalo and Transformers, both structurally and in modeling implications:

• Effect order control:
  Transformers (e.g., in TCNet) implicitly allow interactions across all items, inevitably resulting in full-order interactions (order $|S| - 1$). In contrast, DeepHalo provides explicit layer-wise control, where each layer adds exactly one level of interaction (or $\times 2$ with quadratic activation). This control is essential for interpretability, regularization, and tractability.

• Structural simplification:
  DeepHalo eliminates the need for query/key/value projections and softmax attention, replacing them with fixed linear transformations and polynomial activations. This results in lower computational cost—linear in $|S|$, rather than quadratic—as well as a simpler and more transparent architecture.

Below, we briefly explain the motivation behind our model design to provide a more intuitive understanding of our model architecture.

We begin with the featureless setting, where modeling higher-order context effects reduces to polynomial regression over indicator functions that encode item presence. DeepHalo introduces a structured and recursive mechanism to capture subset-level interactions in this setting, conceptually analogous to the accumulation of interactions based on binary inclusion indicators.

In the feature-rich setting, the core challenge is to encode interactions among a subset of items using their feature representations, while strictly excluding information from outside the subset. This naturally recalls the concept of attention scores, typically computed from item feature pairs. However, when these scores are normalized via a softmax function, the resulting attention weights entangle all item features, thus violating the independence required for controlled halo modeling.

For instance, when evaluating the influence of subset $\{A, B\}$ on the utility of item $C$ within the offer set $\{A, B, C, D\}$, the effect should depend solely on the features of $A$, $B$, and $C$, and remain invariant to unrelated items such as $D$. The softmax operation introduces information from all items into the denominator of each attention weight, which naturally violates this principle.

To address this, we omit softmax normalization, and further observe that the dot-product attention mechanism is not essential for modeling decomposable utility functions. This insight motivates a simplification of the architecture, leading to the formulations presented in Equations (4) and (5).

In short, while DeepHalo draws conceptual inspiration from Transformer-style aggregation, it diverges in architectural design to achieve interpretable, order-controllable, and computationally efficient modeling of contextual choice effects. We thank the reviewer again for raising this connection and will add a discussion in the final version to highlight these points explicitly.

Thank you very much for your thoughtful comments. Please see our response as follows.

Weakness & Questions:

Q: The methodological ML contribution ... the contributions are not very novel.

A: The contribution of DeepHalo lies not merely in modeling interactions or equivariance, but in introducing a structured and order-controllable framework . While prior models have explored interaction effects, they face fundamental challenges:

• First-order models such as CMNL [Yousefi Maragheh et al., 2020], low-rank Halo MNL [Ko and Li, 2024], and FETA [Pfannschmidt et al., 2022] are inherently restricted to pairwise interactions. For instance, FETA proposes a decomposition that resembles the halo effect, but practically, it is limited to first-order pairwise interactions. The architecture is not scalable to higher-order effects, as doing so would require exponentially large tensor storage and computational resources, rendering it impractical for real-world applications.

• Other higher-order models, such as FATE [Pfannschmidt et al., 2022], ResLogit [Wong and Farooq, 2021], and TCNet [Wang et al., 2023], attempt to capture context effects by entangling all item features in the offer set. While their outputs can be post hoc interpreted as halo effects via Eq. (10), the interaction order is uncontrolled, and recovering a meaningful decomposition requires $O(2^{|S|-1})$ complexity due to exponentially many terms. This stems from their architectures: context is not modeled in a structured or decomposable way. For instance, FATE compresses all item features into a set embedding, while TCNet’s softmax-based attention mixes information across items. As a result, none can recover a truly decomposable utility function. We refer the reviewer to Appendix A2.2, A2.3, and Question 2 of our response to reviewer Unm8 for details on how DeepHalo addresses this limitation.

In contrast, DeepHalo is explicitly designed to address this gap by enabling precise control over the maximum order of halo interactions. In e-commerce applications like recommendation and bundle pricing, lower-order context effects are often the most relevant—being more stable, interpretable, and practically useful. While expressive models can recover such effects, they do so implicitly within entangled high-order interactions, requiring up to $O(2^{|S|-1})$ terms to isolate. DeepHalo explicitly models limited-order halo effects, allowing users to control the maximum order $K$ with only $O(2^K)$ complexity, enabling a flexible trade-off between efficiency and expressiveness.

Q: If the contribution relates to DCM behaviour modeling for the ML community, then some aspects should be considered... (Poor choice of (classical DCM) baselines)

A: Since our model targets context effects, we primarily compare DeepHalo with context-dependent DCMs like Contextual MNL. MNL serves as a sanity check to highlight the role of context. To include context-independent RUM-based models, we add Mixed Logit in the featureless setting, as it can approximate any RUM. For ML baselines, we include TCNet and FATE (suggested by reviewer D2h9). We respectfully refer the reviewer to our detailed response to Reviewer D2h9.

Q: Lack of training (computational) performance statistics.

A: Thanks for pointing this out. Below, we add the total training time (until early stop) in a sample run under the setting of the LPMC experiment in Section 5.2.

Q: However, it’s very unclear to me why this fits in the NeurIPS context.

A: We believe this paper is well-suited for the NeurIPS community, aligning with its growing focus on human-AI alignment and feedback modeling. Understanding and formalizing human choice behavior is essential for building AI systems that interact reliably with people. Existing alignment methods (e.g., RLHF, LfD) often assume rational, unbiased feedback—assumptions that rarely hold in practice. Our work introduces a principled, interpretable framework for modeling context-dependent human choices, laying the foundation for more realistic and user-aligned AI.

Additionally, we list below several noteworthy works related to our topic from previous years’ NeurIPS:

[1]Oh S, Thekumparampil KK, Xu J. Collaboratively learning preferences from ordinal data. Advances in Neural Information Processing Systems. 28, 2015

[2]Stephen Ragain and Johan Ugander. Pairwise choice markov chains. Advances in Neural Information Processing Systems, 29, 2016

[3]Arjun Seshadri, Stephen Ragain, and Johan Ugander. Learning rich rankings. Advances in Neural Information Processing Systems, 33:9435–9446, 2020

[4]Yiqun Hu, David Simchi-Levi, and Zhenzhen Yan. Learning Mixed Multinomial Logits with Provable Guarantees. Advances in Neural Information Processing Systems, 9447-9459, 2022

[5]De Peuter S, Zhu S, Guo Y, Howes A, Kaski S. Preference learning of latent decision utilities with a human-like model of preferential choice. Advances in Neural Information Processing Systems, 37:123608-36, 2024

Clarifications:

Q: Some things are a bit unclear in intro.

A: We will explain the ‘Featureless setting’ further by elaborating on ‘items without access to their features’.

Q: “For each j ∈S, define a utility function uj(T), ...”

A: We will only introduce the utility function $u_j(S)$ instead, and only introduce the notation $T$, which emphasizes the context information (subset excluding the item j itself).

Q: “Here, $XT∪{j}∈R^{d_x ×J}$ denotes the matrix formed by replacing the feature vectors not in the subset T ∪{j}with zero.” —> rephrase.

A: We will rewrite the definition of $X_{T∪{j}}$ as “formed from the feature matrix $X_S$ by replacing the feature vectors not in the subset T∪{j} with zero.”

Q: This sentence got me a bit confused: “Here, H denotes the number of interaction heads...”

A: Analogous to multi-head attention, in our context, this can be interpreted as H different consumer types with different tastes. The h-th row of W^1 represents the h-th taste vector, which can map the input with various preferences. Thus, W^1 provides H variety of mappings, which are then aggregated by the sum over $Z_h^1$. This is indeed training each of the h-th mapping, and we use matrix W^1 to compact the notation.

Q: “Residual Connection for Large Choice Sets” is quite confusing

A: As the reviewer noted, one key computational advantage of the polynomial formulation is avoiding the need for deep architectures (line 234). We elaborate on this in Section 4.2 (lines 262–264), showing a logarithmic relationship: a linear number of layers can express exponentially many interaction orders. We will clarify this in the revised version. See also Appendix A2.2, A2.3, and the example in our response to reviewer Unm8 (Section 2).

As for numerical stability, training remains stable in the featureless setting even with high-order interactions (Figure 2). In the feature-rich setting, deep quadratic stacks may cause instability, but we find that modeling up to 4th or 5th order suffices in practice (e.g., Expedia), striking a good balance between expressiveness and interpretability.

Q: For section 4.2., $e_S$ is a vector of 1s...

A: Regarding Section 4.2: $e_S$ is a unit vector with the $(e_S)_j = 1$ if j ∈ S and 0 otherwise. The recursive expression in equation 9 can be represented as a polynomial of $e_S$. Let us consider the case with up to second-order interaction, then the second-order term relates to $e_S \otimes e_S$ where the (i,j)-th element refers to whether both item i and item j exist in the choice set. The coefficients in the polynomial corresponding to each element of the second-order term measure the specific interactions in equation (1).

Q: confusing with the meaning of $\cup$ in equation 3.

A: We apologize for the typo in equation 3: $X_R$ should be $X_{R∪{j}}$, which hopefully entangles the confusion. In equation (10), the relative context effect is specified on item j and item k, whose definition is introduced in Park and Hahn [1998].

Q: About number of parameters in Fig 2...

A: The number of nn parameters in Fig 2 is the total number of trainable parameters, which specifies the model complexity and is influenced by the problem size and the selection of hyperparameters like H.

Q: $\bar{Z}_1 := 1/S$…

A: Yes, thanks for catching that. We will correct to |S|.

Beverage Experiment

Thank you for the thoughtful comment.

This classical hypothetical dataset has been widely used in the literature on halo effects (e.g., Park and Hahn [1998], Batsell and Polking [1985]) for its simplicity and clarity, making it a standard example for evaluating interpretability. The experiment is conducted in a featureless setting, demonstrating that DeepHalo can extract meaningful interaction effects purely from set-level preferences.

Subfigure (b) illustrates how different item subsets (horizontal axis) influence the utility gap between item pairs (vertical axis). The rightmost column (∅) shows baseline preferences without contextual influence—for example, strong preferences for Pepsi over Coke and 7-Up over Sprite are evident both here and in the market shares (Subfigure a). The leftmost column (item 1) reveals a strong negative halo effect of Pepsi on Coke, reducing Coke’s competitiveness when co-offered. A similar pattern is observed between 7-Up and Sprite.

We truly appreciate Reviewer B9Eq’s positive feedback. To address the questions and concerns about our method, we provide point-wise responses as follows.

Weakness and Limitation

Q: Could be clearer what the advantages are of the proposed method vs existing neural context effect models

A: The key advantage of DeepHalo lies in its ability to explicitly control the maximum order of context (halo) effects, enabling a flexible trade-off between model expressiveness and interpretability—something existing neural models lack.

• First-order models (e.g., CMNL [Yousefi Maragheh et al., 2020], FETA [Pfannschmidt et al., 2022]) are interpretable but limited to pairwise interactions and cannot scale to higher-order effects due to computational constraints.
• High-capacity models (e.g., FATE [Pfannschmidt et al., 2022], ResLogit [Wong and Farooq, 2021], and TCNet [Wang et al., 2023]) are expressive but entangle all item features, making it infeasible to control interaction order. Recovering meaningful low-order effects from these models requires evaluating up to $O(2^{|S|-1})$ terms.

In contrast, DeepHalo models context effects in a structured and decomposable way. It allows practitioners to specify the maximum interaction order $K$, with a complexity of $O(2^K)$ to recover the effect terms, making it both efficient and interpretable. This is especially useful in real-world settings like recommendation and bundle pricing, where lower-order effects are often sufficient and desirable.

Q: The title, though, is a bit generic, and could apply to several of the cited papers; ...

A: Thanks for your suggestion! We will consider adding some unique elements to our title that help distinguish it from other context-dependent model papers.

Q: How does the sample complexity and need for hyperparameter tuning compare to existing approaches?

A: Regarding sample complexity and data efficiency, we refer the reviewer to our response to Reviewer Unm8 (Q: About data efficiency), where we provide an experiment evaluating model performance under varying data proportions.

As for the hyperparameter tuning, we find empirically that DeepHalo is relatively easy to tune. Once the maximum effect order is specified, the overall effect complexity is fixed, which is the main tuning difference compared to existing approaches. Additional parameters are then introduced only to improve the approximation accuracy of these effects, rather than to increase model capacity arbitrarily.

1. Clarifying Section 4.1 — why Equation (8) needs only $\log_2l$ layers to model $l$‑th‑order interactions

We agree that Section 4.1 should state this property explicitly. Currently, this is mentioned in Section 4.2, and the explanation is deferred in the appendix. In the revision, we will:

1. Add a sentence to Section 4.1

   “Because each Equation (8) layer doubles the maximum attainable interaction order, an l‑th‑order effect can be represented with at most ⌈log₂ l⌉ stacked layers.”

2. Provide an intuitive explanation

   • A single layer (Equation (8)) captures up to second‑order effects.
   • Stacking two layers composes these interactions, yielding up to 4‑th‑order effects, and so on.
   • After k layers the highest order is $2^{k}$, giving the logarithmic bound.

2. Practical guidance on when to use Equation (5) vs Equation (8)

We appreciate this insightful question. Generally, we recommend:

• Using Eq. (5) when training stability is a concern or when higher-order context effects are not considered.
• Using Eq. (8) to reduce network depth when higher-order interactions are considered.
• Combining them for a tunable trade-off.

Here we provide some details for further intuition. One of the strengths of the DeepHalo architecture is its structural flexibility: the activation function σ can be instantiated as any polynomial function. (5) and (8) can be used interchangeably within the same model. In practice, they can be freely mixed—for example, stacking layers with linear and quadratic activations can achieve effective interaction orders such as $((1+1+1)\times2+1)\times2$, offering nuanced control over expressivity.

• Higher-degree activations (like Eq. (8)) allow faster growth in representational power with depth, efficiently capturing higher-order effects.
• However, very high-degree activations in deep networks may introduce training instability, such as exploding gradients.
• Empirically, we observe that for the same representable order, both variants achieve similar accuracy. Yet, first-order activations (Eq. (5)) tend to be more stable, especially on noisy or real-world datasets.

This modularity makes DeepHalo a unified and controllable framework for context-dependent choice modeling, in both feature-rich and featureless settings. We will add a guidance paragraph in the revision to clarify these trade-offs.

3. Clarifying what makes DeepHalo more interpretable than prior neural context-effect models

We appreciate the reviewer’s thoughtful question. While many prior models can interpret the effects using Eq. (10), this is under two premises:

• The model is expressive enough to include all interaction orders up to $(|S|-1)$ instead of some specific orders (which first-order models like CMNL and FETA cannot achieve)
• Existing expressive models like FATE and TCNet require a computational complexity of $O(2^{|S|})$ for recovering all halo effects in the model.

In contrast, our expressive DeepHalo model can capture up to a specific order and only needs $O(2^K)$ computational time to obtain $k$-order interactions, which provide more interpretability. For example, modeling third-order effects over a catalog of 100 items without such control would require handling over 160,000 subsets, which is computationally prohibitive and interpretively infeasible.

Besides, the exact-interaction-order control inherited in our model is of practical interest. As a concrete example, consider a choice set $\{A, B, C, D\}$. Then, estimating how the subset $\{A, B\}$ influences the utility of item $C$, the function must depend only on the features of $A$, $B$, and $C$, and must not involve irrelevant items like $D$. This subset-specific conditioning is critical for interpretability, but is not enforced in existing context-dependent ML models.

The reason the existing models cannot capture exactly up to $k$-order interactions is that they attempt to learn context by directly entangling all item features. For instance, FATE combines global context with individual features but cannot restrict interaction depth. TCNet’s attention mechanism inherently mixes all items, effectively modeling full-order interactions by design. As a result, these models do not formulate context effects in a structured, decomposable way, making interpretation difficult.

On the contrary, our model achieves more interpretability with a modular design:

• Each layer introduces exactly one additional interaction order or $\times 2$ using quadratic activation.
• The maximum effect order is bounded by the network depth $L$.
• Residual and polynomial structures guarantee halo-decomposable utility functions. We will give a simple example later for better illustration.

4. Clarifying the distinction between DeepHalo and SDA [Rosenfeld et al., 2020]

Thank you for the insightful observation. While DeepHalo shares some high-level similarities with set-dependent embedding (SDA) approaches such as [Rosenfeld et al., 2020, Eq. (2)], we would like to emphasize several key distinctions in both motivation and design, some are similar to Point 3.

DeepHalo is fundamentally grounded in halo effect theory, which prioritizes interpretable higher-order interactions. While simply incorporating context information (e.g., through contextual functions like $\phi$) is straightforward, interpretability requires controlling the maximum order of interactions—a key principle not enforced by SDA.

As the reviewer rightly noted, if Eq. (10) were applied to arbitrary context-based architectures (e.g., SDA), the number of halo terms would grow exponentially with the size of the offer set. This would result in a combinatorial explosion of interaction terms that are computationally hard and practically uninterpretable.

In contrast:

• SDA and related models aggregate item features in a fully entangled way.
  • This makes it virtually impossible to recover the halo effect without enumerating all subsets—an exponentially large and ill-posed task.
• DeepHalo, by design:
  • Uses residual connections and polynomial activations to explicitly control and construct the interaction order.
  • Ensures that utilities are halo-decomposable and that item features remain disentangled.
  • Caps model expressiveness to low-order interactions, preserving interpretability and generalization.

In short, while SDA enables generic set-dependent context modeling, DeepHalo provides a theoretically-grounded and architecturally-enforced approach to interpretable, order-controlled context effects, which is essential for practical discrete choice applications.

For points 3 & 4, we refer the reviewer to Appendix A2.2 and A2.3, as well as the illustrative example in Question 2 of our response to reviewer Unm8, for a detailed explanation of how DeepHalo supports structured, decomposable modeling of context effects.

We sincerely appreciate your recognition of our problem modeling, methodological soundness, real-world validation, and thoughtful baseline selection.

To address the concerns, we provide detailed point-wise responses below.

Q: As far as I can see, the specific decomposition of the utility function which DeepHalo uses mirrors FETA’s almost exactly...

A: The key methodological contribution of DeepHalo lies in introducing a structured and order-controllable framework that balances expressiveness and interpretability—a capability that is lacking in existing context-dependent choice models. Among context-dependent models,

• First-order models such as CMNL (Yousefi Maragheh et al., 2020), low-rank Halo MNL (Ko and Li, 2024), and FETA (Pfannschmidt et al., 2022) are inherently restricted to pairwise interactions. For instance, FETA proposes a decomposition that resembles the halo effect, but in practice, it is limited to first-order pairwise interactions. The architecture is not scalable to higher-order effects, as doing so would require exponentially large tensor storage and computational resources, rendering it impractical for real-world applications.

• Other higher-order models, such as FATE (Pfannschmidt et al.,2022) and TCNet (Wang et al., 2023), attempt to learn context effects by directly entangling all item features within the offer set. While their outputs can be retrospectively interpreted as halo effects via Eq. (10), the interaction order is not explicitly controlled. Recovering a meaningful decomposition in these models would require computational complexity on the order of $O(2^{∣S∣−1})$, due to the exponential number of possible interaction terms. This limitation stems from their architectural design: context effects are not modeled in a structured or decomposable way. Specifically, FATE aggregates all item features into a set embedding, which is then combined with individual item features through complex nonlinear transformations, obscuring the origin and order of interactions. TCNet, similarly, uses attention mechanisms where the softmax denominator implicitly mixes information from all items in the offer set, making it difficult to isolate individual effects. As a result, neither model can recover a truly decomposable utility function.

In contrast, DeepHalo is explicitly designed to address this gap by enabling precise control over the maximum order of halo interactions. To the best of our knowledge, it is the first model that supports flexible trade-offs between model complexity and interpretability in this context.

In e-commerce applications such as recommendation systems and bundle pricing, practitioners are often primarily interested in lower-order context effects, which are more stable, interpretable, and practically useful. While expressive models can theoretically recover such effects, they do so implicitly as part of an entangled representation involving many higher-order interactions. Consequently, low-order effects cannot be meaningfully isolated. DeepHalo offers a principled approximation that explicitly models limited-order halo effects. It allows users to control the maximum interaction order $K$, with computational complexity only $O(2^K)$, offering a flexible trade-off between efficiency and expressiveness.

Our contribution goes beyond what Deep Set offers. While DeepSets provides permutation invariance, DeepHalo integrates domain-specific insights from discrete choice theory and the halo literature, along with architectural elements like residual connections and polynomial activations. This enables interpretable, high-order interaction modeling in a principled and controllable way.

Q: Furthermore, I’m not convinced that the proposed network architecture replicates the utility decomposition the paper aims for (line 189)...

A: Thank you for raising this point. The key intuition is that every recursion depth l in Eqs. (4)–(5) adds exactly one additional order of interaction while preserving all lower‑order terms via the residual connection. The process is the same as Appendix A 2.2 and A 2.3. To better facilitate the understanding, we provide a simple example below.

Consider an offer set ${j, k, l}$ and let $\phi$ be an identity mapping. We focus on the change of $z_{j}$ in DeepHalo.

1. Pairwise (1‑st order) layer.
   The first layer aggregates the linear summary vector $\bar Z^{1}$ from raw embedding vector $z^{0}$ and modulates them with $z_{j}^{0}$. This yields
   $z_{j}^{1}=z_{j}^{0} + \frac1H\sum_{h}\bar Z_{h}^{1} \cdot z_{j}^{0}$, which contains only pairwise interactions between the target alternative $j$ and every other item $k, l$ in the set. If we denote $f_j$ as any function containing only item j's feature information, the above output can be formulated as, $z_{j}^{1} = f_j + \frac1H\sum_{h}(f_j + f_k + f_l) \cdot f_j$, thus it contains zero order term $f_j$ and first order term $f_k \cdot f_j$ and $f_l \cdot f_j$, second order term $f_j \cdot f_k \cdot f_l$ won't appear. For simplicity, we use $f_{ab}$ to represent any term like $f_a \cdot f_b$ and $f_b \cdot f_a$, which only contains the features of item a and item b.

2. Second‑order layer.
   After the first layer, $z_{k}^{1}$ and $z_{l}^{1}$ already embed their pairwise effects with every other item.
   The second layer builds

   $\bar Z^{2}=\tfrac1S\sum_{m}W^{2}z_{m}^{1}$, which contains term $f_j, f_k, f_l, f_{jk}, f_{jl}, f_{kl}$.

   $z_{j}^{2}=z_{j}^{1}+\tfrac1H\sum_{h} \bar Z_{h}^{2} \cdot z_{j}^{0}$ which is analogous to $(f_j + f_{jk} + f_{jl}) + \sum_h(f_j + f_k + f_l + f_{jk} + f_{jl} + f_{kl}) \cdot f_j$, which brings about second order term $f_{jkl} = f_{kl} \cdot f_{j}$. It can be interpreted as the effect of ${k,l}$ on $j$.

This example illustrates how we can add up orders by increasing the number of layers and get a decomposable utility representation. If we change the $\cdot$ in layers to some other polynomial function, the term order will grow faster, like in the first layer we change to $(f_j + f_k + f_l)^2 \cdot f_j$, in the sum will directly bring about the second order term $f_{jkl}$. The quadratic activation naturally brings a $log_2$ form depth requirement.

Q: The purpose of $(-1)^{|T|-|R|}$.

A: The $(−1)^{∣T∣−∣R∣}$ factor in Eq. (3) is the inclusion–exclusion inversion coefficient on the Boolean lattice of subsets. Our $v_j(⋅)$ must invert the cumulative‑sum relation $u_j(X_S)=\sum_{T\subset S\setminus\{j\}} v_j(X_{T\cup\{j\}})$; effects contributed to many supersets are otherwise over‑counted. Alternating signs guarantee that terms appearing an even number of times cancel and those appearing an odd number of times net to the correct single contribution, yielding a unique decomposition.

Q: About Performance Difference between DeepHalo and TCnet...

A: Thanks for the insightful observation. We acknowledge that similar performance between DeepHalo and TCNet is expected; however, we wish to emphasize a key difference in their modeling philosophies.

While TCNet can approximate utility decompositions (maximum order), its Transformer architecture lacks control over the maximum interaction order. Attention layers aggregate across the full set, leading to large quantities of high-order effects that are difficult to interpret. In contrast, DeepHalo explicitly controls interaction order through its recursive structure, enabling better interpretability and identifiability.

Given that low-order interactions often dominate in real-world settings, it is not surprising that TCNet matches DeepHalo's accuracy. However, DeepHalo achieves this with controlled expressiveness, offering a more interpretable alternative. Rather than outperforming black-box models in accuracy, our main goal is to provide a principled, order-aware design that balances predictive performance with transparency.

Q: About data efficiency

To evaluate data efficiency, we subsample the SFOshop dataset—known for strong context effects—at ratios from 10% to 100%, training all neural models under a strict ~1k parameter budget.

Across all settings, DeepHalo consistently achieves the lowest Test NLL, showing strong generalization even with limited data. Its Test NLL steadily improves with more data, while other models (TCNet, MLP, FATE) either plateau early or fluctuate.

Clarification

The expression $u_j(x_j, X_R)$ in Eq. (3) refers to the same function as $u_j(X_{R \cup \{j\}})$ in Eq. (2), and we will revise the notation for consistency. We will also correct the duplicate citation of Deep Sets [Zaheer et al., 2018], fix the misspelling of FETA and FATE (“E” stands for “evaluate”), and use $|S|$ consistently to denote set size. These issues will be addressed in the final version. We appreciate the reviewer’s careful reading.