C3. The reweighting formulation recovers different Sliced-Wasserstein variants

In our paper, we provide $2$ examples of Max-SW and Energy-based SW. Here, to supplement our response, we include additional instances, which will be added to the paper.

• Max-SW [7], Markovian SW [3], and EBSW [2] implicitly use $\phi_{\mu,\nu}(\theta)=W_p^p(\theta_{\sharp} \mu,\theta_{\sharp} \nu)$ to measure the informativeness of $\theta$. In other words, slices that have higher projected distances are considered more important/informative.

• On the other hand, RPSW [4] implicitly uses $\phi_{\mu, \nu}(\theta; \mu, \nu, \gamma_{\kappa}) = E_{(X,Y)\sim\mu \times \nu}\left[ \gamma_{\kappa}(\theta; P_{\mathbb{S}^{d-1}}(X - Y)) \right]$ where $\gamma_\kappa$ is a location-scale distribution (i.e., von Mises-Fisher) and $P_{\mathbb{S}^{d-1}}$ is the projection onto $\mathbb{S}^{d-1}$. Slices that align well with random paths connecting $2$ distributions are considered more important/informative.

For the above instances, one may also refer to $\phi_{\mu,\nu}$ as the discriminant function. However, note that we define the concept of informativeness more broadly, allowing for a broader set of assumptions about data structure rather than strictly referring to the ability to discriminate between distributions.

For completeness, we show below how to recover different SW variants using our formulation with different notions of informative slices:

Note: Since the \widetilde{SW} cannot be displayed, we resort to using \sim SW, referring to the rescaled sliced-Wasserstein.

• Classical SWD [6]: We set $\phi\equiv 1$ and obtain the classical Sliced-Wasserstein distance $\sim SW_p^p(\mu,\nu;\sigma, \rho_\phi) = SW_p^p(\mu,\nu;\sigma)$

• Max-SW [7] : We set $\phi_{\mu,\nu}(\theta)=W_p^p(\theta_\sharp\mu,\theta_\sharp\nu)$ and $\rho_\phi(r)=\delta_{r_{max}},\sigma=\mathcal{U}(\mathbb{S}^{d-1})$ where $r_{max}=\max\phi_{\mu,\nu}(\theta)$. Then, we have that $\sim SW_p^p(\mu,\nu;\sigma,\rho_\phi)=Max\text{-}SW_p^p(\mu,\nu)$

• EBSW [2]: We set $\phi_{\mu,\nu}(\theta)=W_p^p(\theta_\sharp\mu,\theta_\sharp\nu)$, $\rho_\phi(r)=\frac{f(r)}{\int_{\mathbb{S}^{d-1}} f(W_p^p(\theta_\sharp\mu, \theta_\sharp\nu)) d\sigma(\theta)}$, $\sigma=\mathcal{U}(\mathbb{S}^{d-1})$ where $f : [0, \infty) \to (0, \infty)$ is an increasing energy function (e.g., $f(x) = e^x$). Then, we have that $\sim SW_p^p(\mu,\nu;\sigma,\rho_\phi) = \int_{\mathbb{S}^{d-1}} \rho_\phi(\phi_{\mu,\nu}(\theta)) W_p^p(\theta_\sharp\mu,\theta_\sharp\nu) d\sigma(\theta) = \int_{\mathbb{S}^{d-1}} \frac{f(W_p^p(\theta_\sharp\mu,\theta_\sharp\nu))}{\int_{\mathbb{S}^{d-1}} f(W_p^p(\theta_\sharp\mu, \theta_\sharp\nu)) d\sigma(\theta)} W_p^p(\theta_\sharp\mu,\theta_\sharp\nu) d\sigma(\theta) \nonumber\\ =EBSW_p^p(\mu,\nu;f).$

• RPSW [4]: We set $\phi_{\mu,\nu}(\theta) = E_{(X,Y)\sim\mu\times\nu}[\gamma_\kappa(\theta; P_{S^{d-1}}(X - Y))]$, $\rho_\phi(r) = r$, $\sigma=\mathcal{U}(\mathbb{S}^{d-1})$, where $\gamma_{\kappa}$ is a location-scale distribution with parameter $\kappa$. Then, we have that $\sim SW_p^p(\mu,\nu;\sigma,\rho_\phi) = \int_{\mathbb{S}^{d-1}} \phi_{\mu,\nu}(\theta)W_p^p(\theta_\sharp\mu, \theta_\sharp\nu) d\sigma(\theta) = E_{\theta\sim\phi_{\mu,\nu}(\cdot)}\left[W_p^p(\theta_\sharp\mu, \theta_\sharp\nu)\right] = RPSW_p^p(\mu,\nu;\gamma_\kappa)$ where we abuse the notation $\phi_{\mu,\nu}$ to denote the distribution induced by function $\phi_{\mu,\nu}(\theta)$.

• Exploring other notion of informative slices based on different data assumptions is a potential venue for further research.

---

References:

[1] Nguyen et al. "Distributional sliced-Wasserstein and applications to generative modeling." (2020)

[2] Nguyen et al.. "Energy-based sliced wasserstein distance." (2024)

[3] Nguyen et al. "Markovian sliced Wasserstein distances: Beyond independent projections." (2023)

[4] Nguyen et al. "Sliced Wasserstein with random-path projecting directions." (2024)

[5] Nadjahi et al. "Fast approximation of the sliced-Wasserstein distance using concentration of random projections." (2021)

[6] Rabin et al. "Wasserstein barycenter and its application to texture mixing." (2011)

[7] Deshpande et al. "Max-sliced wasserstein distance and its use for gans." (2019)

C4. Relaxing the subspace assumption

In this section, we discuss the situation where the subspace assumption does not strictly hold. Complete proofs will be updated in the paper.

Proposition 1

Let $U,U^\perp$ be defined in the subspace assumption, choose $\mu^d,\nu^d\in P_p(\mathbb{R}^d)$, and let $\mu^k,\nu^k$ be defined by $U_\sharp\mu^d,U_\sharp\nu^d$, we claim the following:

$$W_2^2(\mu^k,\nu^k)\leq W_2^2(\mu^d,\nu^d)\leq W_2^2(\mu^k,\nu^k)+2(m_2(U^\perp_\sharp\mu^d)+m_2(U^\perp_\sharp\nu^d,p))\tag{1}$$

where $m_p(U^\perp_\sharp\mu^d)$ the $p$-th moment of measure $U^\perp_\sharp \mu^d$.

Proof sketch for Proposition 1

For each pair $(x,y)\in (\mathbb{R}^d)^2$, we have $\|P_U(x)-P_U(y)\|^2 \leq \|x-y\|^2, \quad \text{(By definition of projection)}$

$\|x-y\|^2 = \|P_U(x)-P_U(y)\|^2 + \|P_{U^\perp}(x)-P_{U^\perp}(y)\|^2, \quad \text{(Pythagorean theorem)}$

\leq \|P_U(x)-P_U(y)\|^2 + 2\|(U^\perp)^\top x\|^2 + 2\|(U^\perp)^\top y\|^2.$$ Suppose $\gamma$ is an optimal transport plan for $W_2^2\left( (P_U)^\sharp \mu, (P_U)^\sharp \nu \right).$ By integrating both sides of the first inequality via $\gamma$, we prove the first inequality in (1): $$W_2^2(\mu^k, \nu^k) = W_2^2((P_U)^\sharp \mu, (P_U)^\sharp \nu) \leq W_2^2(\mu, \nu).$$ Similarly, by integrating both sides of the second inequality via $\gamma$, we obtain the second inequality of (1). ## Proposition 2:

\frac{k}{d} SW_2^2(\mu^k, \nu^k) \leq SW_2^2(\mu^d, \nu^d) \leq \frac{k}{d} SW_2^2(\mu^k, \nu^k) + 2 \frac{d-k}{d} \big( m_2(U^\perp_\sharp \mu^d) + m_2(U^\perp_\sharp \nu^d) \big)

### Proof sketch for Proposition 2 Pick $\theta \in \mathbb{S}^{d-1}$ and $x, y \in \mathbb{R}^d$. We have that $$\|\theta^\top P_U(x) - \theta^\top P_U(y)\|^2 \leq \|\theta^\top x - \theta^\top y\|^2$$ $$\|\theta^\top x - \theta^\top y\|^2 = \|P_U(\theta)^\top P_U(x) - P_U(\theta)^\top P_U(y)\|^2 + \|P_{U^\perp}(\theta)^\top P_{U^\perp}(x) - P_{U^\perp}(\theta)^\top P_{U^\perp}(y)\|^2$$ $$\|P_U(\theta)^\top P_U(x) - P_U(\theta)^\top P_U(y)\|^2 + \|(U^\perp)^\top \theta\|^2 \|(U^\perp)^\top x - (U^\perp)^\top y\|^2, \quad \text{(Cauchy-Schwarz inequality)}$$ $$\|\theta^\top P_U(x) - \theta^\top P_U(y)\|^2 + 2\|(U^\perp)^\top \theta\|^2 \big(\|(U^\perp)^\top x\|^2 + \|(U^\perp)^\top y\|^2\big).$$ Similar to the proof of Proposition 1, we obtain: Choose $\theta \in \mathbb{S}^{d-1}$. From Proposition 1, we have that: $$W_2^2\big(\theta_\sharp (P_U)^\sharp \mu^d, \theta_\sharp (P_U)^\sharp \nu^d\big) \leq W_2^2\big(\theta_\sharp \mu^d, \theta_\sharp \nu^d\big)$$ $$W_2^2\big(\theta_\sharp \mu^d, \theta_\sharp \nu^d\big) \leq W_2^2\big(\theta_\sharp (P_U)^\sharp \mu^d, \theta_\sharp (P_U)^\sharp \nu^d\big) + 2 \big(m_2(U^\perp_\sharp \mu^d) + m_2(U^\perp_\sharp \nu^d)\big).$$ Take the expected value with respect to $\theta \sim \mathcal{U}(\mathbb{S}^{d-1})$, we complete the proof.

C6. Additional results in the empirical case

In practice, we have access to finite samples and a limited number of slices. We provide the following proposition to extend our results to this practical setting.

Proposition 3

Let $\mu^d, \nu^d \in \mathcal{P}(\mathbb{R}^d)$ satisfy Assumption~\ref{asp:lowdsupp}. Consider the empirical estimator $\widehat{ESSF}(L)$ defined as:

$

\widehat{ESSF}(L) = \frac{1}{L}\sum_{l=1}^L \| U^{\top} \theta_l^d \|^p,

$

where $\{\theta_l^d\}_{l=1}^L \overset{\text{i.i.d.}}{\sim} \mathcal{U}(\mathbb{S}^{d-1})$. Then:

• $\mathbb{E}[\widehat{ESSF}(L)]=\frac{C_k}{C_d}$ and $\text{Var}(\widehat{ESSF}(L))= \mathcal{O}(\frac{1}{L})$

• Let $\epsilon_L = \left| SW_p^p\left(\mu^d, \nu^d; \frac{1}{L}\sum_{l=1}^L \delta_{\theta_l^d}\right) - \widehat{ESSF}(L) \cdot SW_p^p\left(\mu^k, \nu^k; \frac{1}{L}\sum_{l=1}^L \delta_{\theta_l^k}\right) \right|$. Then $\epsilon_L \xrightarrow{\text{a.s.}} 0$ as $L \to \infty$

• Furthermore, there exists a constant $K > 0$ depending only on $\mu^d$ and $\nu^d$ such that for any $\delta > 0$, we have that $\mathbb{P}(\epsilon_L < \delta) \geq 1 - e^{-\delta^2L/K^2}$

Proof sketch for Proposition 3

• It is straight forward to prove $\|U^\top\theta_l^d\|^2\sim Beta(\frac{k}{2},\frac{d-k}{2})$. Thus $\hat{ESSF(L)}$ is essentially of sample mean of $L$ i.i.d Beta distribution variables. We can derive its expected value and variance.

• It is straight forward to derive

$

\epsilon_L \leq K\cdot \left|\frac{1}{L}\sum_{l=1}^L\left(|U^\top \theta_l|^p-\sum_{l'=1}^L|U^\top\theta_{l'}|^p\right)\right|

$

where $K=\max_{x,y}\|x-y\|^p$. Let's denote the sum term as $B_n$. By law of large numbers, with probability 1, $B_n\to 0$. Thus, $\epsilon_L\to 0$.

• It remains to show the convergence rate of $\epsilon_L$. Since each $\|U^\top\theta_l\|^p\in[0,1]$, for each $t>0$, by Hoeffding's we have

$

\mathbb{P}(|B_n|\ge \epsilon)\leq e^{-2\delta^2L}

$

Replacing $\delta$ by $\delta/K$, we have $\mathbb{P}(\epsilon_L\leq \delta)\ge 1-2 e^{\frac{2\delta^2L}{K^2}}$ and we complete the proof.

Proposition 4

Consider discrete measures $\mu=\sum_{i=1}^nq_i^1\delta_{x_i},\nu=\sum_{j=1}^mq_i^2\delta_{y_j}$. Define the empirical gradient error for each $x_i$:

$\delta_d = \sum_{l=1}^L\delta_l^d$

$\delta_k = \sum_{l=1}^L\delta_l^k$

$G_d = \nabla_{x_i} SW_p^p(\hat{\mu}_d,\hat{\nu}_d;\delta_d)$

$G_k = \nabla_{x_i} SW_p^p(\hat{\mu}_k,\hat{\nu}_k;\delta_k)$

$\epsilon_L(x_i) = \|G_d - \widehat{ESSF}(L) \cdot G_k\|$

Then the following holds

• $\epsilon_L(x_i) \xrightarrow{\mathbb{P}} 0$ as $L \to \infty$

• $\mathbb{P}(\|\epsilon_L(x_i)\| \leq \epsilon) \geq 1-2e^{-\epsilon^2L/(pq^1_iK)^2}$, where $K=\max_{x_i,y_j}\|x_i-y_j\|^{p-1}<\infty$.

Proof sketch for Proposition 4

• Similar to previous proof, we can show $\|\epsilon_L(x_i)\|\leq pq_1^1 K |B_L|$ where $K=\max_{x_i,y_j}\|x_i-y_j\|^p-1$, $B_L=\frac{1}{L}\sum_{l=1}^L(\|U^\top \theta_l\|^p-\frac{1}{L}\sum_{l=1}^L\|U^\top \theta_l\|^p)$. By law of large number, with probability 1, $B_L\to 0$ as $L\to \infty$. Thus, $\epsilon_L\to 0$.
• Since $\|U^\top \theta\|\in[0,1]$, by Hoeffding inequality, $\mathbb{P}(|B_L|\ge \epsilon)\leq 2e^{-\epsilon^2/L},$ thus we prove the inequality in the statement.

C5. Clarifying the global scaling factor

Recall that in Proposition 4.4 in the paper, we have that: $W_p^p(\theta^{d^\sharp} \mu^d, \theta^{d^\sharp} \nu^d) = W_p^p\big((U^\top \theta^d)^\sharp \mu^k, (U^\top \theta^d)^\sharp \nu^k\big) = \|U^\top \theta^d\|^p W_p^p(\theta^{k^\sharp} \mu^k, \theta^{k^\sharp} \nu^k).$

where $\theta^k=\frac{U^\top\theta^d}{\|U^\top\theta^d\|}$ with convention $\theta^k=0_k$ if $\|U^\top \theta^d\|=0$.

If we integrate both sides over $\theta^d \in \mathbb{S}^{d-1}$ with respect to the uniform measure $\sigma(\theta^d)$, then we obtain

$

SW_p^p(\mu^d, \nu^d) = \int_{\mathbb{S}^{d-1}} W_p^p(\theta^d_\sharp \mu^d, \theta^d_\sharp \nu^d) d\sigma(\theta^d) \ = \int_{\mathbb{S}^{d-1}} | U^\top \theta^d |^p W_p^p\left( \theta^k_\sharp \mu^k, \theta^k_\sharp \nu^k \right) d\sigma(\theta^d). $

Note that $\theta^k$ depends on $\theta^d$, and the distribution of $\theta^k$ induced by $\theta^d \sim \sigma$ is uniform over $S^{k-1}$. We introduce the change of variables from $\theta^d$ to $\theta^k$ and express the integral in terms of $\theta^k$:

$

SW_p^p(\mu^d, \nu^d) = \int_{\mathbb{S}^{k-1}} W_p^p\left( \theta^k_{\sharp} \mu^k, \theta^k_{\sharp} \nu^k \right) \left( \int_{\theta^d: \frac{U^\top \theta^d}{| U^\top \theta^d |} = \theta^k} | U^\top \theta^d |^p , d\sigma(\theta^d|\theta^k) \right) dT_\sharp \sigma(\theta^k),

$

where $\sigma(\cdot|\theta^k)$ is the conditional distribution of $\theta^d$ under $\theta^k$, and $T:x\mapsto \frac{U^\top x}{\|U^\top x\|}$ is the mapping from $\theta^d$ to $\theta^k$.

The inner integral over $\theta^d$ can be evaluated as a scaling factor $C_{d,k}$ dependent on $\sigma$, $\theta^k$, $U$. When $\sigma=\mathcal{U}(\mathbb{S}^{d-1})$, $C_{d,k}$ is invariant for all $\theta^k$.

Substituting back into and let $\sigma_k=T_\sharp\sigma=\mathcal{U}(\mathbb{S}^{k-1})$ denote the distribution of $\theta^k$, we obtain

$

SW_p^p(\mu^d, \nu^d) = C_{d,k} \int_{\mathbb{S}^{k-1}} W_p^p\left( \theta^k_{\sharp} \mu^k, \theta^k_{\sharp} \nu^k \right) , d\sigma_k(\theta^k).

$

Since $\sigma_k(\theta^k)$ integrates to 1 over $S^{k-1}$, and $W_p^p\left( \theta^k_{\sharp} \mu^k, \theta^k_{\sharp} \nu^k \right)$ is integrated over all $\theta^k$, we can express the right-hand side as $C_{d,k} \cdot SW_p^p(\mu^k, \nu^k; \sigma_k)$. Intuitively speaking, this means the loss of information is due to an implicit constant factor on $SW_p^p(\mu^d, \nu^d)$, which we denote as the Effective Subspace Scaling Factor (ESSF). Thus, rescaling the one-dimensional Wasserstein for all slices becomes multiplying the SWD by the reciprocal of the ESSF. The main theorem 4.5 in the paper makes this connection explicit.