Thank you for your insightful reviews! We appreciate you find our work interesting and results potential.
We hope our responses can address your concerns and improve your rating.

Q1: comparison to more models

We emphasize we have included the newest baselines: HyperMiner(NeurIPS2022), ProGBN(ICML2023), ECRTM(ICML2023), GINopic(NAACL2024).
Here we report the comparisons to CTMneg and CEDC:

We see although CTMneg is better than CombinedTM, FASTopic still outperforms them.
Thank you for mentioning these important work. We have updated the paper and cited them.

Q2: include more evaluation metrics

Here we additionally report the results of Word Embedding Coherence (WEC) and Inversed Rank-Biased Overlap (IRBO) from Adhya_2023. WEC measures coherence with pretrained word embeddings, and IRBO measures diversity.

The above shows our model also performs better on these metrics. Thank you for the suggestion. We have added these results to the paper.

Q3: the number of topics for BERTopic

We explain we set the number of topics of BERTopic to be the same as the other models for fair comparisons.
Each model produces 50 topics in Table_1. Otherwise, it'll be unfair if BERTopic reduces the number of topics to 10.

Q4: how is the training time measured?

We measure the training time from when the dataset is loaded until the training is finished. The training time includes loading document embeddings.

We clarify we use the same document embeddings for each model when applicable for fair comparisons. As mentioned in Section_D Line_574, we use all-MiniLM-L6-v2 for document embeddings by default.

Q5: why FASTopic is faster than BERTopic

As mentioned in Appendix_D Line_578, we train FASTopic with 200 epochs.
We break down the training time (seconds) as

We see BERTopic has to reduce embedding dimensionality, cluster embeddings, and compute word weights.
In contrast, our model enjoys faster training. This is because it employs Sinkhorn's algorithm to solve optimal transport, which is quite fast as proven by previous studies (Cuturi_2013, Genevay_2019). Moreover, its objective is simple and straightforward, optimizing only four parameters: topic and word embeddings and their weight vectors (Eq(8)). This avoids the complicated encoders and decoders of VAE-based methods.
Previous ECRTM also uses optimal transport, but its objective based on the complicated encoder and decoder slows it down.

Thank you for the question. We've added more explanations to the paper.

Q6: the number of hyperparameters in FASTopic

We summarize the hyperparameters as follows:

1. FASTopic has hyperparameters for Sinkhorn's algorithm $\varepsilon_{1}$ and $\varepsilon_{2}$ in Eq(5,6) and the temperature in Eq(9).
2. VAE-based models (CombinedTM, ProGBN, HyperMiner, ECRTM, GINopic) need more hyperparameters to set their encoders, decoders (dimensions, number of layers, and dropout), and prior distributions (Gaussian or Dirichlet). Some need more hyperparameters, like the hierarchy of ProGBN and HyperMiner, the regularization weight of ECRTM, and the graph neural networks of GINopic.
3. BERTopic has hyperparameters to set its clustering and dimension reduction modules, like the number of neighbors and components of UMAP; the min cluster size, min samples, metrics of HDBSCAN.

We explain the fewer hyperparameters of our model result from its extremely simplified structure.
We don't use the traditional VAE with complicated encoders and decoders or the complex dimension reduction and clustering modules as BERTopic. Our model only includes three kinds of embeddings and the embedding transport plans between them.

Thank you for the comment. We've added more discussions to the paper.

Q7: cluster the word embeddings

We clarify our method already works as clustering the word embeddings.
Figure_2(c) illustrates we can view our ETP as a clustering process: topic embeddings as cluster centers, word embeddings as samples, and the semantic relations between them as cluster assignments. We refine these assignments during learning through objective Eq(8).

Q8: take longer time to load high-resolution figures

Thank you for your comment. We've updated the paper with compressed figures.

Thank you for your comments! We're glad that you believe our paper is well-written, our experiments are comprehensive, and our model is straightforward and fresh.
We hope our responses can address your concerns and improve your rating.

Q1: how document embeddings affect the method

Thank you for the question. We clarify that we have experimented with different document embeddings in Appendix_F (Table_10 and 11).   Generally, higher-quality document embeddings bring about better topic modeling performance.
We explain that relying on pretrained document embeddings has become prevalent in topic modeling now. This is because we can easily access plentiful high-quality document embeddings, such as Sentence-Transformers which provides document embeddings for 50+ languages.

Q2: why use pretrained document embeddings and what if train them together

We explain that pretrained document embeddings contain abundant features, and using them is a common practice for topic modeling, such as BERTopic, CombinedTM, and CTMneg. We don't need to retrain an embedding model, which saves us lots of effort.
As mentioned in Section_3.4 Line_183, training the document embedding model together may lead to over-fitting problems, because our target datasets are regularly smaller than their pretraining datasets. This also greatly increases the training time and computational cost. Due to these reasons, the above early studies don't train document embeddings either.

Q3: running speed performance on CPU

Here we report the running time (seconds) of BERTopic and our model, including training (Train) and inference (Infer):

We see that our model is slightly faster than BERTopic on training and has much shorter inference time. This is because BERTopic has to extract and compare the n-grams in documents for inference, while our model directly uses the fast matrix calculations in Eq(9) for inference.
The other performance (like topic quality) is the same on CPU, since the training process remains unchanged.

Thank you for the question. We've added these results to the paper.

Q4: hyperparameter sensitivity of the model

Here we report the results by varying the main hyperparameters $\varepsilon_{1}$ and $\varepsilon_{2}$ in Eq(5, 6):

We see that the performance remains generally stable. Thank you for the question. We have added these results to the paper.

Q5: the line thickness in Figure 2

We vary the line thickness to indicate the values of semantic relations.
Thank you for the comment. We have added this to the Figure 2 caption.

Q6: the positions of some tables

Thank you for the suggestion. We have updated their positions in the paper.

Thank you for your feedback! We're happy that you appreciate our clear writing, effective method, and extensive experiments.
We sincerely hope our responses can address your concerns and improve your rating.

Q1: difference between ETP and the ECR of ECRTM (Wu_2023)

We clarify that the ECR of ECRTM is NOT an alternative to our ETP for ablation study.
The ECR (embedding clustering regularization) of ECRTM only regularizes topic and word embeddings and does NOT model the semantic relations between them as topic-word distributions. ECRTM follows the traditional VAE to model topic-word and doc-topic distributions for topic modeling.
Instead of VAE, our method uses ETP to model the semantic relations as topic-word and doc-topic distributions for topic modeling. Thus, the previous ECR does not model the semantic relations as our ETP, and it is NOT an alternative to our ETP for ablation study.
Besides, only the semantic relations between documents and topics or only between topics and words cannot do topic modeling, because we need both of them for reconstruction in Eq(1).

Thank you for the question. We've added more clarifications in the related work.

Q2: performance on long-text corpora

We clarify that we have experimented on long-context corpora, NeurIPS, ACL, and Wikitext-103.
As reported in Appendix_B Table_7, they are long academic publications or Wikipedia articles, ranging from 1k to 2k words. We have shown that our model works well on these long-context corpora in Section_4.2, 4.3, and 4.4.

Q3: showcase comparisons to ECRTM

We clarify that we have showcased the comparisons to ECRTM on topic words and doc-topic distributions (Section_4.2), running speed (Section_4.4), transferability ( Section_4.5), and adaptivity (Section_4.6).

Here we illustrate some cases of topic words.

We see ECRTM mixes the topics of airlines, software, and typhoon. In contrast, FASTopic produces more separated topics about them respectively.

Q4: Differences between TopicGPT and LDA

We explain that their main difference lies in how they define topics and understand documents.
LDA defines a topic as a word distribution, for example, a distribution [0.1, 0.2, 0.1 ...] over words [farmer, products, agricultural, ...]. Differently, TopicGPT defines a topic as a natural language description, like Agriculture: Discusses policies relating to agricultural practices and products....
Besides, LDA infers distributions over topics to understand documents, while TopicGPT classifies a document with topics as label space by prompting. TopicGPT reaches higher interpretability, but cannot give precise distributions for downstream tasks.

Thank you for the question. We have updated the paper with more explanations.

Q5: does a large number of topics affect FASTopic?

We clarify that we've reported in Table 4 and 5 the results under large numbers of topics (K=75, 100, ... 200). Our model remains high-performance with large numbers of topics, especially on topic diversity.

We clarify that in Line_132, we intend to show using parameterized softmax leads to the relation bias issue even with a small number of topics, as supported by Table_9. This motivates us to propose the new ETP method.

Dear Reviewer FTng,

Thank you sincerely for your previous helpful reviews!
We mention that we've submitted our responses, including clarifications on our differences from ECRTM and our performance on long-text corpora and large numbers of topics.
We hope our responses can address your concerns. We are looking forward to your insightful feedback!

Thank you.

Best,
Authors

Thank you for your helpful feedback!  We're glad that you appreciate our well-written paper, neat method, and extensive experiments.
We sincerely hope our responses can address your concerns and improve your rating.

Q1: comparison to earlier NSTM (2022) and WeTe (2023)

Thank you for your comment. We explain that we've included the newest baselines: ProGBN (ICML 2023), ECRTM (ICML 2023), and GINopic (NAACL 2024).
Here we additionally report the results of NSTM and WeTe:

We see our model can surpass them too.

Q2: why solving the optimal transport is so fast

We explain that we employ the fast Sinkhorn's algorithm to solve the optimal transport and our objective is much simpler.
As mentioned in Section_3.3 and Appendix_D, the Sinkhorn's algorithm is fast and suited to the execution of GPU (Cuturi_2013, Peyré_2019, Genevay_2019). Besides, as indicated in Section_3.4, our objective is straightforward without complicated networks. It only invovles four parameters: topic and word embeddings and their weight vectors (Eq(8)). Previous models, WeTe and ECRTM, also solve the optimal transport, but their objectives rely on complicated encoders and decoders, which slows them down.
Thank you for the question. We've added more explanations in the paper.

Q3: is the comparison fair due to document embeddings?

We clarify that our comparisons are fair since we've included the baselines using the same document embeddings.
BERTopic uses document embeddings for clustering; CombinedTM uses document embeddings as input features. These baselines use exactly the same document embeddings as our method for fair comparisons. Some other models originally cannot incorporate document embeddings but use pretrained word embeddings, like ProGBN, HyperMiner, and ECRTM.

Q4: discuss LLM-based topic models

LLM-based topic models are very promising such as TopicGPT, but we explain they currently have two main limitations:

1. LLM-based topic models require more resources. They need to input each document as prompts to LLMs. This is time-consuming and computationally intensive, especially when handling large-scale datasets.
2. LLM-based topic models cannot produce precise distributions for topics and documents. They can only use natural languages to describe topics and documents.

As shown in the paper, our model has a fast running speed (Figure_1) and also provides precise distributions (Section_3.3).
Thank you for the question. We have added more discussions in the related work.