Thank you for your valuable comments!

Re Weaknesses:

Writing style and presentation:

Thanks for your suggestion! We will move some footnotes into the body of the paper in the revised version. We will also add more clarification of Figure 3.

Methodology:

We will elaborate more on the annotator’s sign language background in revisions; we didn’t go into too much detail to avoid anonymity concerns. In brief: formal instruction in some (primarily ASL with ongoing usage), self-study in others. While more annotators can always be beneficial, the annotations to produce the dataset were relatively coarse and permissive (both in terms of the blanket effect on channels, and the criteria for inclusion in the dataset). The salient potential bias in the annotation procedure, as we acknowledge in Limitations, is that it reduces the number of unique signers because it favors large channels.

We chose to build off the FLEURS-ASL baselines because their random crop-based training made sense for YouTube-SL-25’s videos, which we expect to be slightly more misaligned than even YouTube-ASL. You are right that the additional tasks in the mixture were less important, but we wanted to show how you could straightforwardly extend such a multitask model to support additional language pairs and sign language identification. We briefly reexplain the task format for context but don’t view the details as especially important. This method is still “simple” in the sense of the YouTube-ASL baselines (forgoing inductive bias that won’t matter with scale), and the ablations are all about demonstrating ability to leverage data scale.

Are there particular metrics you would like to see besides BLEU and BLEURT? We picked the best classical metric and the best neural metric commonly used by other sign language and machine translation works.

Relation to previous datasets:

We initially focused on datasets that were the largest for each language, but we will add a reference to PHOENIX on line 248 (after “curate content from a small number of content creators”) in recognition of its historical significance.

Relation to previous modeling techniques:

We will add some motivation for MediaPipe Holistic in revisions: it is a form of anonymization (which is desirable when training on mined web data with human appearance), and it reduces the complexity/potential bias in handling diverse visual inputs because it was trained and evaluated independently for fairness. (Of course, this comes at the expense of introducing a frozen bottleneck.) But as mentioned above we do not view the details of the baseline setup as being key decision points; they are just inherited from reasonable prior work to host the data ablations.

Quality:

We will replace the parenthetical with “boundaries in sign language phrases (which can often be identified based on pauses and nonmanual signals [cite])” with a reference to “Seeing sentence boundaries” by Fenlon et al. Qualitatively, when there is enough space between signed sentences it is easy to see whether the captions are well-aligned—and alignment tends to remain consistent across a video/channel, so you only need to find a few easy points to check.

Re Questions:

Figures:

• We can expand “w.p.” to “with probability”.

Main text:

• We do not have an exact number of unique signers, just a lower bound based on the number of unique channels (described on line 240), so we think mentioning a specific number here would be unhelpful. We mention the specific number of languages (55) in the next paragraph (lines 82-87), contextualized with the fact that most of these languages are very poorly represented.
• We will add detail about the first author's sign language experience.
• Learner competence: This was indeed evaluated by the one annotator based on heuristics, relatively permissively. For the dataset’s primary use case of sign language translation (=understanding), it is not especially harmful to include learners since (to the extent that they sign in a distinguishable way) it just means models will be able to understand people with poor signing proficiency. As we mention on lines 459-462, the wide variety in YouTube-SL-25 poses additional challenges for sign language generation (you want to train on diverse data but generate a particular kind of signing), but not in a way that is unique with respect to learners or that would merit exclusion from the dataset.

Thank you for your valuable comments!

Re Weaknesses:

1. We believe this is a misunderstanding. The contributions of this paper are primarily with respect to the dataset (the final artifact) and new method of constructing it (which uses less/less skilled human annotation). Secondarily, our baselines contribute some new kinds of results (e.g. multilingual transfer at scale, sign language identification). We do not claim particular modeling contributions. For this reason we selected the primary area “datasets and benchmarks”.

2+3. Unfortunately we are currently unable to open source anything beyond the dataset, either because it is proprietary or due to complications of working with YouTube data. This method of data release (video ids only) is standard/necessary for YouTube corpora in order to automatically respect video deletions. We understand that these answers may be unsatisfying, but note that several works have already used YouTube-ASL, in addition to the many works that use YouTube or other url-hosted datasets, with minimal data loss over time and benefit to the research community. We expect the same to be true of YouTube-SL-25.

Thank you for your valuable comments!

Re Weaknesses:

1. On lines 423-424 and 465-469 we explain that the limited selection of benchmarks across sign languages is due to limited availability (including licenses). We highlight this as an opportunity for future work. We believe that our baselines compellingly show the benefit of YouTube-SL-25 for multilingual transfer in high- and low-resource sign languages.
2. While more annotators can always be beneficial, the annotations to produce the dataset were relatively coarse and permissive (both in terms of the blanket effect on channels, and the criteria for inclusion in the dataset). The salient potential bias in the annotation procedure, as we acknowledge in Limitations, is that it reduces the number of unique signers because it favors large channels.
3. We have shown in experiments that YouTube-SL-25 substantially improves downstream performance, demonstrating the knowledge and signal from YouTube-SL-25 clearly outweighing its noise. We also welcome efforts by the community to further refine YouTube-SL-25 or future datasets in lines 93-98, and acknowledge this as a limitation throughout the paper.
4. Additional attributes are always desirable for demographic analysis (especially age could be interesting), but we believe we already met a high standard for the field by predicting skin tone and perceived gender presentation across the entire dataset using classifiers. On lines 491-497 we describe how this dataset transparency still does not guarantee resulting models behave equally for subgroups, and that these factors should receive targeted evaluation.
5. Sign language generation is an interesting task, but it is out of scope for this work. Beyond the general immaturity of sign language generation methods, as we mention on lines 459-462 diverse web data poses additional challenges which should be a research topic in themselves. We provide sign language translation and identification baselines that we believe already demonstrate the value of YouTube-SL-25.

Re Questions:

Data Curation and Annotation:

1. We explain on lines 82-87 that this threshold is just about the name of the dataset, YouTube-SL-25 instead of YouTube-SL-55, to avoid overstating the functional diversity of the dataset. It does not affect actual inclusion of languages in the dataset, so there are no tradeoffs.
2. Specific challenges, heuristics, and observations from the annotation process are described in lines 160-215. For example, in footnote 3 we observe that developing countries tend to have less content or less well-produced content.
3. We avoided elaborating too much on the annotator’s sign language background to avoid breaking anonymity. We can provide more detail in the final paper, but overall do not see annotator background or bias as especially significant given the coarseness and permissiveness of the annotations.

Dataset Analysis and Evaluation:

1. See Weakness #4.
2. Discourse-level features are an interesting topic for future work, but sentence-level metrics still effectively show gaps between our baseline ablations. Note that on 463-464 we acknowledge the limitation that—while our models support discourse-level and other types of tasks—we do not evaluate these capabilities due to lack of uniform benchmarks.
3. See Weakness #5.

Generalizability and Future Work:

1. On lines 95-97 we state that we expect the dataset to be a useful pretraining resource for downstream sign language understanding tasks generally (including caption alignment), and on 459-462 for sign language generation.
2. We describe on lines 97-98 that we expect noisier data could be filtered (using models trained on YouTube-SL-25) to construct the next wave of sign language datasets, on 255-256 that sources of data like Bible translations or national broadcasts could help with language imbalance, and on 465-469 that there is a need for more multilingual sign language benchmarks.