To FP8 and Back Again: Quantifying Reduced Precision Effects on LLM Training Stability

Thank you for carefully critiquing our work and methodology. We address the weaknesses mentioned and hope our response is helpful.

Weaknesses:

We thank the reviewer for the survey of FP8 training. Indeed, the two papers mentioned are most relevant to this work. The first paper from Microsoft proposed the MS-AMP method, and we analyze the results of applying MS-AMP to the nanoGPT codebase in Figure 6 of our work. MS-AMP O1 in our work is equivalent to the FP8 #2b method in the MS-AMP paper.

The second paper from Intel was published more recently, and we were thus unable to apply their Smooth-SwiGLU and FP8 optimizer state conversion methods.

The work by Intel, however, supports our claim that FP8 training is not equivalent to BF16 training rather than dispelling the concerns that we raise. Figure 2 of their work shows that although FP8 initially has no conversion issues compared to BF16, a divergence emerges after 200B tokens of training for Llama 7B models. Although we could not conduct such extensive experiments due to a lack of resources, this is precisely the type of issue that our work aimed to raise awareness of. The authors propose adding scaling factors to the SwiGLU activation to prevent divergence.

Our work aims to shift the focus of FP8 training research away from whether FP8 training can be made to work from an engineering perspective and towards discussions of whether FP8 is worth using from a financial perspective. The main advantage of using FP8 is that it reduces training costs by improving throughput. If the quality of the model deteriorates or requires additional training to compensate, the financial incentives to deploy FP8 are cast into doubt. We do not ask, “Can it be done?” but instead, “At what cost?”.

Production environments often cannot use thoroughly cleaned and filtered datasets for which the optimal learning rates have already been discovered for given model sizes. For new types of data, such as video, standards for how data is to be curated are yet to be established as rigorously as for text data. Even for text data, training on low-resource languages or specialized fields may require using all available data, which can significantly increase training instability. We seek to know how new methods will perform on rough, untrodden paths, not only on well-paved roads.

Questions:

1. Thank you for raising this point. The value of the sharpness indicator as an indicator of training instability is indeed an important issue. In Figure 2 of our work, we show the results for visualizing loss landscape sharpness using the method proposed by Li et al. We found no signs of sharpness when using this method, even when the model was undergoing divergence. As discussed in the reply to all reviewers, we sought additional verification of our proposed indicator by measuring the loss landscape sharpness when a model (Llama 7B) was initially trained with reduced precision (E8M3) but later trained with BF16. We found that the sharpness decreased, strengthening our case that it is a useful indicator of training instability.

2. For most of our experiments, we only clip the mantissa bits and do not add scaling factors. For the experiment in Figure 10 of the appendix, we removed an exponent bit and found that the effect was much greater than when we removed a mantissa bit. As FP16 with scaling is possible, we agree that adding scaling to the exponent clipping would change the result. However, the experiments with intermediate bits aimed to see the effects of increasing instability in higher resolution. We wished to see incrementally increasing instability effects at intermediate bits instead of simply comparing 16-bit and 8-bit schemes. Analyzing reduced exponent bits with scaling factors would be a valuable addition to this work, and we hope to explore this direction in the future.

3. We apply a bitmask to the mantissa to remove the least significant bits, equivalent to rounding down the mantissa. While this may introduce a slight bias, the effect is smaller than the rounding error introduced for FP8 types, which have 3-bit and 2-bit mantissa widths for E4M3 and E5M2, respectively.

4. Thank you for this important question. In exploring the answer, we discovered that data quality was the likely cause of the divergence shown in Figure 6 between baseline BF16 and MS-AMP FP8 training. As noted in the reply to all reviewers, when the FineWeb Edu dataset, a clean dataset with high educational content, was used for training, the differences in training loss disappeared. As discussed in the weaknesses section, we consider this a validation of our argument that FP8 is not equivalent to BF16. The Intel results show divergence even for standard FP8 training only after training on 200B tokens on a Llama 7B model, which is beyond our computational resources. Also, the proposed solution involves architectural modifications to reduce the effects caused by correlations in the SwiGLU weights.

Thank you for your constructive critique of our work. Due to the feedback, we could conceive new experiments to explore aspects of our work in greater depth. We hope to address the raised issues in the following response.

Weaknesses:

The two areas for improvement mentioned in the review were (1) concerns about whether the reduced-precision issues would persist in larger models and (2) why the results in this work diverged from those from the MS-AMP paper.

For the first point, our experiments using Llama 7B models suggest that sensitivities to lower precisions persist even in larger models. As we can see even Llama 7B models show increased training instability when using reduced precision, it is not implausible that the issue persists even in larger models. In addition, although the largest LLMs may reach hundreds of billions or even trillions of parameters, there has been a trend to utilize smaller models of only a few billion parameters on larger datasets, especially for limited tasks. We believe that our work may also inform researchers conducting training work in such areas.

For the second point, we found that the critical issue was the data quality involved in training. In our experiments, we used the OpenWebText dataset from the nanoGPT repository to reproduce the results for MS-AMP, the results of which are shown in Figure 6 of our work. However, Section A.3 of the Appendix in the MS-AMP paper notes that the pretraining data used for experiments consisted of a large custom dataset with extensive filtering and curation, which was not made publicly available.

As mentioned in the comment to all reviewers, we suspected that data quality might be the cause of the divergence and attempted training on a subsample of the FineWeb Edu dataset, which was also extensively filtered and curated for educational content by training on smaller models. We found that the difference between BF16 training and MS-AMP training disappeared after changing the dataset, indicating that the difference in convergence was caused by the lower quality of the OpenWebText dataset.

While this finding may appear to weaken our argument, we argue instead that it strengthens it. In actual training environments, we cannot simply take extensively verified open-source datasets and train them on models for which the optimal learning rate is already known. Low-resource languages may require using any available data, while it is unclear how new domains, such as robotic motion, will interact with the other data during training. We do not argue that FP8 training of LLMs is impossible. However, we must consider that the primary motivation for using FP8 over BF16 is to reduce costs. If costs arising from other sources outweigh the costs saved from the increased throughput, there is no justification for using FP8 over BF16. Considering that the speed boost obtained by applying FP8 to training is, at best, approximately 50% faster, we believe that works studying the use of FP8 for LLM pretraining should address how their methods perform when stepping out of well-trodden paths. If nothing else, we hope to move the conversation from discussing how FP8 training methods could be made to perform well on some well-known tasks to focusing on cost-effectiveness.

Questions:

1. We conduct our experiments on two fronts. First, in Figure 6 of our work, we show the results of applying MS-AMP to nanoGPT training. However, because we wished to gain a more fine-grained understanding of the destabilizing effects of precision reduction, we conducted our other experiments using clipped mantissa values on Llama models of various sizes.
2. Thank you for pointing out this crucial issue. As mentioned above, this question has helped us examine a new axis of data quality in our work. To answer the question, we used the datasets in the respective repositories. For the experiments with nanoGPT, we use OpenWebText, an open-source reproduction of the GPT-2 dataset. For the TinyLlama experiments, we use a mix of the SlimPajama and Starcoder datasets in a ratio of approximately 7:3. The number of tokens per training step can be calculated using the equation $training\ step \times sequence\ length\times global\ batch\ size$.

Thank you for your thoughtful review of our work. Below, we aim to address the points that have been mentioned.

Weaknesses:

The two points mentioned as weaknesses of our work are (1) that it needs to be compared with alternative methods of measuring training divergence and (2) that hyper-parameters other than the learning rate and model size should also be searched.

1. Regarding the first point, we initially attempted to use the loss landscape sharpness visualization technique proposed by Li et al. However, as shown in Figure 2 of our work, we found that the method was unsuitable for our work as the visualized loss landscape surface continued to be smooth even when the model was in the process of divergence. While we are unaware of other methods that measure training instabilities in autoregressive models, the experiment in our response to all reviewers supports our claim that the sharpness indicator proposed in this work can also detect decreases in training instability when a higher precision is used.

2. Regarding the second point, we explored the effect of using higher-quality data in our additional experiments. Although the training losses for MS-AMP FP8 and BF16 diverge for the OpenWebText dataset used for training nanoGPT, using a higher-quality dataset such as FineWeb Edu hides the difference. This finding supports our argument that FP8 introduces subtle instabilities in LLM training, weakening the financial case for deploying it.

Questions:

1. Compared to Keskar et al., our method uses at least millions, possibly billions, of times fewer resources. The reasons are twofold. First, because we use only the last token of the autoregressive prediction, our method skips computation on the rest of the sequence. Second, because we apply noise to the output logits instead of the model’s input embeddings, we need not perform a model forward pass for each iteration of the L-BFGS-B algorithm, requiring only a pass through the cross-entropy loss layer.

2. Regarding this point, we focused on the learning rate as the main hyper-parameter because the learning rate is known to be a key hyper-parameter. Model size was also tested, but to a lesser extent, because increased model size requires greater resources. In addition to the learning rate, we attempt additional experiments on different datasets for this review. In these, we train MS-AMP models on a sample of the FineWeb Edu dataset, which has been extensively cleaned and filtered for “educational content”. We believe that these experiments show that data quality can also expose the reduced stability of FP8.

3. We include results for our naïve implementation of exponent clipping in Figure 10 of the Appendix.

4. We thank the reviewer for reading our work in depth. Further investigation in this area would be of interest. Some of our preliminary results are positive. For example, excluding the first two transformer blocks of nanoGPT models from low precision prevents them from collapsing. However, this trick did not affect Llama models. Also, although we did not try experiments where the model was trained in high precision first and later used low precision, we did attempt to train the model first in low precision, then revert to high precision. The results are shown in the table in our response to all reviewers.

Thank you for your thoughtful feedback on our work. We hope that our reply addresses the points raised by the reviewer.

Weaknesses:

1. As noted in our limitations section, we wholeheartedly agree that training loss alone is insufficient to determine a model’s quality. Unfortunately, because we primarily analyze our models in the early stages of training, making meaningful comparisons on real-world benchmarks is challenging. Models attain meaningful scores only when they have been trained more extensively. This will be an area of further research. However, a model with a divergent loss function will not obtain a good score on downstream benchmarks, making good training loss conversion a necessary, if not sufficient, condition for LLM quality.

2. Exploring reduced exponent bits with common scaling factors would be a significant addition to our research. This would involve the analysis of algorithms to decide the granularity and the values of the scaling factors, adding another dimension of exploration. However, for simplicity of analysis, our experiments mainly analyze the effects of reducing mantissa bits from BF16, sidestepping issues that may arise from the reduced exponent range compared to FP32. Because of this, scaling problems do not affect our experiments on mantissa clipping. We hope that our experiments with MS-AMP, which implements scaling and other techniques, will be informative on this front.

3. Due to resource considerations, we were limited to models of 7B parameters or lower. However, recent trends in LLM training have enabled even relatively smaller models, such as Llama v3.1 8B, to greatly improve in capabilities. The latest generation of Llama v3 models have increased the number of training tokens applied to Llama 8B by orders of magnitude, finding that scaling laws still hold. Because of high inference costs, much research has gone into improving the performance of smaller models. Many applications, especially in restricted domains, are well served by these smaller models. As such, our work can benefit real-world use cases as well as serve as the basis for work on larger models.

4. Finally, we note the absence of FP16 in recent LLM training. Although early models such as OPT used FP16 with scaling factors for model training, nearly all LLMs made public within the last two years have used mixed-precision BF16 for training despite BF16 having three fewer mantissa bits and thus providing lower resolution. Though this only constitutes circumstantial evidence, the transition from FP16 to BF16, despite the former’s superior resolution, strongly suggests that large models are still prone to instabilities, even when scaling is applied. When considering that both E4M3 and E5M2 have less expressive capacity than FP16, we believe that a rigorous investigation of training robustness is due.

Questions:

1. The considerable divergence in loss curves for FP8 training that arises depending on whether the LM head is included or excluded indicates the importance of even small errors at the output logits. This is in line with previous work on post-training quantization, which has found that the embedding layer and the final projection layer, which are sometimes tied together, are highly sensitive to quantization.

2. In Figure 6 of our work, we compare applying MS-AMP to the model excluding the LM head in red and show the baseline BF16 results in blue. As indicated by the black horizontal line, the training loss of the model trained using MS-AMP does not converge to the same level, even after 6 times the number of training steps. This indicates, at least for the conditions examined in Figure 6, that FP8 is less capable of training convergence than BF16. In additional experiments conducted for the review, we find that when a “cleaner” dataset, namely the FineWeb Edu dataset, is used, this divergence disappears, strengthening our claim that FP8 narrows the hyper-parameter space where stable training can occur.

3. Although we were unable to find specific threshold values where model collapse suddenly occurs, we were able to confirm that the sharpness is a valid indicator of training instability. In the reply to all reviewers, we find that the loss landscape sharpness decreases when we train a model that has already been trained in low precision in the original BF16. We summarize the results in the attached table.

4. The experiments for E8M5 help demonstrate one of the conditions that can reveal the hidden instability of reduced-precision training, namely, increased sensitivity to learning rate changes. Although it may be possible to find the optimal learning rate for each model and dataset, we argue that reduced-precision methods, even if they work well under best-case scenarios, are more sensitive to the choice of hyper-parameters, which can be very expensive to find for LLMs.