BOWLL: A DECEPTIVELY SIMPLE OPEN WORLD LIFELONG LEARNER

We thank the reviewer for their assessment and appreciate the additional feedback on how to improve our paper. We have uploaded a revised version of our paper, that already incorporates the mentioned suggestions and provides clarifications with respect to the remaining concerns and questions. For convenience, the changes and additions are highlighted in blue in the pdf. We have also posted a summary of all changes, spanning all reviewers’ feedback at the top. In the following we provide additional short statements and responses to the specific reviewer’s points.

W1 & W4: “further technical insights regarding the implementation and novelty”

We agree that more details were necessary. Whereas earlier we have just had a technical description of the OoD module (A.2), added appendices A.3 and A.4 now provide the full details for the active query and Deep Inversion. With respect to novelty, we emphasize that there exists no “GDUMB equivalent” as the default in open world learning. BOWLL provides this first baseline with a single easy to use common component, by identifying the three-fold use of batch-normalization. Both the OoD detector and the active query are novel by themselves.

W2: limitations and prospects of BOWLL

We agree that these are valuable. We now provide a brief summary in the conclusion and point to a new appendix A.10 for an extended account. In summary, we discuss the diagonal Gaussian covariance in batch-norm to render the easiest baseline, discuss potential caveats of Deep Inversion, and highlight how BOWLL can easily be used in other scenarios without supervision.

W3: “computational complexity of BOWLL”

We now provide an intuitive walkthrough of the computational aspect of BOWLL’s components at the end of new appendix A.4.. In summary, the main critical component is Deep Inversion, not the active query step, as DI requires many updates on a synthetic image before converging. However, we also alleviate this concern by proceeding to discuss the trade-off in light of our findings of figure 3, where BOWLL* (without Deep Inversion) shows only around a 3% accuracy decrease. We agree that it was helpful to expose this to prospective readers in the pdf.

W5: “”delve deeper into the continual learning step

In addition to the new appendix sections to provide more details for the individual components, we have revised the main body’s sections 3.3 and 3.4, where we now more precisely describe the use of the memory and the update step. This should improve clarity and remove potential prior ambiguity.

C1: “Balance data in memory buffer and pseudo-images”

We now describe that we only generate as many pseudo-images as in the memory buffer, a 1:1 balance that is technically a prospective hyper-parameter, discussed in appendix A.4 together with computational trade-offs.

C2: ”Formulation of equation 7 - priors”

We now state the intuition behind the priors in the main body and provide exhaustive definitions in appendix A.4. In short, The l2 term ensures that the range of values of the synthetic data remains in a reasonable range and the total variation (TV) prior ensures that pixels in a vicinity are related (to form entities in the specific case of images) - as motivated by cited prior work.

C3: “meaning of beta in LCA”

LCA sums over mini-batches of data trained on so far. In essence, it asses how quickly the model learns. For instance, beta=1 is one-shot learning. To clarify, we now provide an extended description of definitions and their interpretation of all metrics in appendix A.6.

C4: “Hasn’t the model used those discarded data”

We have revised the text to more precisely point out that data that has been rejected in the OoD step is never queried by the active learner and that data that is not in the memory buffer, is not included in optimization steps. As such, all data may have been passed through the model to figure out whether to include it, but only data that makes it into the memory buffer is ultimately trained on. Table 2 shows the number of data points that are being used in optimization.

C5: ”Relationship between open world learning and open world lifelong learning”

We realize we have not pointed this out precisely before, but open world learning only mentions that data is included or rejected, then queried/labelled and an incremental learning step is conducted. Often this leads to an interpretation that novel identified data is simply concatenated to existing datasets over time (inspired by active learning). To disambiguate this from a continual learning step (where we do not store old data), we have included the term lifelong. We have included a statement in the related work section, pointing to a new appendix A.1 with prior work’s definitions.

Finally, we thank the reviewer again for the feedback and invite them to confirm that our revised manuscript has adequately included their points.

We thank the reviewer for their assessment and appreciate the additional feedback on how to improve our paper. We have uploaded a revised version of our paper, that already incorporates the mentioned suggestions and provides clarifications with respect to the remaining concerns and questions. For convenience, the changes and additions are highlighted in blue in the pdf. We have also posted a summary of all changes, spanning all reviewers’ feedback at the top. In the following we provide additional short statements and responses to the specific reviewer’s points.

“Overstated contributions: paper focuses only on class-incremental learning in continual learning”

We believe there is a misconception of what the paper is doing and trying to achieve, but we taken the point as feedback to improve and have extended the writing to avoid this in the future. To first clarify, the paper is not tackling a purely continual learning problem. In fact, the experiments follow a logical sequence (see next point) to step by step lead the reader to open world learning, where the data does not only change over time, but may also include both irrelevant or corrupt data. Open world learning is a very challenging realistic scenario, that requires the model to predict whether a data point should be rejected/accepted, how informative it is, and how to continue training. Traditional continual learning only focuses on the last aspect. We are not investigating task-incremental learning because it assumes one knows “which task to predict” during inference through a provided label. This is a step back in the opposite direction of open world learning, where we don’t even assume that the data must be related at all to any of the tasks. As mentioned, we have however taken the feedback to improve the paper and now include a) a detailed introduction and definition to open world learning in appendix A.1; b) an improved experimental setup section to clarify our experiments (see also below) c) a more detailed description of the data and training details in appendix A.7; d) extended motivation and discussion of the active query’s formulation in appendix A.3 (active query).

“Paper claims can achieve OOD detection, active learning, continual learning but doesn’t adequately demonstrate the model’s superiority”

We believe the paper shows this already, as the experiments are meant to showcase individual components in a cascade of insights, first on learning speed, then on forgetting, and ultimately in the full open world learning experiment (third experiment). Especially the last, realistic scenario without curated data shows that BOWLL is the only method to succeed here, making it a very reasonable future baseline. However, we have taken the reviewer’s feedback to improve presentation and more rigorously highlight the individual aspects. Primarily we have improved the paper in the following:

• Each experimental subsection has been reworded to be more precise on the experimental take-away and now clearly states which component is supported by the found experimental evidence. In short:
  1. Table 2 focuses on highlighting that BOWLL achieves the same performance in the particular domain incremental setting as GDUMB, but does so with less data and massively faster learning speed (LCA) - highlighting the “active” part.
  2. Figures 2 and 3 support that BOWLL features less catastrophic forgetting through the way it maintains its memory - highlighting the “continual” part.
  3. Figures 4 highlights how BOWLL’s OoD detector is crucial to perform meaningful in the open scenario, where the scenario actually contains non-curated data.
• Based also on reviewer BmgJ’s feedback, we have included two additional baselines in the second and third experiment: experience replay (which visits all task’s data first and then keeps a memory as it proceeds) and Softmax based OoD detection, to highlight that BOWLL is a very meaningful baseline beyond GDUMB alone. The discussion and plots are updated respectively.

Comment: “Bayesian Network Statistics”

We are unsure if there may be a misunderstanding from our side on the reviewers remark “suggests leveraging Bayesian Network statistics”, so we would like to clarify that BN in our case is batch-normalization. While batch-normalization may certainly be framed in a Bayesian Network context and may even be sampled from, we want to emphasize that BOWLL is a simple, yet highly performative baseline that does not require a framing of Bayesian Neural Networks or their use. Any non-Bayesian neural network that contains batch-normalization is convertible into a BOWLL learner, which makes it a meaningful baseline for open world learning.

Finally, we thank the reviewer again for the feedback and invite them to confirm that our revised manuscript has adequately included their points. If the reviewer has any remaining questions, we are happy to clarify them further.

We thank the reviewer for their assessment and appreciate the additional feedback on how to improve our paper. We have uploaded a revised version of our paper, that already incorporates the mentioned suggestions and provides clarifications with respect to the remaining concerns and questions. For convenience, the changes and additions are highlighted in blue in the pdf. We have also posted a summary of all changes, spanning all reviewers’ feedback at the top. In the following we provide additional short statements and responses to the specific reviewer’s points.

“The main weakness that I see in the paper is the limitation of the experiments”

We have provided a more detailed description of our experiments and have clarified their purpose more thoroughly in the main body and appendix. In addition, we agree that there may not exist any SOTA methods designed specifically for open world learning, but that some other methods are interesting for readers to see in this context. To this end, we have run and added two further popular baselines to our experiments: experience replay (ER) and Softmax based OoD detection (according to the published works of Rollick et al and Hendrycks et al). Whereas GDUMB actively fills a memory buffer at random and thus is the closest in spirit to BOWLL, ER is a more intuitive comparison in experiment 2’s focus on forgetting, as ER first trains on all data before maintaining a random memory buffer over time. In figure 3, we can see that this training on all data does not lead to ER outperforming BOWLL. Naturally ER fails in the experiment with corrupted and OoD data. Here, Softmax based OoD detection provides a baseline for the open world learning scenario of the third experiment. We further note that appendix A.8 contains an ablation study, demonstrating the meaningfulness of BOWLL’s components and showing how BOWLL performs when individual parts are stripped away.

Question: could you provide more insights in the case of backward transfer

We agree with the reviewer that observing positive backward transfer is remarkable and interesting. In general, and particularly in the mentioned specific scenario of the sequence of multiple digit based datasets (table 2), we attribute the high performance to the fact that BOWLL contains an active learning query that balances data novelty with similarity (rather than for instance querying based on high entropy alone or doing so at random as in GDUMB). As such, new data that features some form of similarity is prioritized and thus contains partial information to further improve retrospectively. In line with the question of reviewer ELi2, we understand that the precise benefit of the exact active query formulation may not have been fully clear enough in the present manuscript. We have made the wording more precise and now provide an extended discussion of how and why the active query has been conceived in appendix A.3.

Finally, we thank the reviewer again for the feedback and invite them to confirm that our revised manuscript has adequately included their points.

We thank the reviewer for their assessment and appreciate the additional feedback on how to improve our paper. We have uploaded a revised version of our paper, that already incorporates the mentioned suggestions and provides clarifications with respect to the remaining concerns and questions. For convenience, the changes and additions are highlighted in blue in the pdf. We have also posted a summary of all changes, spanning all reviewers’ feedback at the top. In the following we provide additional short statements and responses to the specific reviewer’s points.

“How faithful are the generated images of Deep Inversion (a); Why not opt for feature synthesis, which seems natural and efficient given the maintained feature mean and variance (b). ”

a) We remark that the generated images do not necessarily need to be faithful in the sense of the original data, as the intend is to mitigate catastrophic forgetting only and not train on them from scratch or reconstructing natural images. However, the included L2 and TV priors (now explained better in the main body and in much more detail in appendix A.4) regulate the image values to stay within a meaningful range and ensure that pixels in a local vicinity retain similarity, in order to ensure that entities are formed rather than just producing any (adversarial) noise that fits the statistics.

b) We agree that feature synthesis is a highly intriguing direction, and have included a short discussion and motivation for our choice in the new appendix A.4. In essence, we believe that a strong baseline needs to both be performant but also simple to implement. Using synthesized data points from Deep Inversion matches this requirement, as it allows us to straightforward interleave these instances with the real memory buffer, populated by the active learner, in optimization. It is thus trivial to code and employ in any system that already contains batch-norm, the majority of current NNs. We do however agree that feature synthesis is intriguing and now explicitly point to it, referring also to works such as Pellegrini et al (IROS 2020) that have already demonstrated the efficacy of “latent replay” techniques. (We are happy to include more references here, if the reviewer has explicit pointers). We are excited for future work to pick up on this direction and showcase how such a more involved approach could beat the current BOWLL baseline, in similar spirit to how future work may extend the use of diagonal covariance towards full measures.

“For the active query, … is this the best way to strike a good tradeoff between exploration and similarity?”

Yes indeed, the reviewer’s intuition is correct and the design is about the tradeoff between exploration and similarity. We noticed that we could perhaps highlight this aspect more prominently and are now doing so in the main body, pointing further to a new appendix section A.3. Similar to the extended description of the conception of the OoD detector in A.2, we now describe the conception of the active query in much more detail. Here, we also point out the failure mode of excluding the similarity term. That is, informativeness gauged by the entropy term alone might yield highly novel data that is unrelated to the task or undesired (e.g. very noisy, perturbed, corrupted or fully unrelated data). The memory replacement step follows a similar logic. At the end, we now also note that there could technically be a more elaborate weighting of the exploration and similarity terms, but that we did not introduce respective weights to analyze this option further.

Finally, we thank the reviewer again for the feedback and the already highly positive perception of our work. Nevertheless, we invite them to read our overall additions to the manuscript and to confirm that our revised manuscript has adequately included above points.