Authors' response to reviewer i3cT

We deeply thank reviewer i3cT for their detailed and constructive comments. We are grateful towards the reviewer for recognizing the comprehensiveness of our benchmark and the efficiency of our approach. We appreciate the reviewer mentioning that while this task is under-addressed and constitutes an interesting topic of research. We address the reviewer's remarks as follows:

Weaknesses

Many parts are hard to understand.

We thank reviewer i3cT for their honest feedback. We would be happy if they could point out precise elements that need clarification (phrasing of sentences, the terminology, the notation, etc.) so that we can incorporate it into the final version of the paper.

Approach is not clear after reading section 3.

Similarly, we are willing to rephrase, re-write, re-explain or add more details in section 3 upon specific requests. Before we release the final version, we will make sure to read and re-write this section to make it as clear as possible.

More details on the task could be shared as it is relatively uncommon in the CV community.

Although our baseline papers:

• Self-supervised scene de-occlusion. Xiaohang Zhan et al.
• Amodal instance segmentation with kins dataset. Lu Qi et al.
• Instance-wise occlusion and depth orders in natural scenes. Hyunmin Lee et al.

are all published in CV conferences, we agree that the task has not been yet deeply explored. We are willing to polish our paper and will include more details on the task and the dataset format in the supplementary materials of our final version (cf. "questions" section for more details).

The authors mention that their holistic approach is more efficient as it predicts everything in a single forward pass, but its efficiency compared to other baselines is not tested quantitatively.

We report the inference time and the memory consumption of our proposed proposed InstaFormer and compare it against baselines in section F and figure 7 of the supplementary materials. We choose InstaOrderNet as the baseline for comparison since it constitutes the most recent and competitive approach while being representative of all pairwise approaches. We benchmark the official implementation of InstaOrderNet, but also implement a batched version of it that consists in stacking all the pairs of instances in a single batch and performing inference on them all at once. We refer to this as InstaOrderNet (batched). Note that InstaFormer is not optimized for speed.

In figure 7a, InstaFormer's inference time appears constant, while InstaOrderNet's grows exponentially with respect to the number of predicted instances in the scene. InstaFormer's inference time averages below 1 second and becomes advantageous over InstaOrderNet's default inference anywhere above 11 instances. The batched version of InstaOrderNet consistently outperforms InstaOrderNet and InstaFormer.

Yet, in practice, it is hard to use the batched version of InstaOrderNet since it requires an exponential amount of memory with respect to the number of predicted instances. As a matter of fact, while InstaOrder's default implementation runs in a constant memory size, its batched inference quickly becomes prohibitive. Inference for more than 20 instances results in more than 40Gb of memory usage. On the other hand, InstaFormer's memory usage grows linearly with respect to the number of instances. It proves more affordable than InstaOrderNet's batched inference for more than 10 instances and saves almost 30Gb of memory for 20 instances. We encounter CUDA OOM error for any inference containing more than 20 instances using InstaOrderNet (batched), point at which we stop the experiment.

We believe that these results place InstaFormer in a favorable position considering the runtime speed and memory trade-off. Further optimization of the inference procedure can decrease its runtime even further.

Questions

Major

How is $G$ usually constructed?

For both occlusion and depth order prediction $G$ represents the adjacency matrix of the order relations between the instances in the scene. The rows and columns each represent an instance while the entries represent the order relations (occlusion or depth) between them.

Given $I \in$ 2, \dots, N $$ instances in the scene, we consider a pair of instances $(i, j) \in I \times I$ such that $i \neq j$ and consider their entry in the order matrix $G_{i,j}$. We never consider the case where $i = j$, i.e. the diagonal of the matrix, since it does not make sense for an instance to occlude (resp. be in front/behind) itself. Thus, we always manually fill the matrix diagonal with -1 (this value is arbitrary).

The occlusion order task predicts binary labels $G_{i,j} \in \lbrace0,1\rbrace$ for all $(i, j) \in I \times I$, such that $i \neq j$, symbolizing whether:

1. instance $i$ does not occludes instance $j$ (label 0).
2. instance $i$ occludes instance $j$ (label 1).

In the case where $G_{i,j} = G_{j,i} = 1$, we say that there is a bidirectional occlusion relation between $i$ and $j$.

The depth order task predicts a ternary label $G_{i,j} \in \lbrace0,1,2\rbrace$ for all $(i, j) \in I \times I$, such that $i \neq j$, symbolizing whether:

1. instance $i$ is behind instance $j$ (label 0).
2. instance $i$ is in front of instance $j$ (label 1).
3. instance $i$ and $j$ share the same depth region (label 2). This is the case when the two objects span in a large overlapping portion of the 3D space.

We will complement the paper with this explanation in the final version.

What is the difference between $G_{i,j}$ and $G_{j,i}$?

$G_{i,j}$ represents the order relation from $i$ to $j$ while $G_{j,i}$ represents the order relation from $j$ to $i$.

For occlusion, this is because the occlusion relation is not always symmetric: bidirectional occlusion relations are possible, but not always present. Sometimes, only $i$ might occlude $j$ and sometimes, only $j$ might occlude $i$.

Depth on the other hand is symmetric. An instance if $i$ appears in front of $j$, then $j$ must necessarily appear behind $i$, unless they share the same depth range, in which case they are both annotated using label 2.

How does $G_{i,j}$ and $G_{j,i}$ look like if object $i$ and $j$ are completely unrelated?

Naturally speaking, the notion of "unrelated" only applies to the occlusion order task. In the depth order task, all instances are "related" since they all necessarily appear one behind the others.

For the occlusion order task, we understand the notion of "unrelated" as a case where $i$ does not occlude $j$ and vice-versa, thus, their entries in the order matrix would simply be $G_{i,j} = G_{j,i} = 0$.

More training details (loss, etc.).

We have overlooked the details of our training recipe and we thank reviewer i3cT for offering us the opportunity to clarify it.

• Losses
  • Occlusion order optimization: we use the binary cross-entropy (BCE) loss.
  • Depth order optimization: we use the cross-entropy (CE) loss.
  • Occlusion + depth order optimization: we use both BCE and CE losses, equally weighted.

We add auxiliary losses to help convergence of our networks. We detail them below:

• Auxiliary losses

  • Occlusion order: we add a BCE loss on the occlusion order logits of all transformer decoder layers.
  • Depth order: we add a CE loss on all the transformer decoder layers.
  • Occlusion + depth: we add both BCE and CE losses on all the transformer decoder layers.

• Batch size

  • 16 in all cases.

We hope that these clarifications are sufficient to help the reviewer understand our training setup and full reproduction of our experiments. We will include all those details in the revised version of our paper. We will also provide a link to our repository upon acceptance, allowing to re-train all InstaFormer networks in a single command.

Quantitative analysis on efficiency of "holisticity", i.e. inference time and memory of proposed approach and baselines.

We kindly refer reviewer i3cT to our answer in the "weaknesses" section above and to the supplementary section F along with figure 7.

Minor

Some figures are not hooked in the text.

We thank reviewer i3cT for pointing out this issue. We will hook all figures to the text in the final version of the paper.

Contributions in the introduction are unclear (#2 is unclear, #3 is arguably not a contribution).

We will drop both contribution #2 and #3 in the final version of the paper. We will polish the first item of the contributions to reflect the nature of our submission.

Include appendix at the end of the paper?

We agree with reviewer i3cT. We will move the appendix to the end of the paper in the final version. We do not see any reason not to merge it with the main paper now that it has been finalized.

Why are the baselines in Sec. 3.3 discussed in the main section dedicated to the proposed approach?

We agree with the reviewer after reading the paper again. We propose to move the baseline discussion to the experiment section (sec. 4) of the paper to keep the flow focused on our InstaFormer approach in Sec 3. We welcome any further suggestions on how to improve the flow of the paper.

Limitations

Limitations are not really addressed. A proper evaluation is necessary.

We highlight a failure case of our InstaFormer network in section G.3 and figure 9 of the supplementary materials. We call this failure case "instance mix-up". We kindly refer the reviewer to it. We also observed another failure case which we call "segmentation failure" where mispredicted masks lead to creating nodes in the graph that belong twice to the same instance, resulting in partially wrong order predictions. We will include this failure case in the final version of the paper.

Dear Reviewer i3cT,

Thanks for serving for NeurIPS. Please read the rebuttal and share your thoughts here. The author-reviewer discussion deadline is tomorrow. Please note that per NeurIPS's policy, you have to post a discussion before clicking on the mandatory acknowledgement.

Thanks,

AC

Authors' response to reviewer NnW7

We are pleased to hear that reviewer NnW7 appreciates our work and points out the strong and comprehensive results of our approach while adopting a novel design. We thank reviewer NnW7 for its precious comments and we address them as follows:

Weaknesses

The computational efficiency claims lack empirical validation such as FPS or memory usage, essential for assessing practical applicability.

We ran a computational efficiency comparison experiment in section F of the supplementary materials. We kindly refer reviewer NnW7 to the aforementioned section and figure 7 of the supplementary materials for a detailed analysis of the computational efficiency of our approach compared to the InstaOrderNet baseline.

Nonetheless, we will quickly summarize the results. We compared InstaFormer against the InstaOrderNet baseline since this is the most recent art and competitive approach. We used the official implementation of InstaOrderNet but also realized that there is a more efficient inference procedure that consists in batchifying all the pairs of instances together and performing inference on them in a single batch. We refer to this as InstaOrderNet (batched). For all InstaOrderNet variants, we resize the image to 256x256 pixels. For InstaFormer, we follow Mask2Former's inference script and do not resize the images. Please, note that this results in a disadvantage for InstaFormer since the images have a larger resolution.

We compute inference speed in seconds and memory usage in Gb with respect to the number of instances in the order graph.

For inference speed, we observe that InstaFormer appears with a constant inference time, while InstaOrderNet appears exponential with respect to the number of instances. Our approach's average inference time lies below 1 second. We emphasize the fact that we have not yet optimized the method for speed. Above 11 instances, InstaFormer becomes faster than the default InstaOrderNet. On the other hand, the batched InstaOrderNet is consistently faster than both InstaOrderNet and InstaFormer.

However, this batched performance comes at a cost. Comparing the memory usage, we observe that inference on the batched InstaOrderNet rapidly becomes prohibitive. As a matter of fact, its memory complexity grows exponentially with respect to the number of instances in the scene. On the other hand, InstaFormer shows a linear memory growth. Above 10 instances, InstaFormer's cost falls below this of InstaOrder (batched). At 20 instances it saves almost 30Gb of memory compared to the batched version of InstaOrderNet. We encounter CUDA OOM errors for any inference above 20 instances using InstaOrderNet (batched), thus we stop the benchmark here. Naturally, the default InstaOrderNet memory usage remains constant with respect to the number of instances.

We believe that these results show that InstaFormer obtains the best trade-off between runtime speed and memory usage, especially when instances in the scene grow beyond 10. Further optimizations of the inference method could lead to even better results.

The scalability of the approach beyond the dataset's 10 instances limit remains unverified.

When benchmarking our approach in figure 7 of the supplementary materials, we benchmark results up to 20 instances in the scene which supports the scalability of our approach.

We also empirically observe that our approach scales well largely above 10 instances by predicting the order graphs on the whole validation set of COCO. We report the average and maximum number of instances returned by InstaFormer^o,d in the following table:

In all cases, we observe that we are not bounded by the amount of queries we use to predict the order graphs. Moreover, we showed the computational advantages of using InstaFormer over baselines when the number of instances in the scene grows beyond 10 in the previous question. This is on average the case in COCO's natural scenes.

Questions

We thank reviewer NnW7 for their important questions. We address them as follows:

Scalability.

Inference efficiency.

Please, refer to the previous section in which we address both these items.

Readability of Fig. 3. Could you elaborate on the rationale behind using a transformer with only one layer? There may be underlying motivations that make your design unique and theoretically superior to alternatives. It would be helpful to elaborate on these motivations before delving into the specifics of the design.

We thank reviewer NnW7 for offering us the opportunity to clarify the design choices of our proposed InstaFormer network.

Task formulation

The main motivation behind our method is to reformulate the problem of pairwise order prediction as a full adjacency matrix prediction. We can now think about the problem in reverse order: if we start from the adjacency matrix of the order graph $G$, we can think of each element $G_{ij}$ as the result of an order compatibility score between the instances $i$ and $j$. We can then model this score by performing a simple dot product over tokenized representations of $i$ and $j$.

Since the transformer inherently behaves at the token level, the choice of using a transformer naturally emerges at this point. Yet, there remains a question: "how to encode the instances into tokens?".

Instance encoding

Ablation in table 2a of the main paper shows that simply using the queries of Mask2Former as input is not an optimal choice. However, we Mask2Former's per-pixel decoder latent features are aligned with the query instances of the transformer (i.e. represent the same instances) while containing complementary information. These features are crucial determining delicate order predictions since they are pixel-aligned. Thus, we create descriptors from the per-pixel latent space of Mask2Former and observe (still in table 2a of the main paper) that this formulation of the compatibility score results in the best performance.

We would like to mitigate any ambiguity by recalling that the main transformer decoder of InstaFormer contains 8 layers. However, reviewer NnW7 is right in pointing out that the encoder for the instance-aligned per-pixel features of our geometrical order head contains a single transformer layer.

We empirically observed that simply max-pooling the features results in lower performances (cf. table 2b of the main paper). Adding a transformer layer dramatically increases the performance of the model. Our motivation for adding this layer resides in the fact that the instance-aligned per-pixel features of Mask2Former have not been able to share critical spatial information used by InstaFormer to determine the order relations between instances. Self-attention at the instance level allows the model to generate much more comprehensive and robust instance descriptors that, in turn, lead to better performance across occlusion and depth order accuracy.

Thanks to reviewer NnW7's comment, we extended this ablation and increased the number of transformer layers before the max-pooling operation. We present the results in the following table:

This experiment is performed using the same setup as in table 2 of the main paper.

We observe that the occlusion order performance marginally improves when increasing the number of transformer layers above 1. Worse, it degrades the performance of the model for the depth order prediction task. We believe that this is due to the fact that adding more layers makes the optimization procedure harder. It is also better to minimize the number of layers to reduce inference time and memory usage. Thus, we opt for a single transformer layer before max-pooling.

Theoretical advantages of predicting $G$ in one pass.

The main theoretical advantage of predicting $G$ in one pass is that we achieve $O(1)$ inference complexity with respect to the number of instances in the scene. This claim is empirically supported by our runtime/memory experiments in section F figure 7 of the supplementary materials.

We propose an interpretation of why depth order prediction seems to be easier than occlusion prediction in the holistic setting in section E.1 (task difficulty) of the supplementary materials. We kindly refer the reader to the aforementioned section for details. Note that this property is unique to the holistic setting and does not emerge in pairwise order prediction methods.

Limitations

The paper mentions certain limitations, but it would be beneficial if the method could handle an unlimited number of instances, increasing its practicality and effectiveness in more complex scenarios.

We hope that the previous sections have clarified that our approach can handle easily more than 10 instances. InstaFormer is, in theory, bounded by the number of queries $N$, but the table we provide in this answer shows that this is not a limitation in practice. We also provide failure cases of our approach in section G.3 and fig. 9 of the supplementary materials.

We are planning on extending this section in the final version of the paper to include more failure cases such as the "segmentation failure" in which the segmentation backbone creates multiple masks for the same instance, resulting in multiple nodes for the same object in the order graph.

We thank again reviewer NnW7 for their constructive comments. We will integrate them to our final version.

Authors' response to reviewer iz2C

We would first like to thank reviewer iz2C for its detailed and positive feedback. We strongly appreciate reviewer iz2C's acknowledgment of the soundness, efficiency and performance of our design. We are pleased to hear that they found our baselines comprehensive and benchmark convincing. We reply to their considerations as follows:

Weaknesses

InstaFormer requires a full decode of an instance segmentation mask.

We agree that InstaFormer relies on the output obtained from the Mask2Former segmentation backbone. This means that the detection of the instances influences the order prediction of the geometrical order module. We observed cases where instance segmentation failure resulted in order prediction mismatch.

Nonetheless, we would like to highlight that we are not using the masks as raw inputs to our geometric order prediction head. Instead, we just use the masks to mask the features of the per-pixel decoder. We believe that this is a substantial nuance that is worth keeping in mind. Our network input's remains in the latent space of the segmentation backbone.

Moreover, we would like to emphasize that obtaining the segmentation masks as the output of the network is something desirable. The segmentation masks are what "ground" the order predictions in all cases. Supposing we would not decode the masks, it would be hard to interpret the entries of the order matrices since we would not be able to tell which row and column correspond to each instance in the image. In the case where the instance label is unique in the scene, one could argue that the predicted label would be sufficient to tell the entries apart. However, this solution is not robust in the case where multiple instances of the same class are present in the image (which is frequent on most natural scenes). In that case, it is mandatory to obtain the masks from the segmentation backbone to be able to interpret the output order graphs properly.

About "making real-world inference impractical as the masks might be unavailable to the user".

We hope that this repetition did not interfere with the reading experience in a negative way. We believe that mitigating this limitation constitutes an important contribution of our work, yet, we will make sure to only mention it once in the final version. We agree that this idea does is an "heroic leap of faith", yet, we still believe that it constitutes a significant and natural evolution in the paradigm of order prediction.

The use of the dot product rather than separate classification is good but does not seem sufficient to be called "holistic". Is this implied from the idea of self attention? Is it from using the dot product for prediction ?

The reason why we called our network "holistic" is because it can obtain all the order relations in a single forward pass as opposed to previous baselines which only behave at the pair level. This property naturally emerges from formulating the order prediction problem as a full graph inference and solving it using a transformer architecture where each token represents a single instance. Thus, reviewer iz2C is right in thinking that this comes from the idea of self attention. Since self attention behaves at the token level and since each token has been carefully crafted to represent an instance, we naturally draw the order relations for all the object pairs in a single transformer pass. The final dot product only acts as some aggregation operation to fall back in the $N \times N$ matrix of $N$ detected instances in the image. One can think of it as some object-wise relational compatibility in the latent space.

Questions

More information on how Mask2Order was trained. Is it trained (always) on results from Mask2Former or is this at inference time ?

We thank reviewer iz2C for this question which gives us the opportunity to clarify our setup and propose a follow up experiment.

Results reported for Mask2Order in table 1 of the main paper were obtained as follows. We trained InstaOrderNet from scratch using the same recipe as in the original InstaOrderNet paper. Once training done, we concatenated InstaOrderNet and Mask2Former and conducted evaluation of order predictions on the Mask2Former generated masks. We train on InstaOrder and perform inference on Mask2Former generated masks. Results of Mask2Order are at inference time.

Reviewer iz2C's question gives us the opportunity to train a new version of Mask2Order, which uses the Mask2Former masks as inputs to the order prediction head even during training. We follow the same training recipe as in the original InstaOrderNets with a single exception. As we empirically found this task harder than order prediction on GT masks, we modernize the optimization routine by using AdamW with a learning rate of 0.001 and a cosine decay schedule instead of the SGD optimizer with step decay from the original InstaOrderNet paper.

We report the results of this new version of Mask2Order in the following table:

Compared to the original Mask2Order results, we observe a significant accuracy boost for the occlusion order prediction task (+1.92 points in F1 score for Mask2Former^od), while the depth order performance slightly decreases (+1.1 points in WHDR all for Mask2Former^od).

Yet, these results still all fall short when comparing them to the original InstaOrderNet results, highlighting the difficulty of end-to-end order prediction while strengthening the efficiency and the importance of our InstaFormer architecture. Indeed, we observe that the InstaFormer models all outperform both frozen and trained Mask2Order models.

We lacked time to perform the occlusion and depth only experiments in this rebuttal, but will add them during the conversation phase (if we have the opportunity) and in the final revision of our manuscript along with an analysis. We thank reviewer iz2C for providing this experimental suggestion to us.

The final prediction is a dot product which is just binary classification. How does this determine the {1, 0, -1} results needed for occlusion ordering? Is it a separate MLP for each "class"?

We encourage the reviewer to follow along Fig. 3 of the main paper to understand the following explanations more easily.

One can think of the final dot product as an object-level relational compatibility. The representations of the object queries features $Q^\*$ and object descriptors $D^\*$ are both $N \times C$ and $N \times C$. Thus, $G = Q^\*D^{\*T}$ results in a $N \times N$ matrix which is sent to a final MLP (MLP$_{\omega}$ in figure 3 of the main paper. Note that the final MLP is task dependent, meaning there are 1 MLP for occlusion and 1 MLP for depth).

The output dimension of the final MLP is either 2 for the occlusion task or 3 for the depth ordering task. Indeed, the occlusion task requires to predict whether an object is in occluding/hiding, with label 1, or has no occlusion relation with another object, with label 0. The depth order prediction task requires to predict whether an object is in front of another, with label 1, or behind with label 0. There is a third label, label 2, which corresponds to objects sharing the same depth range.

We manually remove the diagonal of the matrix since it does not make sense for an object to occlude itself or be in front/behind itself and compute the loss on all the remaining elements of the matrix. Thus, we confirm to reviewer iz2C that there are no separate MLPs for each class. The final MLP treats all entries in the matrix uniformly and its goal is simply to output proper dimensions for each of the tasks.

We use BCE for the occlusion order prediction task and CE for the depth order prediction task. At inference, we argmax the predictions on both the occlusion and depth order logits and manually fill the diagonal with -1 (this value is arbitrary).

If more questions arise, we kindly refer you to our answer to reviewer i3cT where we provide explanations to other details of the tasks. If your questions remain, we would be pleased to answer them during the discussion phase.

Authors' response to reviewer SiaZ

We deeply appreciate reviewer SiaZ's thoughtful comments and suggestions. We are glad they found our method interesting, our benchmark comprehensive and our writing clear. We address each of their comments below:

Weaknesses

The related section could benefit from the authors reviewing older works on the subject. For example, "Monocular Object Instance Segmentation and Depth Ordering with CNNs" by Zhang et al. and references therein.

We thank reviewer SiaZ for pointing this relevant work to us. We agree that this paper falls in the range of our work by proposing a pioneering approach in the field of depth order prediction. While the suggested work did not provide open-source implementation of their method, we plan on citing it in the related work section of the final version of our paper.

Thanks to reviewer SiaZ's suggestion, we also found the paper "A Learning-Based Framework for Depth Ordering" by Z. Jia et al. to be a relevant depth order art and will also cite it in the final version of our paper.

We will continue to look for other relevant works and make sure to include them.

Motivating why depth order is important could be done more convincingly. I'm still not sure why someone would actually use this method in practice.

We thank the reviewer for offering us the opportunity to clarify the motivations behind predicting depth order relations in a scene. We currently think that there are several compelling areas in which depth order relations are crucial and where our method can be applied effectively:

1. Real world applications
   • Autonomous driving: understanding the depth order of objects in a scene is crucial for safe navigation and obstacle avoidance.
   • Augmented reality (AR): depth ordering is essential for rendering virtual objects in a way that they appear naturally integrated into the real world.
   • Positional verification in sports: in sports, depth order awareness can become crucial to assess if a rule has been violated. We give a two examples here:
     • In athletics: depth orders can determine the finish order of runners (as illustrated in Fig. 1 of the main paper).
     • In football: depth orders can be used to determine if a player is offside or not.
2. Vision language models (VLMs)
   • We show that predicting order graphs before VLM inference can improve the spatial understanding of VLMs in a zero-shot manner (Fig. 1 of the main paper). This is particularly useful since VLMs often struggle with spatial reasoning, while fine-tuning them is usually expensive in terms of data and compute resources.
3. Robotics
   • Recently, VLMs have been used in robotics in the form of perception modules. Thus, obtaining fine-grained spatial relations of objects in the surrounding world becomes crucial for grounded and safe interactions between the robot and its environment. In some cases the description of an object to pick up or manipulate are given by its location in space (e.g. "the object behind the red box"), which can be poorly interpreted by VLMs. As shown in Fig. 1 of the main paper, providing depth order relations can help the robot to identify the object to be manipulated accurately.
4. Image editing:
   • Recent works such as "LooseControl: Lifting ControlNet for Generalized Depth Conditioning" S. F. Bhat et al., or "Build-A-Scene: Interactive 3D Layout Control for Diffusion-based Image Generation" by A. Eldesokey & P. Wonka, use monocular depth estimation as a mean to edit images efficiently. Although this requires further research, we believe that our method could be used to build similar models without relying on a full depth map of the scene. Generally speaking, the field of layered image editing, where each object is considered as a different plane on the 3D space, is a promising area where our approach has chances to be applied effectively.

As suggested by the reviewer, the paper "Monocular Object Instance Segmentation and Depth Ordering with CNNs" also provides examples of cases where our approach could be used in practice: driver assistance, image captioning with spatial arrangements, Q&A retrieval systems, and 3D scene description.

We will rewrite some part the introduction to include some of these examples and motivate our work more effectively.

We hope that these examples clarify the motivations behind our work and its potential applications. We thank reviewer SiaZ again for giving us the opportunity to clarify these important points.