Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks

1. The results in this paper are empirical and lack of theoretical analysis. For instance, it would be better if the authors could add more discussion on the reason why increasing task complexity will increase the degeneracy in weight space. Maybe some theoretical analysis of some toy examples will help.

2. The strategies proposed for controlling solution degeneracy in practical applications are too general and may not be very useful in practice. For example, increasing task complexity reduces dynamical degeneracy and increases weight degeneracy, it is not clear which strategies we should use for specific tasks. More discussions on this aspect are welcome.

(more important points first)

• Regularization. I could not understand from the paper whether a weight penalty was present when training on the task loss with BPTT. Weight decay for Adam is not specified, I assume it is the default, in case of which there would be some penalty. Was this the case, and if not, how would the measures change if the penalty got increased? For a stronger penalty, I would expect that the DSA metric shows less variation, as presumably noise directions that give rise to dynamical variability would then be penalized to decay. This would make the paper's finding much less interesting to me, because we already know that controlling task-unrelated dimensions is useful. I wonder for example whether Fig. 7 would be induced by strong regularization, biasing the network towards the presumably simpler line attractor solution.
• Expectedness. As the authors state in the discussion, most of the results can be understood with the following argument: As task entropy increases, the network parameters are increasingly constrained. Hence, there are less parameters that are unconstrained and give rise to degeneracy: Effectively, the network has now to solve $N$ tasks in parallel, but has the same representational capacity.
  This makes me wonder how the metrics change when task complexity is not scaled up by replicating the same task over channels, but changing the complexity of the function that needs to be learned? As a very simple and probably too naive example, how would the network behave if task complexity was controlled by the number of Fourier modes in the target signal? It seems that the paper identifies a specific kind of task complexity whose generality is questionable: The proposed measure is not rooted in the literature apart from a loose connection to information theory.
• Weight degeneracy. I find the metric not well motivated. Why should I expect a pair of networks that is non-degenerate (i.e., similar in some way) to have small $d\_{PIF}$? Would I not rather consider an (orthogonal) similarity measure on matrices, for example d\_{sim} \= min\_O ||W\_1 \- O^{-1} W\_2 O||\_F, where $O$ is an orthogonal or general invertible matrix. This is exactly the $d\_{procrustes}$ the authors discuss for DSA. I expect that using this metric on the weights would essentially make it the DSA metric. It would be especially interesting if that changes Table 1.
• Do the authors have a hypothesis why change weight degeneracy depends on control paradigm?
• Relevance. Why should we care about whether a networks solution is degenerate or not? I understand that it is interesting descriptively to know a connection between task parameters and degeneracy metrics, but I am not sure where I would need this knowledge. The paper would benefit from discussing this.
• Controllability. It is hard to generalize from the findings for controlling degeneracy from the DD task only. For example, it is hard to say what an auxiliary loss term would look like for the other tasks, possibly involving some amount of engineering.
• Generality. Do the authors suspect that there a tasks where the observed trends do not hold?
• No code is supplied, limiting transparency and reproducibility.

Again, I like the paper and think it is a valuable contribution, but these points make me question its novelty and generality.

Weaknesses It seems that the central claim of the paper is: “while harder tasks lead to more consistent neural dynamics across task-trained networks, their underlying recurrent weight matrices are more degenerate/variable.” (Section 4). However, this statement may be overly simplistic and may significantly depend on factors that were not considered in the proposed work: (a) Dependence on the initial weight distribution of the RNNs (b) Dependence on the specific distance metrics used to quantify dynamics and weights degeneracy (c) Dependence on the specific notion of complexity

(a) Prior work has shown that the initial weight distribution can have a large influence on the training dynamics and the solutions found by the RNNs. However, there is little information on how the RNN weights were initialized and there is no analysis on how the main claims may depend on their initial distribution. In particular, the decrease in dynamics degeneracy with task complexity may not hold for RNNs initialized with small weights with values close to zero.

(b) The authors measured weights degeneracy with a stricter Frobenius norm than the dynamical degeneracy. The norm used for DSA is invariant under orthogonal transformation, which is less strict than the permutation invariant norm used to quantify the weights degeneracy. The authors found that dynamics degeneracy decreases with task complexity but weights degeneracy increases. How much does this result depend on the specific norm and its invariance class used to measure degeneracy? Would the weights degeneracy decrease with task complexity when considering a similar matrix norm than for the dynamics degeneracy, i.e. orthogonal transformation invariant Frobenius norm?

(c) The authors quantify task complexity as the entropy of the task inputs and outputs. However, this captures only a specific aspect of the task complexity. For example, as stated in section 4: “Moreover, as we modified the task characteristics to study its effect on degeneracy, we observed that degeneracy remains invariant under certain transformations of the task, e.g., changing the delay duration in Delayed Discrimination or altering the environment size in Path Integration tasks.”. It would be really interesting to see these results because increasing the duration of the delay period or increasing the size of the environment can be seen as ways to make the tasks more complex, in the sense that it would probably take longer for the RNNs to learn the tasks. The amount of training required to learn the task is another metric that can be used to measure task complexity. It would be interesting to study how it relates to the entropy based task complexity considered by the authors.

Measuring task complexity as the entropy of the task inputs and outputs implies that it is invariant under shuffling of the timesteps, which completely changes the task structure. Does the relationship between degeneracy and task complexity still hold when considering time shuffled variants of the tasks?

"We chose this measure because it directly reflects the amount of information present in the task’s inputs and outputs, which the network must process and represent through its hidden state (Tishby et al., 2000)." In the experiments done in paper, the dimensions of the RNNs are most of the time higher than the dimensions of your input and output. I do not understand how does this method (of measuring the entropy of input and output separately) works, in the case when the goal of an RNN is to reproduce its input? Say one presented a highly complex input signal. And since the goal is to reproduce, then the output signal is also highly complex. But, the RNN only needs to implement an identity function, but the method suggested would classify this as a difficult task. I think this clearly demonstrates the problem of taking into account the input and output signal separately. At the end of the day, the goal of an RNN is to map its inputs to outputs, so to the best of my understanding, there lies a significant problem in this approach. Since authors base all of their claims to this methods, I am highly confused and would like to have a discussion about this.

Controlling methods are only tried in one task but the results are written in general form. The claims in the paper, I believe could only be made if the authors have tried these methods on different tasks.

In the task settings in the paper I think it is true that increasing the number of channels makes the task more difficult, but I don't think it is always the case. Consider the delayed match-to-sample task. One can either have two different input channels to represent your signals or one. I think in this case representing with two channels makes the task easier. Does this show that this method is not task agnostic?

Fig 4A, third column does not support your claim but you did not discuss it.

Auxiliary loss: I do not believe the defined auxiliary loss is in fact auxiliary in the delayed discrimination task. When the task of a RNN is to output f1-f2 together with sign(f1-f2), I believe the main task becomes the former one.

I have some main concerns/weakness which I will outline here with specifics under questions.

1. The authors present an interesting approach to quantify task-complexity using entropy. However, as detailed in the paper, this is very closely tied to the probability encoding the underlying generative process for each task. Specifically, while adding channels increases the information, a more robust measure would be changing the probability of the generative process (for instance the probability of a bit flip in a single channel). While in principle adding channels increases information associated with tasks, it's unclear if this is representative of the "randomness of the task".
2. The authors discuss ways to control solutions associated with task-trained RNNs. Other than increasing task-complexity and noting it's trend associated with weight and dynamical degeneracy (presented in this paper), the other four methods have been previously observed. Further, the authors do not provide any theoretical or discussion regarding how this relates to the method presented in this paper. Can the authors expand on the novelty of this section?