Block-operations: Creating an Inductive Bias to Route Data and Reuse Subnetworks

On "there is no inductive bias that would cause FNNs to keep activation patterns for the same concept consistent throughout the network.": We refer to Csordás et al. (2020) for a thorough explanation.

On "What, then, is the unique proposition of this paper?": The concept of block operations is entirely novel. Greff et al. (2020) made a very generic call for research in this direction, while block operations are a quite specific solution.

On "there is no inductive bias that would cause FNNs to keep activation patterns for the same concept consistent throughout the network": If a concept is not represented consistently then the network can not learn reusable subnetworks, because it it not possible to reuse subnetworks that expect different input formats. Imagine if you wrote a computer program where numbers are represented as int, float, double, etc. at different parts of the algorithm even if they are all integers, so that you end up needing to write a separate function for each of them.

On "What novel elements or contributions does your paper bring to the concept of a neuro-symbolic system?": There are a lot of different possible neuro-symbolic systems. Our main novelty are block-operations. They are unique in that they are designed to satisfy all the goals outlined in section 3.1., which has not previously been done.

On "Could you provide a clearer and more structured organization of the paper": We are uncertain what parts are unclear, but can make improvements that are pointed out to us.

On "What specific issues or challenges...": The goals outlined in section 3.1. can be achieved with block-operations. This makes networks able to learn to reuse concepts more easily and improves modularity. We expect gains in performance and generalization on tasks that rely on modularity.

On "Are there any empirical results or experiments": We focused on synthetic tasks so far because our focus was on proving the soundness of the underlying logic (block-operations), not on beating SOTA with a specific architecture. Our experiments confirm the presence of the properties we were looking for. Running practical tests are next steps in our research.

On "How do the concepts of "block-operations" and "Multiplexer" relate to the broader field of neural network architectures...": Existing neural networks of all types can be modified to make use of block-operations, which may improve their performance on tasks that require modularity.

On "Could you address the concerns raised about inappropriate tables and figures, as well as minor typographical errors in the paper?": We will be happy to fix any errors we find, but is unclear to us in what sense the tables and figures are inappropriate. Could you provide an example?

On residual connections: Residual connections do not satisfy the criteria outlined in Section 3.1. as our design goals. They only operate on the entire tensor and can not rearrange parts of the tensor independently of each other. This causes overlap and interference between the neurons used to represent different concepts.

On model sizes: We did test models of various sizes and compared performance. As pointed out in the paper, the details on this are in the appendix. Testing performance at even larger model sizes would not be very practical as the tasks are deliberately quite simple and do not require large models.

On the novelty of block operations: Other architectures have certainly used mathematically similar operations, but not for the same purpose. Please read Section 3.1. to understand the purpose of these operations. Saying that block operations are not new is like saying that self-attention was not a novel concept when it was first introduced because the softmax function had been used by other papers before.

On the example tasks: The tasks are deliberately simple in order to test for the presence of certain properties, which we experimentally confirm. Testing MNIST would not be conducive to our goals: MNIST is a task that relies largely on statistical correlations and requires no reuse of concepts in different contexts. SMFRs help specifically for tasks that rely on modularity. We have run tests on MNIST before and got the results we expected: The SMFR learns not to use routing and learns to simulate an FNN instead. It ends up with the same performance as an FNN. SMFRs only achieve better performance on a task where modularity is useful.

On the experiments: The experiments are deliberately designed to be a minimal test of the attributes we want to test. Our goal is to test if the concept works as intended, not to beat SOTA. You are correct that the arithmetic tasks are very simple and can be solved largely by memorization. We did also run tests on 2 digits and noticed the same patterns, but since these take much longer to run we focused on the 1-digit version afterwards. While 1-digit arithmetic can more easily be solved with memorization than 2-digit arithmetic, this does not actually affect the validity of our results. Routing and module reuse are just as relevant for memorization tasks as for other types of tasks.

On how to report performance: You criticize that we present our results as weaker than they are. Isn't that a good thing? Most papers report the best hyperparameters because it makes their results look better, but this also contributes to replication issues. We deliberately took a pessimistic view, under the assumption that in real-life tasks you won't be able to test all possible hyperparameters. We show that performance is good even so, when averaging over many hyperparameters.

On "SMFRs can learn to automatically arrange unstructured data into the block format they need": Note that it is possible for the network to disentangle the meaning of the unstructured data without being exactly in the format we looked for. For example, by multiplying every value with -1. The "1 of 30" is therefore a lower bound on the architectures ability to rearrange the data. Frankly we were positively surprised to see even the 1 out of 30. The additional layer for linear decoding you propose is a good idea and we will look into it in the future.

Weight sharing: No weights are shared. Each of the blue blocks marked "FNN" in the image is just that: A single FNN.

inputs/outputs: The Multiplexer takes M blocks and produces N blocks. The FNNR takes the concatenation of both the M original blocks and the N blocks produced by the Multiplexer as input. In this way it can condition both on the original input before interpolation and reordering and on the interpolated blocks produced by the multiplexer.

Questions

input/output format: The SMFR replaces the FNN and consequently has the same format, with one exception: The input and output tensors' number of neurons must be a multiple of the block size. The SMFR internally treats the tensor as a concatenation of multiple blocks. Any additional dimension such as batch size, iterations, self-attention, etc. all depend on the architecture the SMFR is embedded in. This is analogous to an FNN, whose input dimension is fixed size but which can get additional dimensions based on the architecture it is embedded in.

On "... because the softmax used by the Multiplexer is also commutative": We mean the following: If the network learns (for example) to calculate the softmax as f(x,y, y)=0.5x+0.5y+0.0*z where x and y are some blocks, then this function will be commutative with respect to x and y. This would allow the network to solve commutation-based tasks even for OOD data, i.e. even for values of x and y that were not seen in that order during training. While this is a good thing, it is not what we want to test. We therefore make it harder for ourselves by using gumbel-softmax, which does not have this property. This lets us be more confident that the performance gain indeed comes from the routing abilities of the SMFR.

On transformers: The SMFR does have a lot of similarity with transformers. However, the similarity is mostly just in the math involved. Their roles are quite different. Transformers are integrated into larger architectures alongside fully connected layers. These fully connected layers destroy the ability to route efficiently, and that is what SMFRs set out to achieve. Even though transformers are internally similar to SMFRs, the FNN layers between them lead to a very different inductive bias.

We will address your complaints in order, before answering the questions:

Semantic Grouping: We should clarify what we meant. As you say, The neurons are not grouped semantically in advance. However, the network has an inductive bias that causes the groups to take on semantically distinct roles. The experiments confirm this assumption.

Clarity: Not sure what to say here. Different people have different preferences? We have had positive feedback about the bullet-point style before.

Terminology: We would have liked to give more details on the things you mentioned, but we are constrained by the page limit. Explaining new ideas took priority over explaining things.

Datasets & Baselines: The focus of this paper is on the core idea of Block Operations. Since the SMFR replaces the FNN, we only compare with it and run only tests that evaluate what we want to test. Real life tests are left as future work. As pointed out in the paper, even if SMFRs in their current state of development should perform poorly, our results show that the underlying idea of block-operations works as intended and is worth exploring further.

Novelty: Attention models of all types are mathematically similar to SMFRs, but they have one crucial difference that makes a comparison pointless: Attention models are still connected to each other with fully-connected layers. These re-introduce the difficulties in data transfer that the SMFR sets out to solve.

Background & Related Work: The reason we do not talk about these in detail is because none of them address the design goals mentioned in Section 3.1. We just mention that they exist to show that we are aware of them, because several of them look similar at first glance but they all fail to achieve the goals outlined in 3.1, usually because they were designed for different purposes.

Implicit Grouping: As shown by Csordás et al. (2020), even if implicit groupings arise, they are not maintained throughout multiple layers of the network.

Questions

On “fully-connected layers treat all neurons as independent of each other”: We will phrase this differently. What we meant was that neurons are treated as independent by the learning process. I.e. their index within the tensor has no effect. Once learning has taken place, the meanings they have taken on are not independent, as you say. The existing models you mention that do group neurons into larger layers perform these groupings for different reasons that do not address our design goals in section 3.1.

On “there is no inductive bias that would cause FNNs to keep activation patterns for the same concept consistent throughout the network.”: This is a finding of Csordás et al. (2020).

About the abstract: "Them" refers to block-operations.

Typos: The phrase "investigate in how far" is valid English and means "investigate to what extent". "More dense" should indeed be replaced with "denser". Thank you for pointing this out!

Dear authors, thanks for the comments. Some responses to them:

Writing/organization: of course there could be preferences for different people on that, but personally I think it goes beyond writing style and damages the paper's clarity. It makes explanations kind of shorter, less detailed, and more disconnected. E.g. in slides when things are organized in a way more similar to that, the speaker does the job of making the presentation coherent, and so in the context of a paper, I personally believe this is less suitable / damages it. Indeed, you agree that you would like to add more details but couldn't due to page limit. I think more efforts on making the writing more concise (not just shorter, but rather to find effective phrasings so to say more in fewer words, so to then make space to additional written content), would be really helpful.

Related works and baselines: even if you claim attention to be inferior than a new approach, you still need to show experiments that empirically validate that's the case. All claims should always be empirically or concretely supported. Same for the related work section, even if prior works don't achieve your desiderata, you still need to discuss them, compare to them, explain how you are similar and different, what you do they don't, what prior works inspired or were the basis for your new idea, etc.

I therefore would unfortunately keep my score.

p.s. Thank you for the comment regarding "investigate in how far", I wasn't aware and am happy to learn about it!