On Sampling Information Sets to Learn from Imperfect Information

The paper's main problem seems to essentially reduce to supervised learning: namely, given a dataset $(x_i, y_i)\_{i=1}^n$, estimate a function $g(x) := \mathbb E_{y|x} y$. From this perspective, as I will discuss below, the results in the paper are, in my opinion, not very illuminating given what is already well understood about supervised learning. Perhaps most critically, the paper does not seem to make any use whatsoever of the hidden information $h$ in any of the proposed techniques, preferring instead to simply sample from the conditional distribution $y|x$ and use those samples to compute some summary statistic (majority vote, or average value) to feed to the learner instead.

This paper is framed in such a way that the main application is supposed to be in games. But this bothers me: in games, the assumption of the distribution $h|x$ being known is very false, since in practice the opponent has control over incomplete information (e.g., in poker, by choosing its range in any given situation). I would argue that this lack of knowledge of the distribution---not any issue of statistical estimation---is the fundamental barrier to good value estimation in games.

I am also confused about the setup for the MNIST experiment. From what I am understanding, for each training instance $x_i$, you draw $k$ samples $\hat{y}\_i^{(\kappa)}$ ($\kappa= 1, \dots, k$) and then pass the majority vote of these samples, $\hat y_i$, to the learner as the training instance. But this would not optimize the above objective function: indeed, for example, if $p = 0.9$ (say) and $k$ is very large, one would expect $\hat y_i = y_i$ always for every $i$ (simply by LLN) and therefore the classifier would learn to always output the true class. But that would be wrong: when $p = 0.9$, the expected value $g(x) := \mathbb E_{(h, y)|x} y$ is not the true class, due to the high noise. Measuring the resulting classifiers according to their test accuracy also seems wrong, for the same reason (it's not the objective stated by the paper). I think in the MNIST case, like in the game cases, the aggregator should be the average $\mathbb E_\kappa \hat{y}\_i^{(\kappa)}$ (where the $\hat{y}\_i^{(\kappa)}$ are to be interpreted as one-hot vectors), not the majority vote.

It seems also pretty clear to me why the results in Figure 4 are the way they are: with a fixed number of evaluations, having more samples per instance means having fewer unique instances, which could lead to overfitting; on the other hand, with a fixed number of updates, having more samples per instance only means less noise in the training data, which is a good thing.

• The unclear motivation: The authors argue that the motivation comes from evaluating the game tree of the imperfect information game. However, for evaluating imperfect information game, one should also take opponents' policy into account and this paper completely ignore this. Hence, I don't think this paper can have any help for imperfect information games. Learning with noisy labels can be an alternative understanding but I don't think the authors show some rigorous consideration on that.
• The poor presentation: I believe statements in Section 2 have no significance -- they are just standard Monte Carlo approximations. Until I go through the experiment I realize the authors may want to argue that how to generate samples that can help the neural network training. I don't think the authors make this clear in previous sections and there should be much more discussion on this before experiments.
• Limited significance: For me I think this is just an experimental report and I cannot get any meaningful conclusion from this paper. The authors make no discussion on the reason to use these methods and what are the potential benefits/drawbacks using these methods in a rigorous manner. I also believe that the proposed problem is a delicate learning theory problem and at least there are lots of work on learning with noisy labels that propose different algorithms with theoretical justifications to achieve strong empirical performances. This paper instead shows no effort on understanding the problem and improving the performance upon existing baselines.

• This paper is not scientifically sound. It does not provide a concrete mathematical problem formulation, and it does not provide a principled approach to address a well defined problem either. The main contribution of this work is some empirical observations, which does not provide much insight or guideline on how to design an efficient sampling strategy to mitigate the partial observability in decision making problems.