Creative Style Transfer

The paper claims that the method creates “exceptional creative artworks” and offers “significant improvement” over previous methods. I disagree. The results look like other neural style transfer methods, with many of the same limitations and artifacts as previous methods. They look unlike real artistic styles in ways that other NST fails to capture real artistic styles, and they do not offer something useful and new.

The paper provides quantitative evaluations that don’t really demonstrate much improvement. The quantitative evaluation in Table 1 is measured in part on the same loss used for optimization, and the scores are very similar to baselines (even with overlapping confidence intervals). A user study is provided, but the description of the evaluation methodology is completely inadequate (no description of participant population, recruitment protocol, questionnaire protocol, randomization, how images were chosen, no statistical analysis of results, etc.), making it impossible to understand what the user study actually measured.

At a more fundamental level, no motivation seems to be provided for the permutation approach. While this is not strictly necessary—it would be a valuable contribution if it worked—it would be better to understand why the authors thought this was an approach worth trying, whether there’s any motivation for this versus any other procedural algorithm one could come up with to make images and/or rearrange networks.

The paper suffers from unscientific and anthropomorphic terminology, using words like “creative,” “meaningful,” “novelty,” “fresh,” and so on. What does the paper mean when they say a style is “meaningful”? What does it mean to say that the goal is to “imaginatively Infuse a fresh and meaningful style”? How is “imagination” and “freshness,” in the everyday senses of these words, embodied algorithmically? Certainly not what I think that most people would understand as “meaningful.” The definitions provided for “creativity” (e.g., Boden et al) are contested definitions that remove the human element from creativity, whereas many psychology studies of creativity focus on human processes. The paper would be better if it used careful terminology rather than anthropomorphic advertising words.

1. I do not think the so-called creative style transfer problem is meaningful. When given a pair of content and style images, the task of reference-guided style transfer aims to transfer the style patterns of the style image to the content image, not to create so-called novel style elements from nowhere. It is weird that the proposed method requires the reference style image as the input, while the results have colors and patterns that do not belong to the style image.
2. The neural permutation in the paper does not make sense to me. The original AdaIN method scales and shifts the content image's features by the means and variances of the corresponding style image's features channel-wisely. However, the authors propose to reshuffle the style image's means and variances and then apply them to the content image's features. As the reshuffled means and variances do not come from the same feature extraction branches as those content features channel-wisely, it makes no sense to scale and shift the content features by the reshuffled means and variances. I guess that is why the proposed method can create stylized images that have colors and patterns different from the reference style image.

1. Motivation

• I understand that the goal is to generate diverse (and good) styles. But if we use the style perception distance to select outputs that are not that similar to the provided style image, should we still call it “style transfer”? It seems to me that we can use the same metric to train a model to permeate the mean and variance of the content image and still give you some good results. Or we can even use the mean and variance as input to the model to predict some mean and variance, without even needing the style image?
• Thus, I feel that the main goal of this paper is slightly different from previous style transfer methods. And I am not sure if the metrics in Table 1 are the proper ones.

2. Others

• How to decide the values of K1 and K2?
• What if we don’t use position encoding?
• Will the results be very different if use a greedy algorithm instead of the Hungarian algorithm when converting the soft permutation matrix into a hard permutation matrix?
• The user study seems a bit problematic. For the first question, the style image is too small which seems to encourage the users to ignore them and focus on whether the output is visually better (and thus no need to use a style image as a reference). For the second question, whether the output image looks good highly depends on the input style image. Using multiple style images as input and selecting the one with the highest SSAN score as the baseline seems more reasonable and fair to me.

The method of permuting distributions can nver and will never be creative: the method is nut suitable for the aim.

That paucity of the model can be seen from the output - which looks almost exactly the same as the majority of NST that has gone before it (and most of which models the artistic process very badly).

The experimental section uses automatic measures that fail to capture creativity, and the experiment that involved people merley asked them about a preference. Which means the experiment is uniformative with respect to the hypothesis (that permuting permutations is creative).