Keywords

1 Introduction

Computational aesthetics is a very important field in compute vision. In 1933, George David Birkhoff firstly proposed the quantification theory of aesthetics in his book “Aesthetic Measure” [1]. We call it the beginning of the computation aesthetic because the calculation method is used in this theory. In 2005, Hoenig [2] defined the computational aesthetics: computational aesthetics is a kind of computational method that can make the aesthetics judgment with respect to human. By 2012, Galanter [3] evaluated the past and future of computational aesthetics. So far, there have been a lot of methods used in computational aesthetics. For example, Miquel Feixas and others used the method of information theory [4,5,6] to analyze Van Gogh’s paintings. Graham et al. [7] used the method of Multi-scale to test and research the similarity among art works. Taylor et al. [8] used the fractal analysis method to analyze Pollock’s drip paintings. Claro et al. [9] used the microspectrofluorimetry method to analyze and identify the red parts between Van Gogh’s works and ancient Indian imitations. In addition, harmonic analysis [10], Digital analysis of Van Gogh’s complementary colours [11], wavelet analysis [12], information entropy [13] and other methods have been widely used in the painting style analysis.

Sparse coding is an interdiscipline of computer graphics, biology neuroscience, psychology and statistics. The method decomposes the images that have the same style into a series of linear combination of the basis function and the sparse coefficient. The basis function trained by standard sparse coding algorithm can reflect the essential characteristics of paintings and has the relative stability and universal applicability. It can also remove image redundancy effectively and get the independent painting features and enlarge distance of different kinds of image features at the same time. Hughes et al. [14] used sparse coding to distinguish Pieter Bruegel’s paintings from the fake. And then, Hughes et al. [15] analyzed the painting style by comparing higher-order spatial statistics and perceptual judgments based on sparse coding. Liu et al. [16, 17] used sparse coding to analyze Van Gogh’s painting style. These studies have achieved good results, but they were made based on the high-resolution paintings. In reality, the high-resolution paintings are difficult to obtain. In order to reduce the image storage and be easy to network transmission, the resolution of paintings is decreased in digital process. The paintings of famous artists that we can enjoy in the website almost are low-resolution. If we use these low-resolution paintings obtained from the network to analyze painting style, what will the result be?

Based on sparse coding, contrast experiments were carried out on high and low resolution paintings in this paper. The basis functions of sparse coding were trained from the same paintings of high and low resolution. And features were extracted in Gabor domain and frequency domain from the trained basis functions. Then the normalized mutual information (NMI) was figured out, the value of NMI was used to analyze the painting style. At last, we used the feature with better performance to judge the painting style.

2 Paintings Used in This Paper

The paintings we used were a collection of Van Gogh, Monet, Renoir and Da Vinci. Van Gogh, Monet and Renoir are the representatives of impressionist, and their painting style is different from Da Vinci’s. Van Gogh’s paintings were used to evaluate the ability of features extracted from high and low resolution paintings in analyzing painting style. Monet, Renoir and Da Vinci’s paintings were used to classify the paintings’ style. The quantity, resolution and representative works of different artists used in this paper are shown in Table 1.

Table 1. The quantity, resolution and representative works of paintings
Full size table

3 Experiments and Results Analysis

3.1 Performance Evaluation of Features Extracted from Low-Resolution Paintings

Liu et al. [16] found the basis functions of the high-resolution paintings can well represent the different styles of paintings, to a certain extent. Whether or not can we get the same conclusion that the basis functions of the low-resolution paintings can represent the painters’ style as well as those of the high-resolution paintings? In this paper we use the same methods in [16] to find the answer.

The experiment steps in [16] are as follows.

  1. (1)

    Use standard SC algorithm to train images of Paris Period, Arles Period, Saint-Rémy Period, Auvers Period and the overall group to obtain the basis function.

  2. (2)

    Work out the statistics of the base function including Gabor energy, peak direction, peak space frequency, direction bandwidth, space frequency bandwidth, peak direction-peak space frequency joint distribution and direction bandwidth-space frequency bandwidth joint distribution.

  3. (3)

    Work out NMI using cluster analysis between the overall group and other groups.

  4. (4)

    Compare the NMI of the different parameters and draw the conclusion.

In order to make a comparison with the results of [16], the same Van Gogh’s paintings in [16] with high and low resolution are used in this paper. The group and quantity of these paintings are shown in Table 2.

Table 2. The group and quantity of Van Gogh’s paintings
Full size table

3.1.1 The Comparison of Basis Functions Trained from High and Low Resolution Paintings

We compare the basis functions of Van Gogh’s low-resolution paintings with that in [16], the results are shown in Fig. 1.

Fig. 1.
figure 1

Comparison of basis functions of Van Gogh’s high and low resolution paintings

Full size image

From Fig. 1, we can see that the length of lines in Auvers Period’s basis function is shorter than that of Paris Period’s. This is consistent with critics’ view that Van Gogh was influenced by pointillist style, and his brush became short in his late year paintings. Liu et al. worked out the length of lines in Van Gogh’s basis functions in [17], also found that the length of lines in Auvers Period’s basis function is shorter than Paris Period’s. Compared with the results of Van Gogh’s high-resolution paintings, we find that the length of lines in basis functions of Van Gogh’s low-resolution paintings also showed the same result. That is to say, the basis functions of low-resolution paintings have the ability to represent artists’ painting style.

3.1.2 The Performance Comparison of Features Between High and Low Resolution Paintings

From basis functions, although we can observe the differences of painting style visually, how to quantify them? We extract the same features in [16] from low-resolution paintings in this paper, and analyze whether these features have the same performance with the ones of the high-resolution paintings in [16].

We extract features from Van Gogh’s overall group and other four groups in this paper, and work out the NMI of features between the overall group and other groups. The NMI is a method that can used to measure the similarity between two groups of set, the higher NMI, the more similar two groups of set are.

Specific calculation steps of NMI are as follows.

  1. (1)

    Cluster the basis functions of group Van Gogh overall and each other group for each characteristic into matrix X and Y.

  2. (2)

    Calculate the Mutual Information between X and Y.

    $$ \begin{aligned} & I(X;Y) = \sum\limits_{i} {\sum\limits_{j} {P(x_{i} ,y_{j} )\log \frac{{P(x_{i} ,y_{j} )}}{{P(x_{i} )P(y_{j} )}}} } \\ & = \sum\limits_{i} {\sum\limits_{j} {P(x_{i} |y_{j} )P(y_{j} )\log \frac{{P(x_{i} |y_{j} )P(y_{j} )}}{{P(x_{i} )P(y_{j} )}}} } \\ \end{aligned} $$
    (1)
  3. (3)

    Calculate the Information Entropy between X and Y respectively.

    $$ H(X) = - \sum\limits_{i} {P(x_{i} )\log P(x_{i} )} $$
    (2)
    $$ H(Y) = - \sum\limits_{j} {P(y_{j} )\log P(y_{j} )} $$
    (3)
  4. (4)

    Calculate the Normalized Mutual Information

    $$ NMI(X,Y) = \frac{I(X;Y)}{{\left[ {H(X) + H(Y)} \right]/2}} $$
    (4)

Since the paintings in this experiment were all drawn by Van Gogh and share the same style of drawing, a high NMI shows that the specified measure of characteristic has better effect in style analysis. The NMI of low-resolution paintings and the NMI of high-resolution paintings in [16] are shown in Fig. 2.

Fig. 2.
figure 2

NMI of high and low resolution paintings

Full size image

In the left column of Fig. 2, it can be seen that the NMI calculated by Gabor energy is obviously higher than that calculated by other features, which means Gabor energy is more effective in style analysis. In contrast, the NMI of low-resolution also shows the same result. That is to say features extracted from low-resolution paintings have the ability to descriptive painting style, and Gabor energy has the best effect in style analysis.

3.1.3 The Comparison Analysis of van Gogh’s Paintings Based on Gabor Energy

To further test the performance of Gabor energy, we use it to analyze Van Gogh’s style. As stated before, Van Gogh’s paintings are divided into four periods: Paris Period, Arles Period, Saint-Rémy Period, and Auvers Period, but among the four periods, which one has the most representative for Van Gogh’s painting style? In [16], for the high-resolution paintings, it was proved that in the four periods of Van Gogh’s artworks, those of the Paris Period is more similar to the overall artworks. In other words, artworks of the Paris Period can represent Van Gogh’s art style best, followed by Arles Period and Saint-Rémy Period. Can Van Gogh’s low-resolution paintings also get a similar conclusion? We repeat the experiments of [16] using low-resolution paintings. Meanwhile, in order to compare the differences among artists, we added Monet’s paintings. The results are shown in Fig. 3.

Fig. 3.
figure 3

NMI of Gabor energy for high and low resolution paintings

Full size image

From Fig. 3(b), we can see the value of NIM of Paris Period paintings is the highest. It means that the Paris Period is more similar to the overall. And the value of NMI of Van Gogh-Monet group is obviously lower than the other groups, showing the biggest difference. That is to say, paintings of Monet are the most different from the Van Gogh style, and Gabor energy can used to distinguish Van Gogh and Monet’s paintings. These agree with those in [16] based on high-resolution paintings. So features extracted from low-resolution paintings can be used to analyze the painting style.

In conclusion, based on sparse coding, high and low resolution paintings can obtain similar results in analyzing painting style. So, the low-resolution paintings can be used for painting style analysis.

3.2 The Experiments for Painting Style Classification

Because Gabor energy has the best performance for painting style analysis, we use it to classify the painting styles. Van Gogh, Monet, Renoir and Da Vinci’s paintings are used in the style classification experiment. Firstly, we use Van Gogh, Monet, Renoir and Da Vinci’s paintings to train four basis functions and calculate their Gabor energy. We call them style detectors, respectively named by Tv, Tm, Tr and Td. Secondly, select 60 paintings of Van Gogh and 60 paintings of Monet as test images and extract Gabor energy from their basis functions. Thirdly, figure out the values of NMI between the test images and the four detectors respectively. The style of the test painting will be determined by the style of detector which has the highest value of NMI. The classification results are shown in Fig. 4.

Fig. 4.
figure 4

Classification results of Van Gogh and Monet’s test paintings

Full size image

As can be seen from Fig. 4(a), all of the values of NMI between Van Gogh’s test paintings and Van Gogh’s detector are the highest than those of the other three detectors. It means that the entire test paintings of Van Gogh are classified as the Van Gogh style correctly. For example, the values of NMI of Van Gogh’s “self-portrait” with the detectors of Tv, Tm, Tr and Td are 0.79933, 0.77381, 0.75504 and 0.71770. The max value of NMI is from the detector of Tv. For the same, from Fig. 4(b) we can see that, all of the values of NMI between Monet’s test paintings and Monet detector are the highest than those of the other three detectors. So, all of test paintings of Monet are classified as the Monet style correctly too. For Monet’s “haystacks”, the values of NMI of this painting with the detectors of Tv, Tm, Tr and Td are 0.73841, 0.77717, 0.74621 and 0.75314 respectively. The highest value is from the detector of Monet too. These show again that the Gabor energy can describe the artists’ painting style well and can be used to classify the painting style.

4 Conclusion

Based on sparse coding, the contrast experiments are carried out between high and low resolution paintings to analyze painting style in this paper. We obtained that, to some extent, base functions of low-resolution paintings have the similar characteristics with those of high-resolution paintings. Then, features extracted from low-resolution paintings have similar performance with those from high-resolution paintings, and Gabor energy has the most performance in painting style analysis. Last, the classifiers of painting style designed on Gabor energy can recognize Van Gogh and Monet’s paintings correctly. The results of these experiments indicate that features extracted from low-resolution paintings, to some extent, still reflect artists’ painting style, and low-resolution paintings can be used in the study of painting style analysis.

There are some deficiencies in this paper. For example, the features we extracted are from the gray paintings, without considering the color features. While, we only studied the feature of the Gabor energy, other features can be used to do further research.