A Comprehensive Method for Art Style Analysis and Automatic Classification Based on Multi-Channel Feature Extraction and Fusion

This paper focuses on the research of multi-channel feature extraction and fusion methods, primarily due to the complexity and multi-dimensional characteristics of the comprehensive art style of artworks. The style of a painting is the result of the combined effects of various visual elements, such as color tone, brushstroke texture, compositional logic, and line features. A single feature dimension is insufficient to fully capture the essence of the style. For example, the light and shadow color changes of Impressionist works and the dynamic composition of Baroque style belong to different feature dimensions, and color features alone cannot distinguish the rigor of Classicism from the simplification of Neoclassicism. Similarly, relying solely on texture features makes it difficult to differentiate the emotional brushstrokes of Expressionism from the random brushstrokes of Abstract Expressionism. Multi-channel feature extraction can independently model different style dimensions, such as capturing the color rhythm of Monet’s works through the color channel, extracting the brushstroke intensity of Van Gogh through the texture channel, and analyzing Picasso’s deconstruction techniques of Cubism through the shape channel. The fusion process can then integrate the scattered feature information, forming complementary characteristics and constructing a more comprehensive style description system, thus overcoming the limitations of single-feature representations of complex art styles.

There are several challenges in the feature extraction and fusion of the comprehensive art style of artworks. First, the precision of feature extraction is difficult to achieve. The style of a painting is often highly subjective and creative, and variations in the style of different artists within the same school or cross-style fusion lead to blurred feature boundaries, which may result in feature overlap or misjudgment. Second, the adaptability of fusion strategies is a major issue. The feature weights of different style dimensions are dynamic. For instance, in realism, the weight of shape and composition is higher, while in Fauvism, the weight of color predominates. How to dynamically adjust the fusion ratio to accommodate diverse art styles and avoid feature redundancy or loss of critical information is a core problem that needs to be solved. In addition, differences in the medium used to create the artwork and its preservation state may introduce noise features, further increasing the difficulty of feature extraction and fusion.

To address these challenges, this paper designs a comprehensive art style analysis and automatic classification method based on multi-channel feature extraction and fusion. This method includes three key ideas: model lightweight design, multi-scale feature enhancement module, and multi-channel feature fusion module. The details are described as follows. The model structure diagram is shown in Figure 1.

image003.png

2.1 Lightweight design

Comprehensive art style analysis of artworks often needs to be deployed in mobile devices or resource-constrained industrial platforms, where strict limitations exist on model memory, computation speed, and power consumption. For example, when a user takes a photo of an artwork in a gallery and performs real-time style classification, the model needs to complete multi-channel feature extraction and fusion within seconds. High-complexity models may experience delays, reducing user experience. Additionally, industrial-grade art resource management platforms need to process millions of artwork data, and a lightweight design can reduce server computing consumption and minimize hardware investment costs. Therefore, to meet the practical application requirements of artwork analysis scenarios, the comprehensive art style analysis and automatic classification model proposed in this paper adopts a lightweight design.

The core idea for achieving lightweight design in this paper is inspired by the modular architecture and multi-version adaptation strategy of YOLOv5, constructing a network structure that can be flexibly adjusted. The model is divided into two major modules: the basic feature extraction layer and the multi-channel fusion layer. The basic layer adopts a lightweight backbone network, reducing the number of convolutional kernels and simplifying the residual block structure to lower the parameter scale. At the same time, different complexity versions, such as v, t, and l, are provided. For example, when processing simple features like sketches, a lighter version is selected; when analyzing complex art styles with multi-texture fusion like oil paintings, a slightly deeper version is used, achieving "on-demand" allocation of computing resources. In addition, the Ghost module of GhostNet is introduced to replace some standard convolutions, which generates redundant feature maps through cheap operations while maintaining feature extraction capabilities, thus reducing parameter redundancy in the high-dimensional feature processing of artworks.

Furthermore, an efficient aggregation mechanism is embedded into the multi-channel feature fusion stage to avoid any negative impact of lightweight design on art style recognition accuracy. To address the complementarity of art style features, a lightweight attention mechanism is used. When fusing color and texture channel features, higher weight is dynamically assigned to the color features of Impressionist paintings and to the compositional features of Baroque paintings. This reduces the computational cost of irrelevant features while enhancing the representation of key style dimensions. Additionally, the depthwise separable convolution technology of MobileNet is adopted, which decomposes cross-channel convolutions in the multi-channel fusion process into depthwise and pointwise convolutions, thus reducing the computational complexity when different art style features interact. Through these strategies, the model can both maintain the ability to recognize subtle style differences, such as the color rhythm of Monet and the brushstroke intensity of Van Gogh, and meet the real-time analysis and large-scale processing efficiency requirements of mobile devices and industrial platforms, ultimately achieving a balance of "high precision - high efficiency" for comprehensive art style analysis.

2.2 Feature extraction and enhancement

In the feature extraction and enhancement module, a multi-scale and multi-branch feature enhancement strategy is adopted to extract style information, targeting the multi-dimensional characteristics of the comprehensive art style of artworks. Figure 2 shows the structure diagram of the feature extraction and enhancement module. By initializing multiple branches and convolutional layers, style features are captured from different perspectives, such as brushstroke thickness, color block size, and compositional hierarchy. The channel dimensions of small-size sampling blocks are mapped to 1, and this dimensionality reduction operation effectively filters out redundant channel information unrelated to style, such as background noise and canvas texture, thereby accurately mapping foreground style information, such as key brushstroke features and core color regions. Assuming the convolution operation of a 1×1 convolution kernel is represented by CONV_1×1(d), and the feature map for extracting preliminary information is represented by Q, the operation is as follows:

image004.png

Figure 2. Structure diagram of the feature extraction and enhancement module

$Q=C O N V_{1 \times 1}(d)$     (1)

Next, four feature extraction branches are designed, with multiple branches and convolutional layers initialized to analyze the artwork style from different dimensions. Branch 1 uses 1×1 depthwise convolution to extract local detail information of the style. This convolution method reduces the computation and memory access of redundant pixels in the artwork and more efficiently captures spatial-style features, such as point distribution in Pointillism or the brushstroke intensity variations in Expressionism. At the same time, this branch generates an equivalence feature map that retains the key style semantic information and target features, ensuring that local feature extraction does not lose core style clues. Assuming the convolution operation of the 1×1 convolution kernel is represented by DW_CONV_1×1 and the output feature map of branch 1 is represented by A₁, the operation is defined as follows:

$A_1=D W_{-} C O N V_{1 \times 1}(Q)$   (2)

Branches 2, 3, and 4 first use 1×1 convolution to adjust the channel numbers, ensuring that the channel count of the artwork style features processed by each branch remains consistent, providing a unified foundation for subsequent cross-branch feature fusion. Branch 2 combines multi-scale convolution and depthwise convolution to capture and enhance style features from different spatial directions, such as horizontal, vertical, and diagonal, for example, analyzing the compositional blank space in landscape paintings or the color block stacking in oil paintings through multi-dimensional analysis. This ensures that the extracted feature map retains more contextual information of the style, enhancing the completeness of the style description. Assuming the output feature maps of branches 2, 3, and 4 are represented by A₂, A₃, and A₄, the operation is defined as follows:

$A_2=\binom{D W_{-} \operatorname{CONV}_{3 \times 3}}{\left(\operatorname{CONV}_{3 \times 1}\left(\operatorname{CONV}_{1 \times 3}\left(\operatorname{CONV}_{1 \times 1}(Q)\right)\right)\right)}$   (3)

Branch 3 and 4 have a similar feature extraction logic to branch 2, but use different sizes of convolution kernels to adapt to style elements at different scales in the artwork. For example, branch 3 uses a larger convolution kernel to capture the overall color blending effects in Impressionist works, while branch 4 uses a smaller convolution kernel to extract detailed line features in Realism. The differences in the convolution kernels further enrich the extracted style feature dimensions, covering the full-scale style information from macro composition to micro brushstrokes, thus enhancing the recognition ability of diverse art styles. The operations are defined as follows:

$A_3=\binom{D W_{-} \operatorname{CONV}_{5 \times 5}}{\left(\operatorname{CONV}_{5 \times 1}\left(\operatorname{CONV}_{1 \times 5}\left(\operatorname{CONV}_{1 \times 1}(Q)\right)\right)\right)}$    (4)

$A_4=\binom{D W_{-} \operatorname{CONV}_{7 \times 7}}{\left(\operatorname{CONV}_{7 \times 1}\left(\operatorname{CONV}_{1 \mathrm{~b} 7}\left(\operatorname{CONV}_{1 \times 1}(Q)\right)\right)\right)}$  (5)

The style features from the outputs of the three branches are concatenated and further integrated into a unified feature representation via 1×1 convolution. This process encodes the spatial style feature information of the artwork and compresses the channel dimensions, meaning the color, brushstroke, and composition features captured by different branches are associated, and multi-dimensional style information is condensed into a compact and discriminative feature vector. This facilitates cross-channel style comparison and classification by the subsequent fusion module. The operation is defined as follows:

$\begin{aligned} & A_{C A}=C O N C A T\left[a_2, a_3, a_4\right] \\ & A_{F U}=C O N V_{1 \times 1}\left(A_{C A}\right) \\ & B=\operatorname{RELU}\left(\beta \cdot A_{F U}+A_1\right)\end{aligned}$  (6)

Next, the scale features from the three branches are concatenated along the channel dimension, forming a robust feature representation that covers the multi-layered style of the artwork. Through a shortcut connection, the original style features are added to the fused feature map, followed by the application of the ReLU activation function. This not only preserves the key style information initially extracted, such as the signature color tone, but also strengthens the style differences through the fused features. Here, $A_{C A}$ represents the feature concatenation operation, $A_{F U}$ represents the fused style feature map, and $\beta$ as a scaling factor avoids model instability caused by excessive fluctuation in style features during training, ensuring more stable feature learning for complex art styles.

2.3 Multi-channel feature fusion

In the multi-channel feature fusion module, high-level feature maps and low-level feature maps carry different dimensional information about the comprehensive art style of the artwork. High-level feature maps focus on the overall style attributes of the artwork, such as the light and shadow atmosphere of Impressionism and the symmetrical composition of Classicism, representing macro features; low-level feature maps focus on local details, such as the swirling brushstrokes in Van Gogh’s works or the delicate lines in fine-line paintings, representing micro elements. By fusing the outputs of these two types of feature maps, the semantic expression ability of small-scale style elements in the artwork can be effectively enhanced. For example, when analyzing Romanticism artworks that blend delicate brushstrokes and grand compositions, the fusion operation can relate the agility of the local brushstrokes with the emotional tension of the overall composition, avoiding style information fragmentation caused by relying solely on a single feature layer.

This module is used to fuse multiple channels of the artwork feature maps, adopting the Squeeze-and-Excitation (SE) module mechanism to dynamically weight the channels of each input feature map. Figure 3 shows the structure of the SE module. For artworks of different styles, the importance of each channel feature varies significantly. For example, when analyzing Fauvist works, the high saturation features in the color channel are crucial; whereas, when analyzing Cubism works, the shape deconstruction features in the corresponding channel are more important. The weighting mechanism enhances the expression of important feature channels while suppressing interference from irrelevant channels, such as highlighting the brushstroke intensity channel of Expressionism and diminishing the redundant background texture channel in Realism. Ultimately, the weighted feature maps are concatenated along the channel dimension, forming a fused feature that combines multi-dimensional integrity with the prominence of key features, providing a more accurate basis for style classification.

image011.png

Figure 3. Structure diagram of SE module

First, the feature maps of the artwork extracted from multiple channels are input to obtain the style feature information of each channel, such as the hue distribution in the color channel and the brushstroke density in the texture channel. A global average pooling operation is applied to each input feature map, generating the corresponding channel descriptor. For example, performing global average pooling on the color channel will obtain the average hue and saturation of that channel, and processing the texture channel will yield the average brushstroke thickness. This condenses the global style features of each channel into a compact vector, providing a quantitative basis for subsequent weight assignment. Assuming the global average pooling operation is represented by GAP(·) and the corresponding channel descriptor is represented by t_v, the operation is as follows:

$t_v=G A P\left(A_v\right)=\frac{1}{G \times Q} \sum_{u=1}^G \sum_{k=1}^Q A_u[:,:, u, k]$   (7)

Further, the pooled descriptors from multiple channels are concatenated along the channel dimension, forming a joint descriptor t that covers multiple aspects of the artwork’s style. This joint descriptor integrates the global feature information of all channels, such as color, texture, and composition. Specifically, for an Impressionist landscape painting, t includes not only the light and shadow variation features of the color channel but also the fragmented brushstroke features of the texture channel and the horizon layout features of the composition channel, thereby constructing a comprehensive quantitative representation of the artwork’s style. This provides a complete feature foundation for subsequent weight learning. Its expression is:

$t=\operatorname{CONCAT}\left[t_1, t_2, \ldots, t_u\right]$     (8)

This module uses a two-layer fully connected network to generate the weight coefficients for each channel. After obtaining the concatenated joint descriptor s, the first fully connected network integrates the multi-dimensional style information, then the second fully connected network refines the feature associations. The output is then mapped to the 0-1 range using a Sigmoid activation function, combined with the ReLU activation function to suppress invalid weights. For example, when processing Abstract works, the network will automatically learn to increase the weight of the shape deconstruction channel while reducing the weight of the figurative color channel, thereby adaptively enhancing key style features. Assuming the feature weights of the first and second layers are represented by q₁ and q₂, the generated weight coefficient is represented by q, the operation is as follows:

$q=\operatorname{SIGMOID}\left(\operatorname{RELU}\left(T \cdot q_1\right) \cdot q_2\right)$   (9)

Finally, the weight reshaping operation is performed. The generated weight vector is reshaped to match the dimensions of the input feature maps and applied to the channels of multiple input feature maps. For the color channel of the artwork, the reshaped weight will strengthen the features in high-saturation areas; for the texture channel, the weight will highlight the signature brushstrokes. Assuming the feature maps input from different channels are represented by A_u, the weighted feature maps are represented by A'_u, and the number of fused channels is represented by Z_e. DIM = f specifies the concatenation dimension, and B represents the concatenated feature map, as follows:

$\begin{aligned} & Q_{\text {RESHAPE }}=q \cdot R E S H A P E\binom{V, Z_1+Z_2}{+\ldots+Z_v, 1,1} \\ & A_u^{\prime}=A_u \otimes Q_{\text {RESHAPE }}\left[:, Z_e,:,:\right] \\ & B=C O N C A T\left[A_1^{\prime}, A_2^{\prime}, \ldots, A_v^{\prime}, D I M=f\right]\end{aligned}$  (10)

This precisely matched weight application mechanism can retain the integrity of multi-channel features while selectively amplifying key clues for style recognition, effectively suppressing background noise or interference from secondary features, and ultimately improving the accuracy and robustness of the comprehensive style classification. Figure 4 shows the structure diagram of the multi-channel feature fusion module.

image012.png

Figure 4. Structure diagram of the multi-channel feature fusion module

2.4 Method flow

Based on the proposed model, the process for performing comprehensive art style analysis and automatic classification on artworks is as follows: First, the image of the artwork to be analyzed undergoes preprocessing, including operations such as size standardization and color space unification, to adapt it to the input requirements of the model. The preprocessed image then enters the feature extraction and enhancement module, where features are extracted through a multi-scale, multi-branch structure. The four branches capture style information from different dimensions: Branch 1 uses 1×1 depthwise convolution to extract microscopic features such as local brushstrokes and color dot distribution, while Branches 2-4 use multi-scale convolutions with varying kernel sizes and depthwise convolutions to extract macroscopic and mesoscopic features from aspects like color tone, composition layout, and texture thickness. Redundant information is filtered through channel dimension reduction, and key style clues are enhanced through feature enhancement processing. Next, the extracted multi-channel features enter the fusion module. High-level feature maps representing the overall composition and stylistic atmosphere are subjected to global average pooling, as are low-level feature maps representing local brushstrokes and fine details. The global average pooling generates style descriptors for each channel. These descriptors are then merged into a joint descriptor. Using a two-layer fully connected network, combined with Sigmoid and ReLU activation functions, dynamic weights are generated. After reshaping, the weights are applied to the corresponding channels to enhance important features. Finally, the weighted feature maps are concatenated along the channel dimension, forming a compact feature vector that fuses multi-dimensional style information. The fused feature vector is then input into a lightweight model structure, which utilizes its efficient inference capability to match the style category and output the specific style of the artwork, such as Impressionism, Baroque, Abstract, etc., completing the entire analysis and classification process. The proposed model achieves comprehensive capture of the multi-dimensional features of artwork style through multi-channel extraction and enhancement. It also highlights key style clues through dynamic weighted fusion, while ensuring analysis efficiency and accuracy through lightweight design, achieving end-to-end processing from the raw image to the art style category.