A Deep Image Representation and Temporal Correlation Modeling Approach for Film Scene Style Evolution Analysis

2.1 Overview of the overall general framework

This paper proposes an end-to-end multi-task learning general framework, with the core objective of solving the common computer vision challenge of structured representation and dynamic modeling of aesthetic visual styles. The framework achieves cross-domain adaptation through modular and configurable design, seamlessly integrating temporal visual data such as films, painting sequences, and architectural videos, and using a unified technical paradigm to complete precise style characterization and capture evolution patterns, overcoming the limitations of traditional domain-specific methods.

tu_pian_139.png

Figure 1. Overall architecture of the film style evolution analysis method

Figure 1 shows the overall architecture of the film style evolution analysis method. The framework adopts an "end-to-end process" with the following general flow: "Data Preprocessing → Deep Image Representation Extraction → Scene-guided Temporal Correlation Modeling → Style Evolution Analysis → Multi-task Optimization." Each module collaborates efficiently through standardized interfaces. The specific process is as follows: The data preprocessing module is responsible for the standardization and quality screening of temporal visual data and adaptive processing for different domains: film data adopts frame rate adaptive sampling and visual saliency detection to select key frames and remove invalid frames; painting sequences are sorted by creation timestamp and standardized; architectural videos optimize space scene continuity and subframe extraction. All data is uniformly processed by size normalization, 3×3 Gaussian filtering for denoising, and pixel value normalization, outputting a temporally consistent image sequence. The deep image representation extraction module uses a "local texture enhancement - global semantic encoding - style prototype adaptation" three-level architecture, integrating low-level texture, mid-level semantics, and high-level style information, and outputs a 1024-dimensional deep feature vector. The scene-guided temporal correlation modeling module adapts to the film's narrative scenes, the painting's creation periods, and the architectural space scenes using configurable interfaces. It uses hierarchical attention masking to capture dual temporal correlations: full attention interaction within local scenes ensures coherence, and sparse attention across scenes only links key nodes, enhancing core dependencies and reducing redundant computation. The style evolution analysis module performs three core functions: smoothing and change-point detection to locate style change points, combining a style prototype library to label segments, and calculating the temporal synchronization coefficient between style changes and scene nodes to quantify evolution patterns. The multi-task optimization module uses a composite loss function that integrates prototype clustering loss, temporal reconstruction loss, change point detection loss, and temporal correlation loss to jointly optimize all link parameters end-to-end. Dynamic weight allocation achieves multi-objective coordination, where the prototype clustering loss ensures clustering performance, the temporal reconstruction loss strengthens temporal coherence, and the detection and correlation losses optimize core task performance.

2.2 Data preprocessing module

The data preprocessing module is designed with the core principles of "universal strategy standardization" and "domain adaptation customization" to optimize quality and structurally convert temporal visual data, providing a unified and high-quality data foundation for subsequent deep representation extraction and temporal modeling. The general preprocessing flow includes three core operations: sampling screening, denoising enhancement, and standardization. The key parameters and core formulas are as follows: The denoising step uses 3×3 Gaussian filtering, where the core is to generate a filter kernel using a 2D Gaussian function to apply weighted smoothing to image pixels, effectively suppressing high-frequency noise while retaining style texture details. The Gaussian filter kernel's mathematical expression is:

$G(x, y)=\frac{1}{2 \pi \sigma^2} \exp \left(-\frac{x^2+y^2}{2 \sigma^2}\right)$        (1)

where, (x, y) are the coordinates of pixels relative to the center of the filter kernel, and σ = 1.0 is the Gaussian standard deviation. This parameter is determined through grid search and can balance the denoising effect and detail retention. The standardization step uniformly adjusts the image size to 224×224 or 384×384 and uses the default CLIP normalization method to eliminate pixel scale differences. The specific formula is:

${{\text{I}}_{\text{norm}}}\text{=}\frac{\text{I-127}\text{.5}}{\text{127}\text{.5}}$         (2)

where, I is the original pixel value and I_norm is the normalized pixel value. This normalization directly adapts to the feature distribution of pre-trained models, enhancing feature learning efficiency.

The domain adaptation processing optimizes the core logic based on the data characteristics of different carriers, focusing on preserving key information and temporal integrity. The key frame extraction for film data uses a "frame rate adaptive sampling + ITTI visual saliency detection" composite strategy: first, an initial sampling is done at 2 fps, and then the ITTI algorithm calculates the image saliency map to select the top 30% of frames with the highest saliency as candidate key frames. The core formula for calculating saliency values in the ITTI algorithm is:

$\text{S=}{{\text{ }\!\!\omega\!\!\text{ }}_{\text{c}}}{{\text{S}}_{\text{c}}}\text{+}{{\text{ }\!\!\omega\!\!\text{ }}_{\text{l}}}{{\text{S}}_{\text{l}}}\text{+}{{\text{ }\!\!\omega\!\!\text{ }}_{\text{o}}}{{\text{S}}_{\text{o}}}$          (3)

where, S_c, S_l, and S_o are the saliency maps for color, luminance, and orientation features, and ω_c=ω_l=ω_o=1/3 are the weights for each channel, ensuring a balanced contribution from multi-dimensional visual information. Additionally, meaningless frames are removed: frames such as black screens and overexposed frames are eliminated using luminance thresholds, and frames with large areas of subtitles are removed using text detection algorithms. For painting sequences, sampling and sorting are based on the creation timestamps, and multiple works from the same period are uniformly sampled at equal intervals. Architectural videos combine scene segmentation results to perform dense sampling at scene transition points to ensure that style change details are not lost.

Temporal and scene alignment is the final step of preprocessing and provides the core constraints for subsequent scene-guided temporal modeling. Its core is to construct a unified temporal coordinate system and bind scene semantic labels. Universal temporal alignment is achieved through timestamp mapping. For data with native timestamps, such as film and architectural videos, the temporal index is directly established using the original timestamp t. For data without native timestamps, such as painting sequences, the creation year/month is converted into a continuous temporal scale t′. The mapping formula is:

$\text{t }\!\!'\!\!\text{ =}\frac{\text{Y-}{{\text{Y}}_{\text{0}}}}{{{\text{Y}}_{\text{max}}}\text{-}{{\text{Y}}_{\text{0}}}}\text{ }\!\!\times\!\!\text{ T}$           (4)

where, Y is the creation year of the painting, Y₀ is the earliest creation year in the dataset, Y_max is the latest creation year, and T is the total length of the temporal sequence, ensuring that the temporal relationship between works from different periods is continuous and comparable. Scene labels are defined according to domain adaptation: for film data, they are associated with the script's narrative structure and labeled as plot units; for architectural videos, they are based on spatial functions and labeled as architectural space regions; for painting sequences, they are based on art historical periods and labeled as creation periods. Finally, the output is a "temporal image sequence - temporal index - scene label" triplet, ensuring the unified constraint of temporal and semantic relationships.

2.3 Deep image representation module

The deep image representation module is the core of the framework for accurately encoding aesthetic visual styles. The design core is to construct a "global semantics - local texture - style prototype" triune multi-dimensional fusion representation, which adapts to cross-domain data through a universal architecture and enhances the targeted representation of style attributes through hierarchical feature encoding. Figure 2 shows the architecture of the deep image representation module. The basic visual feature extraction uses the CLIP-ViT model, which, through large-scale image-text cross-modal pretraining, has strong general semantic representation capabilities and can effectively capture high-level semantic information from visual data across domains, significantly outperforming single-modal pre-trained models like ResNet in cross-domain transfer. The input is the preprocessed standardized image, and after being encoded by CLIP-ViT, a 768-dimensional global semantic feature vector is output, which encodes the overall content semantics of the image, providing a semantic anchor point for style representation and ensuring that the style analysis remains within the content context.

tu_pian_140.png

Figure 2. Architecture of the deep image representation module

The local texture feature enhancement module adopts a lightweight 4-layer convolution + pooling structure, specifically designed to extract low-level texture and color details for aesthetic style, compensating for CLIP-ViT’s insufficient capture of local details. The specific structural parameters are designed as follows: The first layer convolution uses a 3×3 convolution kernel with a stride of 1, outputting 64 channels, with the activation function being ReLU; the second layer convolution uses a 3×3 convolution kernel with a stride of 1, outputting 128 channels, followed by 2×2 max pooling; the third layer convolution uses a 3×3 convolution kernel with a stride of 1, outputting 192 channels; the fourth layer convolution uses a 3×3 convolution kernel with a stride of 1, outputting 256 channels, followed by 2×2 max pooling. This structure enhances feature expression capabilities by progressively increasing the number of channels, while pooling operations reduce spatial dimensions and retain key details, outputting a 256-dimensional local texture feature vector. The design principle is that aesthetic style differences are often reflected in details such as local color distribution and texture, and this lightweight structure ensures feature extraction capabilities while avoiding overfitting and computational redundancy brought by complex networks.

The style prototype adaptation network (APN) is the core universal module for achieving targeted style representation through prototype learning, reinforcing the encoding of multi-dimensional aesthetic styles. It mainly includes three core steps: style prototype memory bank construction, cross-attention matching, and prototype activation feature generation. The style prototype memory bank is initialized with 30 style prototypes, covering four general aesthetic dimensions: color, composition, lighting, and movement. The prototype initialization uses the k-means clustering algorithm to cluster cross-domain style sample features, and a dynamic expansion mechanism is designed to adaptively add prototypes based on new domain data, ensuring cross-domain adaptability. The cross-attention mechanism calculates the similarity between the current frame’s features and each style prototype in the memory bank, using cosine similarity to quantify the matching degree, with the core formula as:

${{\text{ }\!\!\alpha\!\!\text{ }}_{\text{i}}}\text{=}\frac{\text{exp(cos(}{{\text{f}}_{\text{local}}}\text{,}{{\text{p}}_{\text{i}}}\text{))}}{\mathop{\sum }_{\text{j=1}}^{\text{N}}\text{exp(cos(}{{\text{f}}_{\text{local}}}\text{,}{{\text{p}}_{\text{j}}}\text{)}}$          (5)

where, α_i is the weight of the i-th style prototype, f_local is the local texture feature, p_i is the i-th style prototype, and N=30 is the total number of prototypes. After obtaining the prototype distribution weights through this formula, the residual features r_i=f_local−p_i are calculated, encoding the style difference information.

Prototype activation feature generation is achieved by weighted summation, aggregating the features of each style prototype according to similarity weights to obtain a 256-dimensional prototype activation feature vector, with the formula:

${{\text{f}}_{\text{proto}}}\text{=}\underset{\text{i=1}}{\overset{\text{N}}{\mathop \sum }}\,{{\text{ }\!\!\alpha\!\!\text{ }}_{\text{i}}}\cdot {{\text{p}}_{\text{i}}}$           (6)

This feature vector condenses the matching information between the current frame and general style prototypes, directly encoding high-level style attributes, which enhances the style discrimination capability of the representation. The feature fusion step uses a "concatenation - fully connected" universal strategy. First, the CLIP global semantic features, local texture features, prototype activation features, and prototype residual features are concatenated to obtain a 1536-dimensional concatenated feature vector (768 + 256×3). Then, a fully connected layer maps it to a 1024-dimensional feature vector. The mathematical expression for the fusion process is:

${{\text{f}}_{\text{fusion}}}\text{=ReLU(W}\cdot \text{ }\!\![\!\!\text{ }{{\text{f}}_{\text{global}}}\text{,}{{\text{f}}_{\text{local}}}\text{,}{{\text{f}}_{\text{proto}}}\text{,}{{\text{f}}_{\text{res}}}\text{ }\!\!]\!\!\text{ +b)}$             (7)

where, W is the weight matrix of the fully connected layer, b is the bias term, and [ ] denotes feature concatenation. The fused 1024-dimensional feature combines the correlation of global semantics, the richness of local details, and the targeted style attributes, enabling precise encoding of aesthetic visual styles across domains and providing a high-quality feature foundation for subsequent temporal modeling.

2.4 Scene-guided temporal correlation modeling module

The core goal of the scene-guided temporal correlation modeling module is to accurately capture the long-term temporal dependencies of aesthetic styles while enhancing the targeting and efficiency of temporal modeling through scene semantic constraints. Its design follows the principle of "universal architecture + scene adaptation," which can seamlessly interface with temporal visual data from different domains. Figure 3 shows the architecture of the scene-guided temporal correlation modeling module. The input layer construction is the foundation of temporal modeling. First, the deep image representation is arranged in temporal order, forming a temporal feature sequence F=[f₁,f₂,...,f_T]∈R^T×1024, where f_t is the fused feature of the t-th frame. To address the issue that Transformer is insensitive to temporal order, sinusoidal position encoding is introduced to inject temporal information. The calculation formula for the position encoding P_t∈R¹⁰²⁴ is:

${{\text{P}}_{\text{t,2k}}}\text{=sin}\left( \frac{\text{t}}{\text{1000}{{\text{0}}^{\text{2k}}}\text{/1024}} \right)\text{, }\!\!~\!\!\text{ }{{\text{P}}_{\text{t,2k+1}}}\text{=cos}\left( \frac{\text{t}}{\text{1000}{{\text{0}}^{\text{2k}}}\text{/1024}} \right)$           (8)

where, k is the feature dimension index. This encoding is generated by sine/cosine functions with different frequencies to distinguish the feature differences at different temporal positions. Meanwhile, to integrate scene semantic constraints, scene type labels are encoded into a 1024-dimensional scene type embedding S_t using an embedding layer. The final input features are fused by feature addition, with the fusion formula as:

${{\text{X}}_{\text{t}}}\text{=}{{\text{f}}_{\text{t}}}\text{+}{{\text{P}}_{\text{t}}}\text{+}{{\text{S}}_{\text{t}}}$

This results in an input sequence X=[X₁,X₂,...,X_T]∈R^T×1024, which provides high-quality input for subsequent attention modeling.

tu_pian_141.png

Figure 3. Architecture of the scene-guided temporal correlation modeling module

The hierarchical attention mask mechanism is the core innovation of the module. Through a dual-layer design of "local full attention + cross-scene sparse attention," it captures temporal continuity and strengthens key node associations while reducing redundant computation. The local scene mask constructs a full attention mask for the frame sequence within the same scene, with the mask matrix M_local∈R^T×T defined as: If frame t and frame t' belong to the same scene, then M_local[t,t′]=1, otherwise 0. The core principle of this design is that styles within the same scene have strong continuity, and full attention interaction can fully capture the fine-grained style dependencies between frames, ensuring the integrity of temporal modeling within the scene. The cross-scene global mask sparsifies the constraints on frame interactions between different scenes. The mask matrix M_global∈R^T×T is mathematically defined as:

$M_{\text {global }}\left[t, t^{\prime}\right]=\left\{\begin{array}{l}1, \text { if } t^{\prime} \in\left\{t_{\text {prev-end }}, t_{\text {curr-start }}\right\} \\ 0, \text { other cases }\end{array}\right.$          (9)

where, t_prev-end is the index of the last frame of the previous scene, and t_curr-start is the index of the first frame of the current scene. This rule ensures that cross-scene interactions focus only on the key nodes at scene transitions, enhancing the temporal correlation of key turning points in style evolution while reducing the attention computation complexity from O(T²) to O(T), significantly improving computational efficiency. Since the scene definition can be adapted to different domains via a configurable interface, this mask mechanism has inherent cross-domain universality.

The Transformer encoder is responsible for the deep encoding of temporal features, using a general structure with 6 stacked encoder layers. The core parameter configuration is: each layer contains 8-head self-attention mechanisms with a head dimension of 128; the feed-forward network uses a 2-layer fully connected structure, with a middle dimension of 2048 and the ReLU activation function; each layer is equipped with layer normalization and residual connections to ensure training stability. The encoding process is as follows: The input sequence first undergoes masking with both the local scene mask and the cross-scene global mask to constrain the interaction range of self-attention. The final attention mask is M=M_local∨M_global, where "∨" represents logical OR. Then, a multi-head self-attention mechanism aggregates effective temporal context information, followed by a feed-forward network for nonlinear feature transformation. Finally, residual connections and layer normalization are applied to output the layer features. The progressive encoding of the 6-layer encoder gradually strengthens the ability to fuse long-term temporal contexts, ultimately outputting a 1024-dimensional style temporal representation sequence Z=[Z₁,Z₂,...,Z_T]∈R^T×1024, which retains frame-level style details and encodes long-term temporal dependencies, providing core temporal feature support for subsequent style evolution analysis.

2.5 Style evolution analysis module

The core goal of the style evolution analysis module is to achieve precise localization of style changes, label annotation, and temporal correlation quantification based on the encoded style temporal representation. It also improves result reliability through cross-modal auxiliary validation. The overall design follows the universal logic of "lightweight decoding + precise quantification + auxiliary validation" to adapt to the evolution analysis needs of temporal visual data from different domains. The style evolution decoding process adopts a progressive procedure of "temporal convolution smoothing - mutation detection - label matching," with the core component being the lightweight temporal convolution network (TCN). Its design aims to smooth the noise in temporal representations and strengthen the continuity of style evolution while retaining key change information. The TCN uses a 3-layer causal convolution structure, with the following parameters: each layer uses a 3×1 convolution kernel, dilation rates of 1, 2, and 4, respectively, and the output channel number is 1024, which is consistent with the input temporal representation dimension. The ReLU activation function is used, and layer normalization is applied to stabilize training.

The smoothing principle of the TCN is to only use current and past frame features for calculation through causal convolutions, avoiding future information leakage. At the same time, dilated convolutions efficiently capture long-term temporal dependencies, making the output smoothed temporal representation Z′=[Z₁′,Z₂′,...,Z_T′] more aligned with the true trend of style evolution. Based on the smoothed representation, a threshold-based mutation detection algorithm is used to locate style change points. The core judgment rule is: calculate the L2 distance ${{\text{d}}_{\text{t}}}\text{=}{{\text{Z}}_{\text{t}}}\text{ }\!\!'\!\!\text{ -}{{\text{Z}}_{\text{t-1}}}{{\text{ }\!\!'\!\!\text{ }}_{\text{2}}}$(t ≥ 2), and if d_t>τ, the t-th frame is determined to be a style change point, where τ is an adaptive threshold determined by the dataset's statistical distribution. After locating the change points, the frame sequences between adjacent change points are divided into style segments, and similarity matching with the style prototype memory bank is performed to assign corresponding style labels to each segment. The final output is a sequence of style change points and style label sequences.

The temporal correlation quantification step aims to quantify the temporal synchronization between style changes and scene nodes, providing a quantitative basis for in-depth interpretation of style evolution patterns. First, the time difference between the style change point and the nearest scene node is calculated. The general calculation method is: let the style change point index be t_c, and the nearest scene node indexes before and after it be ${{\text{t}}_{\text{s1}}}$ and ${{\text{t}}_{\text{s2}}}$, respectively, then the time difference $\text{ }\!\!\Delta\!\!\text{ t=min( }\!\!|\!\!\text{ }{{\text{t}}_{\text{c}}}\text{-}{{\text{t}}_{\text{s1}}}\text{ }\!\!|\!\!\text{ , }\!\!|\!\!\text{ }{{\text{t}}_{\text{c}}}\text{-}{{\text{t}}_{\text{s2}}}\text{ }\!\!|\!\!\text{ )}$. To further quantify overall synchronization, the Pearson correlation coefficient is used to measure the linear correlation between the style change point sequence and the scene node sequence. Let the temporal distribution vector of style change points be C, and the temporal distribution vector of scene nodes be S, then the Pearson correlation coefficient formula is:

$\text{r=}\frac{\mathop{\sum }_{\text{t=1}}^{\text{T}}\text{(}{{\text{C}}_{\text{t}}}\text{-}\overset{}{\mathop{\text{C}}}\,\text{)(}{{\text{S}}_{\text{t}}}\text{-}\overset{}{\mathop{\text{S}}}\,\text{)}}{\sqrt{\mathop{\sum }_{\text{t=1}}^{\text{T}}{{\text{(}{{\text{C}}_{\text{t}}}\text{-}\overset{}{\mathop{\text{C}}}\,\text{)}}^{\text{2}}}}\sqrt{\mathop{\sum }_{\text{t=1}}^{\text{T}}{{\text{(}{{\text{S}}_{\text{t}}}\text{-}\overset{}{\mathop{\text{S}}}\,\text{)}}^{\text{2}}}}}$             (10)

where, $\bar{C}$ and $\bar{S}$ are the mean values of C and S, respectively. The value of r ranges from [-1, 1]. The closer r is to 1, the stronger the synchronization between style changes and scene transitions; conversely, the weaker the synchronization.

The cross-modal auxiliary validation step uses a general fusion approach with attention weighting to adapt to auxiliary modality information, such as text and audio, improving the reliability of the style evolution analysis results while ensuring the core focus remains on image temporal analysis. The specific process is as follows: extract the feature sequence of the auxiliary modality and map it to a 1024-dimensional space using a linear layer to match the dimensionality of the image temporal representation. A cross-modal attention mechanism is introduced to calculate the matching weight between image features and auxiliary modality features. The weight formula is: ${{\text{ }\!\!\beta\!\!\text{ }}_{\text{t}}}\text{=softmax(Z}_{\text{t}}^{\text{ }\!\!'\!\!\text{ }}\cdot \text{A}_{\text{t}}^{\text{T}}\text{)}$, where A_t is the auxiliary modality feature at the t-th frame. The auxiliary modality features are then weighted and fused into the image temporal representation, resulting in the fused feature ${{\text{Z}}_{\text{t}}}\text{ }\!\!'\!\!\text{  }\!\!'\!\!\text{ =}{{\text{Z}}_{\text{t}}}\text{ }\!\!'\!\!\text{ +}{{\text{ }\!\!\beta\!\!\text{ }}_{\text{t}}}\cdot {{\text{A}}_{\text{t}}}$. Based on the fused features, change point detection is performed again. If the overlap rate with the image-only analysis result is ≥85%, the analysis result is verified as reliable; if the overlap rate is low, it is only used as a reference correction without altering the core image temporal analysis conclusions, ensuring the module’s reliance on image data and the stability of the results. Figure 4 shows the architecture of the style evolution analysis module.

tu_pian_142.png

Figure 4. Architecture of the style evolution analysis module

2.6 End-to-end optimization strategy

The core of the end-to-end optimization strategy is to achieve global optimal convergence of the parameters across the framework modules through a multi-task collaborative loss function and a staged training mechanism, while ensuring the flexibility of cross-domain adaptation. The design of the multi-task loss function follows the principle of "optimizing core tasks separately, collaboratively balancing overall performance," integrating four main loss terms: prototype clustering loss, temporal reconstruction loss, change point detection loss, and temporal correlation loss. Dynamic weight allocation is used to realize multi-objective collaborative optimization. The total loss function expression is:

${{L}_{\text{total}}}\text{=}{{\text{ }\!\!\lambda\!\!\text{ }}_{\text{1}}}{{L}_{\text{proto}}}\text{+}{{\text{ }\!\!\lambda\!\!\text{ }}_{\text{2}}}{{L}_{\text{rec}}}\text{+}{{\text{ }\!\!\lambda\!\!\text{ }}_{\text{3}}}{{L}_{\text{cp}}}\text{+}{{\text{ }\!\!\lambda\!\!\text{ }}_{\text{4}}}{{L}_{\text{corr}}}$            (11)

where, λ₁, λ₂, λ₃, λ₄ are the weights of each loss term, determined through grid search on the validation set as λ₁=0.3, λ₂=0.2, λ₃=0.3, λ₄=0.2. This allocation ratio balances the quality of style representation and the accuracy of evolution analysis.

The general mathematical definitions and optimization goals of each loss term are as follows: Prototype clustering loss is based on contrastive learning, strengthening the discrimination of style representations by reducing the distance of features within the same style and increasing the distance of features from different styles. The formula is:

${{L}_{\text{proto}}}\text{=-}\frac{\text{1}}{\text{N}}\underset{\text{i=1}}{\overset{\text{N}}{\mathop \sum }}\,\text{log}\left( \frac{\text{exp(sim(}{{\text{f}}_{\text{i}}}\text{,}{{\text{p}}_{{{\text{y}}_{\text{i}}}}}\text{)/ }\!\!\tau\!\!\text{ )}}{\mathop{\sum }_{\text{j=1}}^{\text{M}}\text{exp(sim(}{{\text{f}}_{\text{i}}}\text{,}{{\text{p}}_{\text{j}}}\text{)/ }\!\!\tau\!\!\text{ )}} \right)$            (12)

where, f_i is the deep feature of the i-th sample, ${{\text{p}}_{{{\text{y}}_{\text{i}}}}}$ is the prototype vector of the style to which the sample belongs, ${{\text{p}}_{\text{j}}}$ is the j-th prototype in the memory bank, sim( , ) is the cosine similarity, τ=0.07 is the temperature parameter, N is the batch size, and M is the total number of prototypes. Temporal reconstruction loss ensures temporal continuity by reconstructing the input temporal feature sequence using a decoder. It is defined using mean squared error (MSE):

$L_{\text {rec }}=\frac{1}{T \cdot D} \sum_{t-1}^T\left\|f_t-\hat{f}_t\right\|_2^2$          (13)

where, $\hat{f}_t$ is the reconstructed feature output by the decoder, T is the temporal sequence length, and D=1024 is the feature dimension. Change point detection loss is used for the binary classification task of "change point / non-change point," optimized by cross-entropy loss to improve detection accuracy:

$L_{c p}=-\frac{1}{T} \sum_{t=1}^T\left(y_t \log \hat{y}_t+\left(1-y_t\right) \log \left(1-\hat{y}_t\right)\right.$       (14)

where, y_t∈{0,1} is the true label for the t-th frame, and $\hat{y}_t$ is the predicted probability of the change point. Temporal correlation loss uses MSE to optimize the prediction accuracy of the Pearson correlation coefficient:

$L_{\text {corr }}=\left\|\hat{r}-r_{g t}\right\|_2^2$           (15)

where, $\hat{r}$ is the predicted temporal synchronization correlation coefficient, and r_gt is the true correlation coefficient.

The training strategy adopts a two-stage "pre-training - fine-tuning" model, combining an adaptive optimizer and learning rate scheduling to ensure training stability and convergence efficiency. The basic optimization configuration is as follows: the AdamW optimizer is used with an initial learning rate of 10⁻⁴, weight decay of 10⁻⁵, and a batch size of 32. Learning rate scheduling follows a cosine annealing strategy with a period of 10 training epochs, and the minimum learning rate is 10⁻⁶. This periodic adjustment of the learning rate avoids local optima and accelerates global convergence. The core advantage of the two-stage training is to optimize the goals in layers: during the pre-training phase, only the deep image representation module and style prototype memory bank are trained, freezing the temporal modeling and evolution analysis modules. The prototype clustering loss is optimized alone to first complete basic clustering of style features, forming a stable foundation for style representation and avoiding interference from multi-module collaborative training. In the fine-tuning phase, all modules are unfrozen, and end-to-end joint training is performed using the total loss function, optimizing the collaborative adaptation between modules and improving overall task performance.

The domain adaptation fine-tuning strategy further ensures the framework’s cross-domain universality. The core is to adjust training parameters based on the characteristics of new domain data: first, freeze the CLIP-ViT pre-trained backbone network to avoid disrupting pre-trained semantic knowledge; second, reduce the initial learning rate to 5×10⁻⁵ and use a smaller learning rate for parameter updates to reduce overfitting risks caused by domain differences; and finally, dynamically adjust the number of training epochs based on the new domain data size. When the data volume is small, an early stop strategy is applied, using the peak validation set performance as the stopping criterion. This staged and domain-adaptive strategy ensures the training stability of the base model and improves the flexibility of cross-domain migration, enabling efficient convergence and precise modeling on temporal visual data from different domains.

A deep interpretation of the experimental results reveals the scientific rationale behind the core design of this method. The innovative architecture, which integrates multi-dimensional fusion representation and scene-guided temporal modeling, provides key technical insights for visual aesthetic computing. The three-in-one fusion representation of "local texture-global semantics-style prototype" precisely aligns with the non-rigid and multi-dimensional characteristics of aesthetic style by collaboratively encoding low-level texture details, mid-level content semantics, and high-level style attributes. This is the core reason why it adapts to various fields such as film, painting, and architecture. In the experiments, this representation significantly improved the differentiation of non-rigid styles, confirming the critical role of multi-dimensional feature fusion in overcoming the limitations of traditional single-feature methods. The hierarchical attention mask mechanism optimizes temporal modeling by guiding scenes. Compared to the generic full-attention mechanism with indiscriminate interactions, the local full-attention mechanism ensures the capture of scene coherence within scenes and sparse cross-scene attention focuses on key nodes. This design reduces redundant computational overhead while enhancing the modeling of the core dependencies of style evolution, significantly improving change point detection accuracy and temporal correlation quantification. This general architecture also endows the method with excellent cross-domain generalization ability. Its precise representation of universal aesthetic dimensions and the general temporal modeling logic guided by scenes provide an important reference for the design of general methods in visual aesthetic computing, advancing research in non-rigid, high-level visual attribute modeling. At the same time, the method's multi-domain influence is significant. It improves the technical system of long video structured understanding in the field of computer vision; in the film industry, its potential for real-time analysis can support editing assistance, trailer generation, and shooting style monitoring; in the digital humanities, it provides objective tools for large-scale quantitative research on film and cultural heritage, helping verify art history theories; and the cross-domain verification results in fields such as painting and architecture demonstrate its broad application potential.

However, the method still has limitations that need optimization, which also point to future core research directions. The current model's high computational complexity limits its application in ultra-long videos and large-scale image sequence analysis. Future work should explore sparse Transformer architectures based on local windows and dynamic pruning strategies for the style prototype library to reduce computational overhead while ensuring performance. The current style prototype library does not cover niche and emerging aesthetic styles sufficiently. Future work could combine unsupervised contrastive learning and open vocabulary learning to enable automatic discovery and dynamic expansion of style prototypes, enhancing the model's adaptability to diverse styles. Currently, cross-domain adaptation requires manual adjustment of key parameters. Future research should introduce domain adaptation mechanisms to achieve automatic model adaptation across different domains, simplifying the cross-domain application process. Additionally, the existing multi-modal fusion is merely auxiliary and lacks depth. Future work can explore cross-modal attention fusion strategies between images, text, and audio, integrating multi-dimensional information to improve the comprehensiveness of style evolution analysis. These explorations will further improve the general methodology of visual style temporal analysis, advancing the field of visual aesthetic computing and expanding the method's application value in more practical scenarios.