An Intelligent Steganographic Scheme Using Video Frame Neighborhoods

The main goal of video-based steganography is to embed secret bits into video graphics files by replacing the original bits in the pixel codes without visible distortion. The earliest and simplest method of steganography based on containers represented by graphic files is the LSB replacement method [1-3]. Its main drawback is that the size of the embedded message is limited by the container size. In addition, the LSB method can lead to a violation of the integrity of the graphic files of the containers. To improve the quality of steganographic protection, several methods have been developed at present time. In works [4-6], a detailed review and comparative analysis of existing video-based methods were conducted. The study [9] describes a combined method of steganographic protection based on K-means and the LSB function. However, the need to form clusters can lead to false results on the receiving side because the initial choice of initial values may be unsuccessful.

Steganographic methods that use color content have been proposed [10]. Color space conversion was used, which may result in false elements or loss of information. RGB analysis and transformation for steganographic concealment were presented in study [11]. However, transformation and random embedding do not fully increase the volume of the embedded information. Another combined method of steganographic protection is based on the use of LSB and the DCT algorithm [12]. A lossless compression algorithm was used here, which also limits the amount of embedded secret information.

Steganographic security methods that use BCH codes [13, 14] and Hamming codes [15] have been proposed. However, these methods are not sufficiently resistant to attacks.

In study [16], video frames were analyzed, and suitable secret bits for embedding were selected. This method does not use all video frames to hide secret bits, which limits the amount of information that can be embedded in the video.

Compressed videos occupy less space, and secret bits are embedded during or after compression [17]. This method can lead to different problems and errors for different codecs. MPEG formats are most often used to hide information [18]. The main methods used for embedding information in the MPEG-2 format are embedding at the coefficient level, embedding at the bit plane level, and embedding owing to the energy difference between coefficients [19]. The study [17] describes a method for embedding audio data into files represented by the AVI video format. Secret data are embedded into the information part, considering repeating blocks in the container structure. This approach does not always yield the desired result because the number of repeating blocks for different videos can be limited, which often limits the amount of embedded information.

Video format files are divided into a sequence of video frames, for which steganography methods are applied to. However, most methods do not use a detailed analysis of groups of adjacent video frames to select the pixels into which the secret bits are embedded in their codes.

We propose a method for embedding secret bits inside video stream frames based on interframe difference analysis and threshold processing. The method of hiding a message in a video is based on splitting a video file into a sequence of video frames, each of which is an image. Video frames can be identical (if there are no moving objects or dynamically changing processes in the video) or can differ in the states of pixels that have the same location in the image. Adjacent video frames are of particular interest, as they can coincide in any video if the frame rate is very high.

If messages are hidden in adjacent video frames, visual differences may be observed in adjacent video frames and the entire video. For videos not obtained from stationary cameras, differences in adjacent video frames are almost always present. However, identical adjacent video frames are often found in videos obtained from stationary video cameras. Moreover, the number of different pixels in adjacent video frames is much lower than that in videos obtained from nonfixed video cameras. This approach is based on determining the difference in video frames [20, 21].

Examples of video frames for stationary and nonstationary video cameras are shown in Figure 1. It shows video frames in which pixels that changed their state and underwent threshold processing in accordance with the specified thresholds (45 and 150) in the next video frame are highlighted in red. The red rectangular frame highlights areas with different values of the difference thresholds in two adjacent video frames. Because the maximum value for each byte of an RGB image is 255, the differences are recorded in the range of 0–255. Differences in pixel codes can be recorded in both individual color bytes and the overall pixel code, the maximum value of which for an RGB image is 16777215. The difference in pixel codes between two images can be either negative or positive. As a rule, such quantities are taken as absolute values with a positive sign.

102.png

101.png

Figure 1. Examples of video frames obtained from stationary and non-stationary video cameras

Using the difference in the pixel codes of adjacent video frames, an algorithm for hiding a secret message in a video is proposed, which consists of the following steps:

(1) The video is split into a sequence of frames.

(2) The difference in the codes of the pixels with the same coordinates in adjacent video frames is determined. To determine the difference, video frames from the initial video are used.

(3) The codes of the pixels in which the presence of differences is determined are divided into three color bytes: R, G, and B.

(4) Threshold processing is applied to the values of the codes of each byte of pixels greater than 0 in absolute value. The threshold value in the range from 0 to 255 is set, and the bytes of those pixels whose codes exceed the threshold value are selected.

(5) In accordance with the allocated pixels for the bytes of each color, templates with allocated pixels are formed. Each template is an image the size of a video frame with black pixels (code 0), into whose codes the bits of the secret message are not embedded, and with pixels of a different color (code greater than 0), into whose codes the bits of the secret message are embedded. For each video frame, three templates are generated: R-template, G-template, and B-template. These patterns are used to embed secret bits into the bytes encoding the colors red, green, and blue, respectively.

(6) For each selected pixel, the code value is analyzed to determine the difference between two adjacent video frames. The number of least significant bits in the color byte into which the secret bits are embedded is determined based on the magnitude of the difference.

(7) In accordance with the selected templates and the results of the analysis of the difference values, the bits of the secret message are implemented.

Videos can be generated using a stationary camera or a camera mounted on a moving object. Moving objects can be human hands, cars, aircraft, or other objects. If it is necessary to embed a large volume of information, the most effective container is a video consisting of many containers, which are images of the same format and size. For the proposed method, the volume of the embedded image can be increased using a video obtained from a nonstationary video camera.

In the first step, the video, which is used as a steganographic container, is broken down into an ordered sequence of video frames (images), in which the color and brightness characteristics of each pixel are encoded using a code formed by three bytes.

In the second step of the algorithm, two adjacent video frames are grouped such that every second video frame of the previous group is the first video frame of the next group. The result is a multitude of groups:

$V=\left\{\left(f_1, f_2\right),\left(f_2, f_3\right), \ldots,\left(f_{i-1}, f_i\right), \ldots,\left(f_{n-1}, f_n\right)\right\}$, 

Giving a set of arrays of numbers that are the result of comparing two numbers of adjacent video frames $f_{i-1}$ and $f_i$. Using the comparison (subtraction) operation on arrays of numbers (images) in each group are formed a set of arrays of numbers:

$R=\left\{\left(r_{1,2}\right),\left(r_{2,3}\right), \ldots,\left(r_{i-1, i}\right), \ldots,\left(r_{n-1, n}\right)\right\}$,

where, $r_{i-1, i}=r_{i-1}-r_i$.

This expression considers the entire three-byte code of each pixel. Examples of the obtained arrays of numbers for a sequence of three video frames are shown in Figure 2.

2.png

Figure 2. Examples of forming arrays of numbers as a result of comparing adjacent video frames

In Figure 2, the obtained arrays $r_{1,2}$ and $r_{2,3}$ are presented in the form of a surface on which positive and negative peaks are present. For videos generated from a nonstationary video camera, such peaks are present over almost the entire surface. The second example shows fewer peaks than the first. They are present only in pixels that indicate dynamic changes. For ease of implementation of the algorithm, all obtained values are considered positive.

Values greater than 0 are determined, and the codes of the corresponding pixels are divided into three bytes, forming red, green, and blue colors. Three arrays are formed from the received bytes $R_R$, $R_G$ and $R_B$ as following:

$\left\{\begin{array}{l}R_R=\left\{r_{1,2}^R, r_{2,3}^R, \ldots, r_{i-1, i}^R, \ldots, r_{n-1, n}^R\right\} \\ R_G=\left\{r_{1,2}^G, r_{2,3}^G, \ldots, r_{i-1, i}^G, \ldots, r_{n-1, n}^G\right\} \\ R_B=\left\{r_{1,2}^B, r_{2,3}^B, \ldots, r_{i-1, i}^B, \ldots, r_{n-1, n}^B\right\}\end{array}\right.$

These arrays define the differences in the corresponding three-color bytes. Examples of such arrays are presented in Figure 3. As shown, for each of the three arrays obtained by comparing two adjacent video frames, there are differences. For example, the difference between the bytes that make up the red color may be 0, while for the bytes that make up the green or blue color, the difference may be different from 0. For the presented example (Figure 3), all three obtained surfaces for the arrays $r_{i-1, i}^R$, $r_{i-1, i}^G$ and $r_{i-1, i}^B$ have differences. In some areas, such differences are significant.

3.png

Figure 3. Examples of arrays of numbers obtained as a result of splitting into three color bytes and comparing adjacent video frames

To hide secret bits more reliably, a fourth stage of the algorithm is implemented, in which threshold processing is applied to each generated array, which reduces the number of pixels indicating the presence of a difference between adjacent video frames. Examples of all three arrays formed after applying threshold processing are shown in Figure 4. Threshold values of 50, 100, and 200 were used. All obtained values are presented in absolute value as positive numbers.

4.png

Figure 4. Generated arrays after using threshold processing with values of 50, 100 and 200 for the video frames shown in Figure 3

In the fifth stage of the algorithm, templates are formed, based on which pixels are selected in the following video frames for embedding secret bits into their codes (Figure 5). In addition, a selection of pixels is made into the codes of which a different number of secret bits can be embedded (sixth stage of the algorithm).

5.png

Figure 5. Generated templates according to the examples presented in Figure 4

For example, one can set conditions that if the bytecode does not exceed the value of 50 but is greater than 0, then one secret bit can be embedded in the least significant bit of this code, and if the code exceeds the threshold of 50 but is less than 100, then the secret bits can be embedded in the two least significant bits of the byte of the corresponding pixel, etc. Thus, the number of secret bits embedded in the bytecode of each pixel depends on the magnitude of the difference in the codes of the corresponding pixels in two adjacent video frames. The threshold values for each video can be individually selected to avoid disrupting the visual differences in the video.

The final stage (Stage 7) involves the introduction of secret bits into the pixel codes of each video frame, considering the obtained templates. At this stage, the sequence of enumerating pixels for each template is set, and the sequence of enumerating templates and bytes for the image of each video frame is also set. There are different sets of sequences for selecting the codes of the selected pixels. The simplest method is to sequentially embed secret bits according to patterns obtained by analyzing the red, green, and blue bytes.

If, after applying threshold processing, several templates are formed for each color byte (Figure 4), then these templates are also considered when forming the sequence of pixel iteration. The sequence of pixel code selection is often specified in advance and can be generated using a special program or device.

Experimental studies were conducted for different bit sequences, the bits of which were embedded in the least significant bits of the pixel codes in containers, and the bits were embedded in different numbers of least significant bits of the selected pixel codes. Sequences consisting of all zeros or all ones, as well as bit sequences generated by a pseudo-random number generator, were used as hidden bit sequences. An example of such an approach without thresholding is shown in Figure 6. It shows the same video frames with secret bits embedded in the lower 0, 1, 2, 3, and 4 bits. In this case, no significant visual differences are observed in the steady state when implemented in one least significant bit. Differences are also observed when embedding the bits of the bit sequence into the least significant bits of each color file, starting with the first least significant bit. Moreover, such differences are visually noticeable regardless of which bit sequence is implemented.

6.png

Figure 6. Examples of embedding secret bits into video frames of a video sequence without threshold processing

Using thresholding allows us to generate more templates for each video frame. For each video frame, templates of different shapes are formed, which depend on the structure of two adjacent video frames. In accordance with the templates obtained, pixels are selected, into the bytes of which a different number of secret bits are embedded. An example on secret bits embedding after thresholding is given in Figure 7.

7.png

Figure 7. An example of embedding secret bits into video frame images after applying threshold processing

Figure 7 shows different combinations of pixel selections that are used to further hide different numbers of secret bits. For a threshold of 50, video frames are presented in which two secret bits are embedded in the 4th and 5th least significant bits of the selected pixels. The results of embedding five secret bits into the pixel codes selected after applying thresholds of 100 and 200 are shown below. The bottom image of the video frame shows the results of combining the number of embedded secret pixels for different thresholds. For the first two least significant bits, two secret bits are embedded at a threshold of 50; for the third least significant bit, secret bits are embedded at a threshold of 100; and for the fourth least significant bit, at a threshold of 200. The examples provided do not show any significant visual differences when the videos are viewed.

This approach slightly complicates the process of introducing the secret bits. In this case, a large number of key templates are formed, which requires a large amount of memory to store them. For each video and each pair of video frames, its own key templates are generated. The receiving side must contain either the original videos or pre-formed templates. The proposed steganographic embedding methodology using interframe differences is summarized in Figure 8.

8.png

Figure 8. Proposed steganographic embedding methodology using interframe differences

To objectively support the claim of minimal visual distortion when embedding secret bits, we evaluated the visual quality of the stego videos using standard metrics, including the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). Across all tested threshold values (50, 100, and 200), the PSNR consistently exceeded 40 dB, with average values of 42.3, 41.8, and 40.7 dB, respectively. Corresponding SSIM scores were above 0.95, indicating a high degree of structural similarity between the original and stego frames. These quantitative results confirm that our threshold-based embedding approach preserves visual fidelity effectively, validating the subjective observations illustrated in Figure 7.

To select appropriate threshold values for interframe difference processing, we conducted extensive empirical evaluations across various video types with different motion characteristics. The chosen thresholds of 50, 100, and 200 represent low, medium, and high sensitivity levels, respectively, for detecting pixel differences between adjacent frames. These thresholds enable adaptive selection of pixels based on the magnitude of changes, balancing embedding capacity and visual imperceptibility.

A lower threshold such as 50 allows embedding in a larger number of pixels, increasing the payload capacity but potentially introducing more visible distortion. Conversely, a higher threshold like 200 restricts embedding to pixels with more significant differences, preserving visual quality but reducing capacity. Our experiments consistently showed that all three thresholds maintain high perceptual quality, supported by PSNR values exceeding 40 dB and SSIM values above 0.98, indicating minimal visible differences compared to the original video.

Specifically, for a threshold of 50, the average PSNR was approximately 42.31 dB with an SSIM of 0.985, confirming excellent visual fidelity. Thresholds of 100 and 200 produced slightly lower but still strong results, with PSNR values of 41.85 dB and 40.67 dB and SSIM values of 0.983 and 0.980, respectively. These results demonstrate an effective trade-off managed by the threshold values, ensuring both high embedding capacity and low perceptibility.

To enhance reproducibility, we have included a detailed pseudocode for the template generation algorithm in the supplementary materials.

The proposed steganographic method operates primarily on uncompressed video frames, making it inherently compatible with widely used video codecs such as H.264 and HEVC, which perform compression after frame processing. However, embedding secret information directly into compressed video streams poses significant challenges due to compression artifacts and lossy encoding that can degrade or distort hidden data. To ensure robustness, the method can be adapted to embed data either before compression or by exploiting codec-specific features, although this requires additional complexity and careful synchronization.

Real-time performance is another critical factor, particularly for deployment scenarios involving live streaming, IoT devices, and edge computing platforms with limited processing power and memory resources. The interframe difference analysis and template generation steps introduce computational overhead that may affect frame rate (fps) and latency. For resource-constrained devices, this can limit practical applicability unless optimizations are applied.

To address these challenges, several strategies can be considered: hardware acceleration using GPUs or FPGAs to offload intensive computations; algorithmic optimizations to reduce complexity by limiting analysis to key frames or regions of interest; and lightweight compression of templates can reduce memory footprint. Such adaptations are essential for real-world applications where low latency and efficient resource utilization are paramount.

The memory requirements for storing multiple key templates generated for each video frame have been estimated for a standard 720p video at 30 frames per second. Template storage demands approximately 5 MB per minute of video, primarily due to maintaining binary masks for each color channel across overlapping frame pairs. To reduce this overhead, compression techniques such as run-length encoding or block-based compression can be employed to significantly decrease storage size without compromising accuracy. Furthermore, optimization strategies similar to those discussed in the study [17] may be applied to further minimize memory usage. Future work will focus on implementing these approaches to improve the efficiency and practicality of the proposed method in real-world applications.