Assessment of Combined Shape, Color and Textural Features for Video Duplication

With the on-going advances in low-cost multimedia devices and digital communication technologies, vast amounts of video content would be able to be caught, stored, and transmitted through the internet each day [1]. The number of online videos has encountered an exponential development in current decades, particularly when social video sharing sites came into existence [2]. The techniques involved in digital video forensics fall under two classes namely active and passive. The approaches in the main class implant a watermark, uniquely made, in the video while it is generated and the later does not require any authentication for the information [3]. Subsequently there is also a requirement of significant measure for compressing the videos due to cost-effective storage and bandwidth limitation [4]. Thereby an issue is exacerbated on account of the video because of its impressively huge volume (contrasted with content and images), which make it an extraordinary test for each web-based video platform as well as for systems that analyze and list a lot of web video content [5]. A video format can be expected as a sequence of images namely frames, considered over a period of time, whereby the tampering detection methods produced for image forensics [6-10] could be connected at the frame level. The feature vectors encode the properties of the image, so as to be specific color, texture, and shape. The comparison among two images is figured as a function of the distance between their feature vectors [11]. In the spatial area, an extensively used feature extraction technique forms the spatial gray-level co-occurrence matrix (GLCM) from which a great deal of the second-order statistics could be resolved and examined and also textural features can likewise be extracted from the spectral domains namely frequency domain, wavelet domain, and Gabor domain [12]. Texture investigation and examination assume a fundamental job in many image processing applications [13]. The trained system when provided with a query image retrieves and demonstrates the images which are similar and relevant to the query considered from the database [14]. Finally the work in the paper aims at developing a successful plan for detecting and localizing duplicates in videos [15].

The purpose of our work is to identify the duplication in a video database with the aid of features. To pursue this objective, preliminary work includes converting video into frames and then into blocks. Further work includes image feature extraction (shape, color, and texture) for duplicate identification. The color contains 256 features, shape contains 200 features, the texture contains two different features namely GLCM (22 features in 4 degrees) and GLRLM (11 features). Subsequently, a query video is taken for features extraction and compared with the normal video, if the distance between query video and normal video is found to be similar then the video is identified as duplicate. The results are performed for various evaluation matrix and plotted graphs are shown.

3.1 Video to frame conversion

To perform the work the video cannot be directly utilized for identification. The process includes converting of video into image frames and further converting them into blocks so as to retrieve accurate predicting rate feature extraction which plays a significant role.

3.2 Feature extraction

The images subsequent to frames and the feature extracted by a method for adjusting shape, color, texture (GLCM and GLRLM) in the feature extraction process.

3.2.1 Shape feature extraction

The shape is a fundamental source of data which is utilized for object recognition. Without shape, the visual content of the object cannot be recognized properly. Image is incomplete without recognizing the shape feature. In this mathematical morphology, a method is utilized that gives a way to deal with digital images which depends on the shape. Properly utilized, mathematical morphological operations tend to extract their basic shape attributes and wipe out irrelevancies.

3.2.2 Color feature extraction

Color space represents the color in the form of intensity value. We can specify, visualize and create the color with the help of the colorspace method. There exist many dissimilar color feature extraction methods.

Color histogram. The color histogram represents the image from an alternate point of view. The image in which color bins of the frequency distribution are represented by color histogram counts the pixels which are comparative and store it. Color histogram analyzes every statistical color frequency in an image. The change which happens in the translation, rotation, and angle of view are the issues solved by color histogram and furthermore, it focuses on individual parts of an image. The computation of local color histogram is simple and it is impervious to minor variations in the image for indexing and retrieval of image database, thus proved to be imperative.

3.2.3 Texture feature extraction

The texture contains critical data about the fundamental arrangement of the surface such as clouds, leaves, bricks, fabric, etc. It additionally defines surface with environment relationship. Texture feature additionally depicts the physical composition of the surface. There are many diverse techniques of texture feature extraction

Grey-level co-occurrence matrix (GLCM). A GLCM always signifies a matrix in which the no. of rows and columns are proportionate to the no. of combination of gray levels with value G, in the image. The matrix element p (u,v/d₁,d₂) symbolizes the identical isolation through a pixel separation (d1 and d2) . The GLCMs are arranged for get-together suitable valuations by a method of greycoprops function, which ensemble the insights with respect to the texture of an image, that is set in the below section.

Each feature implies the texture uniformity and non-uniformity, similarity, dissimilarity and different parameters. The Angular Second Moment or Energy signifies the uniformity in the image and is calculated utilizing equation (8) where p (u,v) is the pixel value at the point u,v of the texture image which is of size (MXN). The entropy chooses the dispersal change in the image. It is a proportion of non-consistency and is assessed by the equation (9). The homogeneity estimates the consistency of the non-zero areas in GLCM. As the grey values are higher, lower, GLCM homogeneity, thus reassuring an unrivalled GLCM contrast. The homogeneity is inside scope of [0, 1]. On the off chance that the image is simply sufficiently varied, at that point the homogeneity is more unmistakable and if the image is not in any way changed, at that point the homogeneity becomes equivalent to one. The features considered for GLCM are

$Autocorrelation=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{uvp(u,v)}}\,(1)$           (1)

$Contrast\,=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{{{(u-v)}^{2}}}}\,p(u,v)$           (2)

$Correlation1=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{(u-{{m}_{u}})(v-{{m}_{v}})p(u,v)}{{{\sigma }_{u}}{{\sigma }_{v}}}}}\,(3)$           (3)

$Correlation2=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{(uvp(u,v)-{{m}_{u}}{{m}_{v}})}{{{\sigma }_{u}}{{\sigma }_{v}}}(4)}}\,$           (4)

$Cluster\Pr o\min ence=\,\,\,\,\sum\limits_{u}{\sum\limits_{v}{{{((u-{{m}_{u}})+(v-{{m}_{v}}))}^{4}}p(u,v)}}$ (5)

$ClusterShade=\,\,\,\,\sum\limits_{u}{\sum\limits_{v}{{{((u-{{m}_{u}})+(v-{{m}_{v}}))}^{3}}p(u,v)}}$ $ClusterShade=\,\,\,\,\sum\limits_{u}{\sum\limits_{v}{{{((u-{{m}_{u}})+(v-{{m}_{v}}))}^{3}}p(u,v)}}$           (6)

$Dissimilarity=\sum\limits_{u}{\sum\limits_{v}{|u-v|p(u,v)}}\,\,$           (7)

$Energy=\sum\limits_{u}{\sum\limits_{v}{p{{(u,v)}^{2}}}}\,\,$           (8)

$Entrop{{y}^{{}}}Huv\,=\,\,\,-\,\sum\limits_{u,v}{p\,(u,v)\log (p\,(u,v))}\,$             (9)

$Homogeneity1=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{p(u,v)}{1+|u-v|}}}\,$           (10)

$Homogeneity2=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{p(u,v)}{1+{{(u-v)}^{2}}}}}\,$           (11)

      $Maximum\Pr obability=\max (\max (p(u,v)))$           (12)

$SumOfSquares\,=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{(u-mean(}}p(u,v)){{)}^{2}}p(u,v)$           (13)

$SumAverag{{e}^{{}}}=\,\,\,\,\sum\nolimits_{m=2}^{2{{N}_{g}}}{{{p}_{u+v}}(m)}$           (14)

where ${{p}_{u+v}}(m)$  is the probability of P(u,v) summing to u+v

${{p}_{u+v}}(k)=\sum\limits_{m}{\sum\limits_{n}{p(u,v)}}$  for i+j =k with k=0, 1 2, …, 2(N-1)

${{p}_{u-v}}(k)=\sum\limits_{m}{\sum\limits_{n}{p(u,v)}}$  for |i-j| =k with k=0, 1 2, …, (N-1)

$SumVariance=\,\,\,\,\sum\nolimits_{m=2}^{2{{N}_{g}}}{{{(m-{{f}_{8}})}^{2}}{{p}_{u+v}}(m)}$          (15)

$SumEntrop{{y}^{{}}}{{f}_{8}}=\,\,-\sum\nolimits_{m=2}^{2{{N}_{g}}}{{{p}_{u+v}}{{(m)}^{{}}}\log \{{{p}_{u+v}}(m)\}}$         (16)

$DifferenceVariance=\,\,\,\,\sum\nolimits_{m=0}^{{{N}_{g}}-1}{{{m}^{2}}{{p}_{u-v}}(m)}$            (17)

$DifferenceEntropy=\,\,\,\,-\sum\nolimits_{m=0}^{{{N}_{g}}-1}{{{p}_{u-v}}{{(m)}^{{}}}\log \{{{p}_{u-yv}}(m)\}}$           (18)

$InverseDifferenc{{e}^{{}}}INV=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{p(u,v)}{|u-v{{|}^{k}}}}}\,$           (19)

$InverseDifferenceNomalised=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{p(u,v)}{1+{{\left( \frac{|u-v|}{N} \right)}^{{}}}}}}\,$            (20)

$InverseDifferenceMomentNomalised=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{\frac{p(u,v)}{1+{{\left( \frac{u-v}{N} \right)}^{2}}}}}\,$         (21)

$InformationMeasureOfCorrelation1=\,\frac{Huv-Huv1}{\max \{Hu,Hv\}}$      (22)

$InformationMeasureOfCorrelation2=\,{{(1-\exp [-2(Huv2-Huv)])}^{\frac{1}{2}}}$           (23)

where Hu, Hv are the entropies of p_u and p_v probability density functions with m as u index and n as v index.

${{p}_{u}}(m)=\sum\limits_{n}^{{}}{p(m,n)}$   and  ${{p}_{v}}(n)=\sum\limits_{m}^{{}}{p(m,n)}$             (24)

$Huv1=\,\,\,\,-\sum\limits_{m}{\sum\limits_{n}{p(m,n)\log \{{{p}_{u}}(m){{p}_{v}}(n)\}}}$           (25)

$Huv2=\,\,\,\,-\sum\limits_{m}{\sum\limits_{n}{{{p}_{u}}(m){{p}_{v}}(n)\log \{{{p}_{u}}(m){{p}_{v}}(n)\}}}$           (26)

${{m}_{u}}=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{up(u,v)}}\,$     ${{m}_{v}}=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{vp(u,v)}}\,$           (27)

${{\sigma }_{u}}=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{{{(u-{{m}_{u}})}^{2}}p(u,v)}}\,$  ${{\sigma }_{v}}=\,\,\,\,\sum\limits_{v}{\sum\limits_{u}{{{(v-{{m}_{v}})}^{2}}p(u,v)}}\,(28)$            (28)

Grey-level run length matrix (GLRLM). The texture features considered based on the GLRL matrix are namely the short run emphasis feature (SRE), long run emphasis feature (LRE), gray-level non-uniformity feature (GLN), run length non-uniformity feature (RLN), and run percentage feature (RP).

$SRE=\frac{1}{n}\sum\limits_{q,r}{\frac{p\,(q,r)}{{{r}^{2}}}}\,\,$            (29)

$LRE=\frac{1}{n}\sum\limits_{q,r}{{{r}^{2}}*p\,(q,r)}\,\,\,\,$             (30)

$GLN=\frac{1}{n}{{\sum\limits_{q}{\left( \sum\limits_{r}{p\,(q,r)} \right)}}^{2}}\,\,\,\,$            (31)

$RLN=\frac{1}{n}{{\sum\limits_{q}{\left( \sum\limits_{u}{p\,(q,r)} \right)}}^{2}}\,\,\,\,\,\,\,$         (32)

$RP=\sum\limits_{q,r}{\frac{n}{p\,(q,r)*r}}\,\,$             (33)