Deep Learning Based Compression with Classification Model on CMOS Image Sensors

The demand for lower-power sensing, higher-resolution devices with incorporated image processing capabilities, particularly compression capability, is growing. The complementary metal oxide semiconductor (CMOS) technique permits the incorporation of image processing and sensing, making it feasible to enhance the complete performance of the system. The CMOS is a semiconductor technology used in the manufacturing of integrated circuits (ICs). In a CMOS circuit, each transistor operates as either an on or off switch. When a transistor is on, it allows current to flow through it, while when it's off, no current flows. By using complementary pairs of transistors, CMOS circuits can efficiently switch between these two states, which makes them ideal for use in digital circuits. One of the main advantages of CMOS technology is its low power consumption. Since the transistors are either fully on or fully off, they consume very little power when they are not actively switching. This makes CMOS ideal for use in battery-powered devices, such as mobile phones and laptops. Another advantage of CMOS technology is its high noise immunity. The complementary pairs of transistors ensure that noise signals are canceled out, resulting in a high signal-to-noise ratio.

The CMOS is used in a wide range of applications, from digital logic circuits to analog signal processing circuits. The CMOS technique permits the incorporation of image processing and sensing, making the CMOS image sensor the optimal solution to enhance the performance of the overall system [1]. Figure 1 shows the block diagram of CMOS image sensor. Over the previous years, image sensors integrating distinct on-chip compression methods, namely, wavelet-based image processing, predictive coding, conditional replenishment, image processing, and SPIHT algorithm were introduced. The integral part of image compression lies in internet browsing, medical sciences, TV broadcasting, navigation applications, etc., with the development of science and techniques, the need for reduction in transmission time and the storage space requirement for digital images becomes an important concern. The broadcast range has been restricted which makes issue [2]. A proper and efficient image compression method is of major need to address the issue of constrained bandwidth.

tu_pian_1.jpg

Figure 1. Block diagram of CMOS image sensor

Over the past decades, the research field has paid more towards the presented method for image compression. Vector Quantization (VQ) approach outperforms another method, namely, pulse code modulation (PCM), Adaptive DPCM, differential PCM (DPCM) that belongs to the group of scalar quantization methods [3]. VQ, the more common lossy image compression method is mainly a c-means clustering method commonly employed for pattern recognition and image compression, face detection, speech and image coding speech recognition due to the advantage that includes high compression rate and simple decoding architecture it offers with lower distortion [4]. Object detection and image classification are two applications of artificial intelligence (AI) commonly employed in the industry.

The research gap in the existing works is need to optimize the convolutional neural network (CNN) architecture and CMOS sensor to train the edge platform for larger data. The research motivation of this study is deep learning (DL) model plays an integral part in enhancing the performance and accuracy. High speed or resolution CMOS image sensor provides vital image quality to exact processes like CNN based object detection and image classification [5]. The CNN method that has been trained is extremely based on the training data. When the training data is contained in lower-quality images, the CNN cannot recognize the object under deliberation and is more possible to give poor accuracy for object detection and image classification methods.

The results of the proposed metaheuristics with deep learning based compression with image classification model (MDL-CCIM) method has been provided to identify and compress the images taken by CMOS image sensors. The butterfly compression and classification processes are the two main steps in the proposed MDL-CCIM approach. The BOA with LBG model is primarily used to compress images. Second, DenseNet with a softmax layer is used to classify images. The Adam optimizer then selects the DenseNet model's hyperparameter tweaking in the best possible way. The results are looked at from a variety of angles. A set of 300 images from each category are taken into consideration in this work. The compression result analysis of the MDL-CCIM methodology with new methods under various Codebook Size (CS) and pictures. The results showed that every CS and image had higher Peak signal-to-noise ratio (PSNR) values as a result of the MDL-CCIM technique.

Section 2 of the paper, which analyses the connected works involved in compression with image classification model, is how the other portions of the essay are structured. The suggested MDL-CCIM model is described in Section 3. The results are then analyzed and discussed in Section 4, along with a performance comparison with alternative approaches. Finally, the important findings of the suggested investigation are summarized in Section 5.

In this research study, a new MDL-CCIM technique has been developed to compress and classify the images captured by CMOS image sensors. The proposed MDL-CCIM technique follows two major processes, namely, butterfly compression and classification. Primarily, the BOA with LBG model is applied for image compression [11]. Secondly, the DenseNet with softmax layer is employed for image classification. Finally, the hyperparameter tuning of the DenseNet model is optimally chosen by the Adam optimizer.

3.1 Design of BOA-LBG based compression technique

The BOA-LBG technique is derived to compress images. An efficient image compression approach is named as BOA-LBG algorithm is established for compressing the image. The BOA-LBG approach creates to utilize of BOA technique with LBG algorithm. The VQ is a lossy data compression method in block coding [12]. The generation of codebook was detected as a vital process of VQ. Considering that size of new images Y={y_ij} be M×M pixel that separated into various blocks with size of n×n pixel. Especially, there are $N_b=\left[\frac{N}{n}\right] \times\left[\frac{N}{n}\right]$ block that is indicated as set of input vectors=(x_i, i=1, 2, …, N_b). Consider that L be n×n. An input vector $\chi_i, \chi_i \in \mathfrak{R}^L$ where $\Re^L$ is L dimension Euclidean space. The codebook C has of N_c L dimensions code word, i.e., $=\left\{c_1, c_2, \ldots, c_{N_C}\right\}$, $c_j \in \Re^L, \forall j=1,2, \ldots N_c$.

Every input vector was signified as row vector x_i=(x_i1, x_i2, …, x_iL) and j^th codeword of codebook is illustrated as c_j=(c_j1, c_j2, …, c_jL). The VQ methods assign every input vector for connecting codeword, and the codeword is changing the linked input vector finally to attain the resolve of compression. The optimizing of C interms of MSE was written as minimizing distortion function D. “Usually, the minimal value of D was optimal the quality of C in Eq. (1).”:

$D(C)=\frac{1}{N_b} \sum_{j=1}^{N_c} \sum_{i=1}^{N_b} \mu_{i j} \cdot\left\|x_i-c_j\right\|^2$   (1)

According to the succeeding constraint in Eq. (2):

$\sum_{j=1}^{N_c} \mu_{i j}=1, \forall i \in\left\{1,2, \ldots, N_b\right\}$       (2)

$\mu_{i j}=\left\{\begin{array}{l}1 \,\, { if }\, x_i \, { is \,in \,the } \,\, { jth\, cluster; } \\ 0 \,\, { otherwise }\end{array}\right.$   (3)

$\mu_{i j}=\left\{\begin{array}{l}1 \text { if } x_i \text { is in the } \text { jth cluster } \\ 0 \text { otherwise }\end{array}\right.$     (4)

where, L_k implies the minimal of k^th elements in all trained vectors, and U_k refers the higher of k^th elements in every input vector. The ‖x-c‖ indicates the Euclidean distance among the vector x as well as codeword c. The two important states that an optimum vector quantize in Eq. (3) and Eq. (4).

The partition R_j, j=1, …, N_c be needed to be done in Eq. (5):

$R_j \supset\left\{x \in \chi: d\left(x, c_j\right)<d(x, c \forall k)\right.$          (5)

The codeword c_j was offered as the centroid of R_j in Eq. (6):

$c_j=\frac{1}{N_j} \sum_{i=1}^{N_j} x_i, x_i \in R_j$    (6)

where, N_j implies the whole count of vectors going to R_j.

3.1.1 LBG algorithm

This approach to a scalar quantizer was projected by Lloyd. This technique was named LBG or generalization Lloyd algorithm (GLA). It implements the 2 abovementioned states for the input vector for defining the codebook. For providing input vectors x_i, i=1, 2, …, N_b, distance function d, and first codewords c_j(0), j=1, …, N_c. The LBG iteratively executes 2 states to produce a better codebook due to the succeeding approach:

(1) Partition the input vector as numerous groups employing the lesser distance rule. This outcome partition is stored from N_b×N_c binary indicator matrix U when the element is defined as subsequent in Eq. (7):

$\mu_{i j}=\left\{\begin{array}{cc}1 &  { if } d\left(x_i, c_j(k)\right)=\min _p d\left(x_i, c_p(k)\right) \\ 0 &  { otherwise }\end{array}\right.$    (7)

(2) Resolve the centroid of every partition. Interchange the old codeword with centroids Eq. (8):

$c_j(k+1)=\frac{\sum_{i=1}^{N_b}\, \mu_{i j}\, x_i}{\sum_{i=1}^{N_b}\, \mu_{i j}}, j=1, \ldots, N_c$         (8)

(3) Repeat steps (1) & (2) until no c_j, j=1, …, N_c alters already.

3.1.2 Codebook construction process

For the optimal codebook construction process, the BOA is utilized. The BOA is derived from the characteristics of BFs at the time of matting and food source discovery. It employs a pair of navigation patterns for searching the area. During exploration (r₁≤p), BFs goes in the direction of the optimal BF of the colony, whereas during exploitation (r₁>p), the BFs carry out an arbitrary searching process within the searching area by searching the arbitrary BFs. They can be defined in a mathematically way as follows in Eq. (9) and Eq. (10).

If r₁≤p, Global searching:

$t+1_{X_i}={ }^t X_i+\left(r_2^2 \times g^*-{ }^t X_i\right) \times \varphi_i$            (9)

If r₁>p, Local searching:

$t+1_{X_i}={ }^t X_i+\left(r_3^2 \times{ }^t X_j-{ }^t X_k) \times \varphi_i\right.$           (10)

where, t and t+1 represent the present and update levels of the respective variables. Besides, the place of the optimal BF can be represented using g^*, and ^tX_i and ^tX_k, indicates the places of two randomly chosen BF; r₁, r₂ and r₃ implies random scalars and φ_i indicates the fragrance factor which can be represented as follows in Eq. (11):

$\varphi_i=c I^a$          (11)

where, φ_i indicates the fragrance magnitude for ith BF; I and a denote the stimulus intensity and fluctuating absorption degree. $I$ undergoes correlation to the objective function value, and for ith BF as f(X_i), where f defines an objective function. The coefficients a and c are chosen from intervals of [0,1], p imply probability switching that computes the searching nature. The entire procedure contained in the BOA-LBG method was provided as follows.

During the initial phase, the parameter initialization occurs so that the codebook generated by LBG method is allocated to the initial solution (i.e., satin bowerbirds), however, the rest of the initial solutions are arbitrarily created. Every solution represents the codebook of NC codeword. In addition, the initialization of BOA occurs.

Secondary phase, the presented optimal solution is selected by computing the fitness of every place and the higher fitness place is detected as the optimal one.

During the tertiary stage, a novel solution is generated in the bowerbird mutation process.

The solution is ranked on the basic of FF in the subsequent step among that the optimal is chosen.

The secondary and tertiary steps are iteration until the termination state is met.

3.2 Design of image classification technique

The proposed image classification module incorporates three major processes, namely, DenseNet based feature extraction, Adam based hyperparameter tuning, and SM based classification.

3.2.1 DenseNet model

The DenseNet technique is a DL approach established on ResNet [13]. Recently, DenseNet is attaining optimum outcomes in the domain of image classification. The fundamental model of ResNet and DenseNet is similar; but, DenseNet introduces a dense connection amongst every preceding layer and the latter, and it recognizes feature reprocess with the connection of features to the channels. This feature creates DenseNet to obtain optimum efficiency than ResNet with some parameters and computational cost and alleviate gradient vanishing problems.

The DenseNet is mostly collected from DenseBlock and Transition layer. The DenseBlock implements a radical dense connection process; i.e., every layer is linked to everyone. Specially, all layers accept the resultant of each previous layer as their input. In DenseBlock, all layers have a similar size and all layers are concatenated with every previous layer in the channel dimensional. In order to network with L layer, DenseBlock has an entire of L(L+1)/2 connection. The input of layer L is as follows in Eq. (12):

$x_L=H_L\left(\left[x_1, x_2, \ldots, x_{L-2}, x_{L-1}\right]\right)$           (12)

where, L implies the amount of layers. H_L stands for non-linear transformation that is, a group of Batch Normalization (BN), ReLU, Pooling, and Conv functions. The common DenseNet-B infrastructure was employed, and the bottleneck layer was utilized for reducing the count of computation; i.e., the infrastructure BN+ReLU+1×lConv+BN+ReLU+3×3 Conv was implemented. All layers from DenseBlock output k feature map were then convolutional, specifically, the number of convolutional kernels. When it can be set the channel count of input DenseBlock as k₀, afterward, the input channel count of L layer is k₀+ k(L-1). At this point, the last convolutional of all layers are k, and k is named as growth rate. In DenseBlock, with improvement in the number of layers, the number of input channels is superior. While the input size of the model passing to DenseBlock remains unvaried, the channel dimensional is remaining for increasing. So, dimensional decrease is essential for reducing computational complexity.

The Transition layer was mostly collected of 1×1 convolutional and 2×2 Avg_Pooling or Max_Pooling, and their infrastructure was BN+ReLU+1×1Conv+2×2 Avg_Pooling. It attaches 2 neighboring denseblock and decreases the dimensional resultant of the denseblock. In this case, DenseNet-121 model is utilized. Figure 2 shows the layered architecture of DenseNet-121 model. To adjust the hyperparameter values of the DenseNet model, the Adam optimizer is utilized in Eq. (13):

$x_L=H_L\left(\left[x_1, x_2, \ldots, x_{L-2}, x_{L-1}\right\rceil\right)$            (13)

where, L implies the amount of layers. H_L stands for non-linear transformation that is, a group of Batch Normalization (BN), ReLU, Pooling, and Conv functions. The common DenseNet-B infrastructure was employed, and the bottleneck layer was utilized for reducing the count of computation; i.e., the infrastructure BN+ReLU+1×lConv+BN+ReLU+3×3 Conv was implemented. All layers from DenseBlock output k feature map were then convolutional, specifically, the number of convolutional kernels. When it can be set the channel count of input DenseBlock as $k_0$, afterward, the input channel count of L layer is k₀+k(L-1). At this point, the last convolutional of all layers are k, and k  is named as growth rate. In DenseBlock, with improvement in the number of layers, the number of input channels is superior. While the input size of the model then passing with DenseBlock remains unvaried, the channel dimensional is remaining for increasing. So, dimensional decrease is essential for reducing computational complexity.

tu_pian_2.png

Figure 2. DensNet 121 architecture

3.2.2 Softmax classifier

The SM layer is the forecast the label probability of input x_j with employing the feature learned from the 3^rd hidden state representation $h_i^{(3)}$. The number of nodes existing in SM state was chosen as equivalent to the number of labels [14, 15].

The SM state is 5 nodes equivalent to grading groups in 1-5. But the classifiers like SVM are employed, softmax LR permits for optimizing the whole deep network to fine-tuned jointly. The succeeding main function was minimizing to a finetune the network to softmax state in Eq. (14) [16, 17].

$\begin{gathered}J_{S S A E-S M C}(W, b, x, \hat{z}) \\ =\min _{W, b} J(x, \hat{z})+\lambda^{s m c}\left\|W^{s m c}\right\|_2^2\end{gathered}$        (14)

where, W and b refers the weight and bias of entire deep network, $J(x, \hat{z})$ represents the LR cost amongst the classifier attained with input feature x and unsupervised outcome $\hat{Z}$ and W^smc implies the weight and λ^smc stands for the weight decomposed parameter [18, 19]. By implementing finetuned, the weight and bias of SM are optimized together, and SM state was employed to the classifier. Assume that y_i refers the label of trained instances x_i. The probability of x_i refers to the k^th class was written in Eq. (15):

$p\left(y_j=k \mid x_j ; W_{s m c}, b_{s m c}\,\right)=\frac{e^{W_{s m c}^{(k)}\,\quad x_i^T\,\,\,+\,b_{s m c^{\prime}}^{(k)}}}{\sum_{j=1}^N \,\,e^{W_{s m c}^{(j)^T}}\,,\, x_i+b_{s m c}^{(j)}}$         (15)

where, $W_{s m c}^{(k)}$ and $b_{s m c}^{(k)}$ implies the distribution of weight and bias from the k^th class. N signifies the entire amount of classes that relate to the 5 grade groups [20]. Due to the maximal probability, it can calculate the grade group of instances x_j utilizing the formula in Eq. (16):

${Grade}\left(x_i\right)=\arg_k \max _{k=1 \ldots N} p\left(y_i=k \mid x_i ; W_{s m c}, b_{s m c}\right)$      (16)