Noor Jabbar Abraham | Hazim G. Daway* | Rafid Abbas Ali
© 2022 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Enhancement of images with very low light has become an important role in the field of digital image processing, especially during night photography, tracking and medical imaging using binoculars. In this study, a new algorithm was proposed to enhance images with very low light on the basis of the development of brightness low lightness areas algorithm with the treatment of lighting component (Y) by using Sigmoid function in accordance with YIQ colour space. The proposed method was compared with several algorithms as (contrast enhancement approach, multi-scale retinax with color restoration, histogram equalization, fuzzy logic based-on sigmoid membership function, second-order Taylor series approximation and parallel nonlinear adaptive enhancement) by using non-reference quality measures on the basis of LIME data. Results showed the success of the proposed method on improving images with very low light, obtaining the best quality values rates of Entropy (6.81), NIQE (3.46) and PIQE (35.87).
brightness low lightness areas, image enhancement, sigmoid function, very low lightness, YIQ
Enhancement images are one of the important topics in image processing because it is involved in several areas such as medical images, object detection [1, 2] surveillance, tracking, underwater images [3, 4], etc. images taken in low illumination have poor lighting and low contrast [5, 6].There are several techniques to improve lighting and contrast based on traditional methods based on Histogram Equalization (HE) [7] or Adapted Histogram Equalization (AHE) [8], and some method using the Multi-Scale Retinex (MSR) algorithm [9]. There are many previous studies dealing with the topic of improving the brightness and contrast in digital images, Xuan D, et al proposed an algorithm to improve low-light images for clips, which depended on the use of a dusty image optimization algorithm based on the Dark Channel Prior (DCP) technique. One of the positive points of this method is its high execution speed [10]. Gupta and Agarwal [11] suggested technique depends on replacing the intensity values with the cumulative sum of the probability, but technical depends on the same method as histogram, except for the values of fragmentation of images into several regions. Lin and Shi [12] introduced important techniques that many researchers add in improving images with unbound lighting is the retinax function. The initial results that appeared for the retinax function were a great computational cost, despite the low performance in some cases of the retinax function, a problem for night images with different lighting, as the performance of the retinax function was observed. On the central retina, it was characterized by the characteristics of easy implementation and lower computational cost. Enhancement-based retinax includes two types Single-Scale Retinex (SSR) and Multi-Scale Retinax with Color Restoration (MSRCR). This method has a better improvement between night images, where they replaced the logarithm function with the retina. Kansal and Tripathi [13] suggested a technique for enhancing contrast in the color image using discrete cosine transform and adaptive gamma correction, the proposed method was used to improve the aerial photos, as it succeeded in increasing the contrast of these image. Salas and Lisani [14] proposed an algorithm to improve the contrast in areas with poor lighting, the suggested method relied on Gamma mapping for the red, green, and blue channels. To improve the areas in the color image without color distortions, color transformations such as (HIS, HSL, and YPbPr) are used Through their results, a good contrast improvement was obtained in those images. Fu et al. [15] proposed an algorithm depending on the weighted variational technique, this algorithm does not use log-transformed to enhance contrast and lightness in low illuminance images, the regularization terms used are different from conventional variational models and maintain the estimated reflectance with good details. This algorithm has good results in the distribution of light, although some noise was noticed, which weakened the overall optical quality. Daway et al. [16] introduced many techniques of Fuzzy Logic Based-on Sigmoid Membership Function (FLSMF), that have been developed to improve the color image. In their study, it was used the technique of fuzzy logic. it was successful in enhancing the contrast and brightness of colored images. Zhou et al. [17] proposed an algorithm Second-Order Taylor Series Approximation (SOTSA) to improve the low-light color images to protect the edge information. They adopted the technique of second-order Tayl or series expansion approximation. This method has a good improvement in image density, local contrast, and edge information protection. Gupta and Agarwal [11] proposed Contrast Enhancement Approach (CEA) to improve the images with unbounded light. They used the brightness part to improve the contrast of the image without affecting the color information of the input image. This method succeeded in improving the dark area and the high light area. Zhou et al. [18] introduced Parallel Nonlinear Adaptive Enhancement (PNAE) techniques to improve color images with different lighting, first used intensity mapping by using local neighborhood depending on the intensity mapping function by Taylor transform and then used Adaptive contrast enhancement based on the local neighborhood, this method succeeded in improving the contrast, but it did not succeed in improving the areas with irregular lighting.
The traditional methods of improvement work on improving the compounds HE, AHE and MSRCR which causes interference in the color information and causes an error in color due to the high correlation of the compounds RGB. In this study, we will go to improve images with very low lightness based on brightness low lightness areas and the sigmoid function based on YIQ color space this algorithm retrieves lighting and contrast information without any color distortion.
When capturing an image at low levels of illuminance this leads to a decrease in brightness and contrast. In this study, the lighting will be improved based on brightness Low Lightness Areas (BLLA) [10], and the contrast will be improved by using the Sigmoid function of the lighting component using space YIQ.
2.1 Lightness enhancement
The lighting will be improved depending on the algorithm PLLA which depends on the negative image that is applied on DCP, for an arbitrary negative image Jin is defined as [19]:
$J_{d a r k}(x)=\min _{i \epsilon\{r, g, b\}}\left(\min _{y \in \mathrm{R}(x)}(\operatorname{Jin}(x))\right)$ (1)
where, Jin is a image in R, G and B color space of Jin and $\mathrm{R}(x)$ is a local patch with x coordinate depending on the DCP, the value of the dark channel of $J_{d a r k}$ is low and will be zero where Jin is an outdoor (no haze) free image, except for the bright area [10]:
Jin $=255-c(x)$ (2)
$\mathrm{c}(\mathrm{x})$ being the color image.
$J_{d a r k}(x) \cong 0$ (3)
Thus the transmission value can be determined by [19]:
$\operatorname{tr}(x)=1-w J_{d a r k}(x)$ (4)
Then recover the scene radiance by using [19]:
$J(x)=\frac{I(x)-A}{\max (\operatorname{tr}(x), 0.1)}+A$ (5)
(A) is Air light in this study (0.1) [19], then introduce a multiplier $P(x)$ , and through extensive experiments, we find that $P(x)$ can be set as [10]:
$\mathrm{p}(\mathrm{x})=\left\{\begin{array}{c}2 t(x)-A 0 \leq(x)<0.5 \\ 10.5<t(x) \leq 1\end{array}\right.$ (6)
then the recovery equation becomes [10]
$t(x)=\frac{c(x)-A}{P(x) t(x)}+A$ (7)
Figure 1 shows the improvement by using BLLA method.
Figure 1. A very low lightness image enhanced by BLLA algorithm
2.2 Contrast enhancement
The contrast is improved depending on the space YIQ and for the lightness component Y only by relying on the Sigmoid function. This process allows to improve the lightness of the component while preserving the color information (IQ) the transform RGB to YIQ is done by [20]:
$\left[\begin{array}{l}Y \\ I \\ Q\end{array}\right]=\left[\begin{array}{ccc}2.99 & 0.587 & 0.114 \\ 0.596 & -0.270 & -0.322 \\ 0.211 & -0.211 & 0.312\end{array}\right]\left[\begin{array}{l}R \\ G \\ B\end{array}\right]$ (8)
$Y=2.99 R+0.587 G+0.114 B$ (9)
$Y n=Y / 255$ (10)
By a Sigmoid function. the process of treating the luminous compound with the sigmoid function allows for preserving the chromatic information (IQ) and gives the sigmoid function according to [16]
$\mathrm{Ys}=\frac{1}{\left(1-\left(\frac{\sqrt{1-\mathrm{Yn}}}{\mathrm{Yn}}\right)\right)}$ (11)
The sigmoid function is considered one of the implicit functions that connect with the network or Type equation here. any program, where you change the lighting. Low lighting raises it. As for medium lighting, it remains the same, but it does not affect it. As for high lighting, it decreases, such a conversion increases the contrast of the illumination complex. All illuminated areas have a better illumination that is not incorporated in the image see Figure 2. The inverse from RGB to YIQ is given by [20]
$\left[\begin{array}{l}R e \\ G e \\ B e\end{array}\right]=\left[\begin{array}{ccc}0.24 & 0.33 & 0.25 \\ 0.41 & -1.48 & -1.68 \\ 0.11 & -1.23 & 1.89\end{array}\right]\left[\begin{array}{l}Y_s \\ I \\ Q\end{array}\right]$ (12)
Figure 3 illustrated the stages of the proposed algorithm with the example of an image and Figure 4 shows that the scheme represents the proposed method. In this study there is no strong distortion, especially at low light levels, as long as there is little information present in the image, and color information is preserved when processing the lighting component only.
Figure 2. A sigmoid function mapping [16]
Figure 3. In (a) original image, (b) BLLA enhancement, (c) lighting component (Y) in the YIQ color space, (d) Sigmoid transform for Y-component and (e) Final enhancement, after the inverse transform to RGB
Figure 4. A block diagram of the proposed algorithm
In this research, very low-light images were enhanced on the basis of Proposed (Pro.) algorithm and other algorithms (Pro., CEA, MSRCR, HE, FLSMF, SOTSA and PAEN). All algorithms were used in Matlab program [(Ra 2020) with PC, 2.7 GHZ core i5], depending on LIME data [21]. The data contained 10 images with type bmp, as shown in Figure 5. Four images were chosen as a general model for the improvement, as shown in Figure 6, several non-reference quality standards, including Entropy (EN) [7], Naturalness Image Quality Evaluator (NIQE) [22] and Perception Image Quality Evaluator (PIQE) [23], were adopted to obtain the efficiency of improving very-low-light images. Table 1 shows the average quality of the non-reference measures. The best results were obtained from the proposed method, with the highest values of En, indicating its success in obtaining high contrast for improving low-light images. The proposed method also obtained the lowest values of two scales (NIQE and PIQE), suggesting increased colour information in the enhanced images. The same behaviour could be found in Table 2 when choosing three images from the data, as shown in Figure 7, which illustrates the 3D bar plot for Table 2. Figures 8, 10 and 12 represent the third selected image, which was enhanced by all methods. The best improvement was observed in the proposed method, followed by the method FLSMF. The distribution of histograms for those images (a–c) was represented in Figure 9, 11 and 13, where the proposed method achieved wider ranges of improvement for the red, green and blue components than the other methods. Two areas were identified in image (d) of Figure 14 and 15 to determine the efficiency of the improvement in the areas of low light and high light. The proposed method succeeded in obtaining high contrast and chromatic retrieval for a very light area without colour error or colour loss (go image to grey).
Table 1. The average qualities
|
Method |
EN |
NIQE |
PIQE |
|
Pro. |
6.81 |
3.46 |
35.87 |
|
CEA [11] |
6.49 |
4.37 |
45.62 |
|
MSRCR [9] |
6.45 |
6.76 |
48.46 |
|
HE [7] |
5.62 |
4.71 |
43.95 |
|
FLSMF [16] |
6.48 |
4.59 |
40.47 |
|
SOTSA [17] |
6.64 |
5.94 |
42.82 |
|
PNAE [18] |
6.67 |
6.02 |
43.11 |
Figure 5. The data images that used in this study [20]
Figure 6. The images were selected form LIME dada
Table 2. The qualities for enhanced images (a, b & c)
|
Image _a |
Image _b |
Image _c |
|||||||||
|
Method |
EN |
NIQE |
PIQE |
Method |
EN |
NIQE |
PIQE |
Method |
EN |
NIQE |
PIQE |
|
SUG |
7.41 |
1.97 |
23.56 |
SUG |
7.16 |
2.50 |
20.59 |
SUG |
6.86 |
2.14 |
27.31 |
|
CEA |
5.93 |
2.75 |
29.34 |
CEA |
5.95 |
3.23 |
21.77 |
CEA |
5.69 |
2.61 |
28.19 |
|
MSRCR |
6.47 |
2.93 |
36.53 |
MSRCR |
6.63 |
2.99 |
21.60 |
MSRCR |
6.13 |
2.92 |
36.42 |
|
HE |
5.93 |
2.75 |
29.34 |
HE |
5.95 |
3.23 |
21.77 |
HE |
5.69 |
2.61 |
28.19 |
|
FLSMF |
7.14 |
2.49 |
27.71 |
FLSMF |
7.21 |
3.27 |
20.69 |
FLSMF |
6.38 |
3.13 |
28.49 |
|
SOTSA |
7.15 |
4.80 |
38.27 |
SOTSA |
6.42 |
2.55 |
20.39 |
SOTSA |
6.65 |
3.552 |
25.58 |
|
PNAE |
7.01 |
4.68 |
40.85 |
PNAE |
6.44 |
2.64 |
21.31 |
PNAE |
6.66 |
3.44 |
24.6 |
Figure 7. The pars plot for qualities averages is in Table 1
Figure 8. The Image _a is enhanced using various methods
Figure 9. A histogram of the Image _a is enhanced using various methods
Figure 10. Image _b is enhanced by using various methods
Figure 11. A histogram of the Image _b is enhanced using various methods
Figure 12. The Image _c is enhanced using various methods
Figure 13. A histogram of the Image _c is enhanced using various methods
Figure 14. Determining a low-light region in the Image _d and enlarging it to know the efficiency of the enhancement by using all methods
Figure 15. Determining a high-light region in the Image _d and enlarging it to know the efficiency of the enhancement by using all methods
In this study, an algorithm was proposed that improves lightness and contrasts where, images captured at very low lightness levels were enhanced by using modified Brightness Low Lightness Areas Algorithm on the basis of sigmoid function via YIQ colour space. The LIME data were used. The proposed method was compared with several other methods by using CEA, MSRCR, HE, FLSMF SOTSA and PAEN and non-reference quality measures EN, NIQE and PIQE. The results showed that the proposed method succeeded in improving images with very low light. The best quality values of 6.81, 3.46 and 35.87 were obtained for EN, NIQE and PIQE, respectively. Thus, the proposed method could retrieve lighting and colour information without any colour distortion.
The code for the suggested method can be downloaded via the link: https://drive.google.com/file/d/1v1OlG152bwY9wZ-Y2igUWGh98KWAd31s/view?usp=sharing.
[1] Mutar, A., Dway, H. (2018). Smoke detection based on image processing by using grey and transparency features. Journal of Theoretical and Applied Information Technology, 96(21): 6995-7006. http://www.jatit.org/volumes/ninetysix21.php.
[2] Mohammed, M.H., Daway, H.G., Jouda, J. (2020). Automatic Cytoplasm and Nucleus detection in the white blood cells depending on hisogram analysis. In IOP Conference Series: Materials Science and Engineering. https://doi.org/10.1088/1757-899X/871/1/012071
[3] Mirza, N.M., Al-Zuky, A.A.D., Dway, H.G. (2014). Enhancement of the underwater images using modified retinex algorithm. Al-Mustansiriyah Journal of Science, 24(5): 511-518. https://www.iasj.net/iasj/download/a5cdf5864a881bf0.
[4] Abd-Al Ameer, Z.S., Daway, H.G., Kareem, H.H. (2019). Enhancement underwater image using histogram equalization based on. Journal of Engineering and Applied Sciences, 14(2): 641-647. https://doi.org/10.3923/jeasci.2019.641.647
[5] Daway, E.G., Abdulameer, F.S., Daway, H.G. (2022). X-ray image enhancement using retinex algorithm based on color restoration. Journal of Engineering Science and Technology, 17(2): 1276-1286. https://jestec.taylors.edu.my/Vol%2017%20Issue%202%20April%20%202022/17_2_33.pdf.
[6] Habeeb, R.M. (2020). Video images enhanced by using sigmoid-logarithm transform. In IOP Conference Series: Materials Science and Engineering, 871: 12062. https://doi.org/10.1088/1757-899X/871/1/012062
[7] Babu, V.S., Ram, N.V. (2020). Deep residual CNN with contrast limited adaptive histogram equalization for weed detection in soybean crops. Traitement du Signal, 39(2): 717-722. https://doi.org/10.18280/ts.390236
[8] Pizer, S.M., Johnston, R.E., Ericksen, J.P., Yankaskas, B.C., Muller, K.E. (1990). Contrast-limited adaptive histogram equalization: speed and effectiveness. In [1990] Proceedings of the First Conference on Visualization in Biomedical Computing, pp. 337-338.
[9] Rahman, Z., Jobson, D.J., Woodell, G.A. (1996). Multi-scale retinex for color image enhancement. In Proceedings of 3rd IEEE International Conference on Image Processing, 3: 1003-1006. https://doi.org/10.1109/ICIP.1996.560995
[10] Dong, X., Wang, G., Pang, Y., Li, W., Wen, J., Meng, W., Lu, Y. (2011). Fast efficient algorithm for enhancement of low lighting video. In 2011 IEEE International Conference on Multimedia and Expo, pp. 1-6. https://doi.org/10.1109/ICME.2011.6012107
[11] Gupta, B., Agarwal, T.K. (2018). New contrast enhancement approach for dark images with non-uniform illumination. Computers & Electrical Engineering, 70, 616-630. https://doi.org/10.1016/j.compeleceng.2017.09.007
[12] Lin, H., Shi, Z. (2014). Multi-scale retinex improvement for nighttime image enhancement. Optik, 125(24): 7143-7148. https://doi.org/10.1016/j.ijleo.2014.07.118
[13] Kansal, S., Tripathi, R.K. (2019). Adaptive gamma correction for contrast enhancement of remote sensing images. Multimedia Tools and Applications, 78(18): 25241-25258. https://doi.org/10.1007/s11042-019-07744-5
[14] Salas, J.G.G., Lisani, J.L. (2011). Local color correction. Image Processing on Line, 1: 260-280. https://doi.org/10.5201/ipol.2011.gl_lcc
[15] Fu, X., Zeng, D., Huang, Y., Zhang, X.P., Ding, X. (2016). A weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2782-2790.
[16] Daway, H.G., Daway, E.G., Kareem, H.H. (2020). Colour image enhancement by fuzzy logic based on sigmoid membership function. International Journal of Intelligent Engineering and Systems, 13(5): 238-246. https://doi.org/10.22266/ijies2020.1031.21
[17] Zhou, Z., Chen, L., Hu, X. (2015). Color images enhancement for edge information protection based on second order Taylor series expansion approximation. Optik, 126(3): 368-372. https://doi.org/10.1016/j.ijleo.2014.08.126
[18] Zhou, Z., Sang, N., Hu, X. (2014). A parallel nonlinear adaptive enhancement algorithm for low-or high-intensity color images. EURASIP Journal on Advances in Signal Processing, 2014(1): 1-14. https://doi.org/10.1186/1687-6180-2014-70
[19] He, K., Sun, J., Tang, X. (2010). Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12): 2341-2353. https://doi.org/10.1109/TPAMI.2010.168
[20] Reinhard, E., Khan, E.A., Akyuz, A.O., Johnson, G. (2008). Color Imaging: Fundamentals and Applications. CRC Press.
[21] Guo, X., Li, Y., Ling, H. (2016). LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26(2): 982-993. https://doi.org/10.1109/TIP.2016.2639450
[22] Mittal, A., Soundararajan, R., Bovik, A.C. (2012). Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters, 20(3): 209-212. https://doi.org/10.1109/LSP.2012.2227726
[23] Venkatanath, N., Praneeth, D., Bh, M.C., Channappayya, S.S., Medasani, S.S. (2015). Blind image quality evaluation using perception based features. In 2015 Twenty First National Conference on Communications (NCC), pp. 1-6. https://doi.org/10.1109/NCC.2015.7084843
