Ukrainian Journal of Physical Optics


2025 Volume 26, Issue 1


ISSN 1816-2002 (Online), ISSN 1609-1833 (Print)

LOW-LIGHT IMAGE ENHANCEMENT BASED ON DEPTHWISE SEPARABLE CONVOLUTION

Y. Qiu, H. Wang, T. Demkiv, O. Kochan and L. Yan

Author Information
Y. Qiu ORCID logo,
1Department of Computer, Hubei University of Technology Engineering and Technology college, Wuhan 430068, China
H. Wang ORCID logo ,
2School of Computer Science, Hubei University of Technology, Wuhan 430068, China
T. Demkiv ORCID logo SCOPUS logo,
3General Physics Department, Ivan Franko National University of Lviv, 8 Kyryla and Mefodiya Str., 79005 Lviv, Ukraine
O. Kochan ORCID logo SCOPUS logo,
2School of Computer Science, Hubei University of Technology, Wuhan 430068, China
4Department of Measuring-Information Technologies, Lviv Polytechnic National University, 79013 Lviv, Ukraine
L. Yan ORCID logo ,
2School of Computer Science, Hubei University of Technology, Wuhan 430068, China

ABSTRACT

Since the appearance of deep learning algorithms, convolution neural networks (CNN) based algorithms have significantly progressed in weak light image enhancement. However, they still face a major problem: the CNN-based low illumination enhancement algorithm has excessive computational complexity and needs sufficient memory. Although the algorithm's accuracy is improved, the computational efficiency is reduced. This paper introduces the lightweight network for low illumination, and image enhancement is proposed. We first introduce the background of the technology used. Based on the principle of MobileNetV2, we use the generative adversarial networks with improved attention mechanisms as our base algorithm. Then, three comparative algorithms are built for experiments. Experiment results confirm that the proposed network needs fewer algorithm parameters while guaranteeing the low-light image enhancement effect.

Keywords: machine learning, image enhancement, adversarial networks, depthwise separable convolution

UDC: 004.89

    1. Rybář, J., Hučko, B., Ďuriš, S., Pavlásek, P., Chytil, M., Furdová, A., & Veselý, P. (2020). Factors affecting measurements of IOP using non-contact eye tonometer. Strojnícky časopis - Journal of Mechanical Engineering, 70(2), 133-140.
      doi:10.2478/scjme-2020-0026
    2. Jun, S., Kochan, O., & Kochan, R. (2016). Thermocouples with built-in self-testing. International Journal of Thermophysics, 37(4), 37.
      doi:10.1007/s10765-016-2044-2
    3. Song, W., Beshley, M., Przystupa, K., Beshley, H., Kochan, O., Pryslupskyi, A., Pieniak, D., & Su, J. (2020). A software deep packet inspection system for network traffic analysis and anomaly detection. Sensors, 20(6), 1637.
      doi:10.3390/s20061637
    4. Krolczyk, G. M., & Legutko, S. (2014). Experimental analysis by measurement of surface roughness variations in turning process of duplex stainless steel. Metrology and Measurement Systems, 21(4), 759-770.
      doi:10.2478/mms-2014-0060
    5. Su, J., Beshley, M., Przystupa, K., Kochan, O., Rusyn, B., Stanisławski, R., Yaremko, O., Majka, M., Beshley, H., Demydov, I., & Kahalo, I. (2022). 5G multi-tier radio access network planning based on Voronoi diagram. Measurement, 192, 110814.
      doi:10.1016/j.measurement.2022.110814
    6. Abdullah-Al-Wadud, M., Kabir, M. H., Dewan, M. A. A., & Chae, O. (2007). A dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics, 53(2), 593-600.
      doi:10.1109/TCE.2007.381734
    7. Kwan, C., Larkin, J., & Ayhan, B. (2020). Demosaicing of CFA 3.0 with applications to low lighting images. Sensors, 20(12), 3423.
      doi:10.3390/s20123423
    8. Kwan, C., Larkin, J., & Budavari, B. (2020, April). Demosaicing images in low lighting environments. In Signal Processing, Sensor/Information Fusion, and Target Recognition XXIX (Vol. 11423, pp. 267-284). SPIE.
      doi:10.1117/12.2557820
    9. Kwan, C., & Larkin, J. (2019). Demosaicing of Bayer and CFA 2.0 patterns for low lighting images. Electronics, 8(12), 1444.
      doi:10.3390/electronics8121444
    10. Land, E. H., & McCann, J. J. (1971). Lightness and retinex theory. Journal of the Optical Society of America, 61(1), 1-11.
      doi:10.1364/JOSA.61.000001
    11. Jobson, D. J., Rahman, Z., & Woodell, G. A. (1997). Properties and performance of a center/surround retinex. IEEE Transactions on Image Processing, 6(3), 451-462.
      doi:10.1109/83.557356
    12. Jobson, D. J., Rahman, Z. U., & Woodell, G. A. (1997). A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7), 965-976.
      doi:10.1109/83.597272
    13. Fu, Q., Jung, C., & Xu, K. (2018). Retinex-based perceptual contrast enhancement in images using luminance adaptation. IEEE Access, 6, 61277-61286.
      doi:10.1109/ACCESS.2018.2870638
    14. Yuan, L., & Sun, J. (2012). Automatic exposure correction of consumer photographs. In Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part IV 12 (pp. 771-785). Springer Berlin Heidelberg.
      doi:10.1007/978-3-642-33765-9_55
    15. He, K., Sun, J., & Tang, X. (2011). Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2341-2353.
      doi:10.1109/TPAMI.2010.168
    16. Li, C., Fan, T., Ma, X., Zhang, Z., Wu, H., & Chen, L. (2017, June). An improved image defogging method based on dark channel prior. In 2017 2nd international conference on image, vision and computing (ICIVC) (pp. 414-417). IEEE.
      doi:10.1109/ICIVC.2017.7984589
    17. Lore, K. G., Akintayo, A., & Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650-662.
      doi:10.1016/j.patcog.2016.06.008
    18. Shen, L., Yue, Z., Feng, F., Chen, Q., Liu, S., & Ma, J. (2017). MSR-net: Low-light image enhancement using deep convolutional network. arXiv. https://arxiv.org/abs/1711.02488
      doi:10.48550/arXiv.1711.02488
    19. Cai, J., Gu, S., & Zhang, L. (2018). Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing, 27(4), 2049-2062.
      doi:10.1109/TIP.2018.2794218
    20. Wei, C., Wang, W., Yang, W., & Liu, J. (2018). Deep Retinex decomposition for low-light enhancement. arXiv. https://arxiv.org/abs/1808.04560
      doi:10.48550/arXiv.1808.04560
    21. Lv, F., Lu, F., Wu, J., & Lim, C. (2018, September). MBLLEN: Low-light image/video enhancement using cnns. In BMVC (Vol. 220, No. 1, p. 4).
    22. Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018). Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3291-3300).
      doi:10.1109/CVPR.2018.00347
    23. Zhang, Y., Zhou, C., Chang, F., & Kot, A. C. (2019). Multi-resolution attention convolutional neural network for crowd counting. Neurocomputing, 329, 144-152.
      doi:10.1016/j.neucom.2018.10.058
    24. Zhou, Z., Feng, Z., Liu, J., & Hao, S. (2020). Single-image low-light enhancement via generating and fusing multiple sources. Neural Computing and Applications, 32(11), 6455-6465.
      doi:10.1007/s00521-018-3893-3
    25. Fang, M. T., Chen, Z. J., Przystupa, K., Li, T., Majka, M., & Kochan, O. (2021). Examination of abnormal behavior detection based on improved YOLOv3. Electronics, 10(2), 197.
      doi:10.3390/electronics10020197
    26. Li, H., He, X., Tao, D., Tang, Y., & Wang, R. (2018). Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recognition, 79, 130-146.
      doi:10.1016/j.patcog.2018.02.005
    27. Yan, L., Li, K., Gao, R., Wang, C., & Xiong, N. (2022). An intelligent weighted object detector for feature extraction to enrich global image information. Applied Sciences, 12(15), 7825.
      doi:10.3390/app12157825
    28. Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv. https://arxiv.org/abs/1607.08022
      doi:10.48550/arXiv.1607.08022
    29. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv. https://arxiv.org/abs/1704.04861
      doi:10.48550/arXiv.1704.04861
    30. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4510-4520).
      doi:10.1109/CVPR.2018.00474
    31. Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018). Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6848-6856).
      doi:10.1109/CVPR.2018.00716
    32. Ma, N., Zhang, X., Zheng, H. T., & Sun, J. (2018). Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 116-131).
      doi:10.1007/978-3-030-01264-9_8
    33. Trahanias, P. E., & Venetsanopoulos, A. N. (1992, January). Color image enhancement through 3-D histogram equalization. In 11th IAPR International Conference on Pattern Recognition. Vol. III. Conference C: Image, Speech and Signal Analysis, (Vol. 1, pp. 545-548). IEEE Computer Society.
      doi:10.1109/ICPR.1992.202045
    34. Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., Romeny, B.H., Zimmerman, J.B. & Zuiderveld, K. (1987). Adaptive histogram equalization and its variations. Computer Vision, Graphics, and Image Processing, 39(3), 355-368.
      doi:10.1016/S0734-189X(87)80186-X
    35. Cheng, H. D., & Shi, X. J. (2004). A simple and effective histogram equalization approach to image enhancement. Digital Signal Processing, 14(2), 158-170.
      doi:10.1016/j.dsp.2003.07.002
    36. Pisano, E. D., Zong, S., Hemminger, B. M., DeLuca, M., Johnston, R. E., Muller, K., Braeuning, M. P., & Pizer, S. M. (1998). Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. Journal of Digital imaging, 11, 193-200.
      doi:10.1007/BF03178082
    37. Gonzalez, R. C., & Woods, R. E. (2001). Digital image processing (2nd ed.). Prentice Hall. Upper Saddle River, NJ.
    38. Krutsch, R., & Tenorio, D. (2011). Histogram equalization. Freescale Semiconductor, Document Number AN4318, Application Note, 30.
    39. Kaur, M., Kaur, J., & Kaur, J. (2011). Survey of contrast enhancement techniques based on histogram equalization. International Journal of Advanced Computer Science and Applications, 2(7).
      doi:10.14569/IJACSA.2011.020721
    40. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2009, April). BM3D image denoising with shape-adaptive principal component analysis. In SPARS'09-Signal Processing with Adaptive Sparse Structured Representations.
    41. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2008, March). Image restoration by sparse 3D transform-domain collaborative filtering. In Image Processing: Algorithms and Systems VI (Vol. 6812, pp. 62-73). SPIE.
      doi:10.1117/12.766355
    42. Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736-3745.
      doi:10.1109/TIP.2006.881969
    43. Chen, T., Ma, K. K., & Chen, L. H. (1999). Tri-state median filter for image denoising. IEEE Transactions on Image Processing, 8(12), 1834-1838.
      doi:10.1109/83.806630
    44. Cheng, H. D., & Shi, X. J. (2004). A simple and effective histogram equalization approach to image enhancement. Digital Signal Processing, 14(2), 158-170.
      doi:10.1016/j.dsp.2003.07.002
    45. Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008, July). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning (pp. 1096-1103).
      doi:10.1145/1390156.1390294
    46. Jain, V., & Seung, S. (2008). Natural image denoising with convolutional networks. Advances in Neural Information Processing Systems, 21.
    47. Xie, J., Xu, L., & Chen, E. (2012). Image denoising and inpainting with deep neural networks. Advances in neural information processing systems, 25.
    48. Schuler, C. J., Hirsch, M., Harmeling, S., & Schölkopf, B. (2014). Learning to deblur. arXiv preprint arXiv:1406.7444. https://arxiv.org/abs/1406.7444
      doi:10.48550/arXiv.1406.7444
    49. Agostinelli, F., Anderson, M. R., & Lee, H. (2013). Adaptive multi-column deep neural networks with application to robust image denoising. Advances in Neural Information Processing Systems, 26.
    50. Lore, K. G., Akintayo, A., & Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650-662.
      doi:10.1016/j.patcog.2016.06.008
    51. Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018). Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3291-3300).
      doi:10.1109/CVPR.2018.00347
    52. Wang, R., Jiang, B., Yang, C., Li, Q., & Zhang, B. (2022). MAGAN: Unsupervised low-light image enhancement guided by mixed-attention. Big Data Mining and Analytics, 5(2), 110-119.
      doi:10.26599/BDMA.2021.9020020
    53. Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P., & Wang, Z. (2021). Enlightengan: Deep light enhancement without paired supervision. IEEE transactions on image processing, 30, 2340-2349.
      doi:10.1109/TIP.2021.3051462
    54. Bhattacharya, J., Modi, S., Gregorat, L., & Ramponi, G. (2022). D2bgan: A dark to bright image conversion model for quality enhancement and analysis tasks without paired supervision. IEEE Access, 10, 57942-57961.
      doi:10.1109/ACCESS.2022.3178698
    55. Kim, G., Kwon, D., & Kwon, J. (2019, September). Low-lightgan: Low-light enhancement via advanced generative adversarial network with task-driven training. In 2019 IEEE International Conference on Image Processing (ICIP) (pp. 2811-2815). IEEE.
      doi:10.1109/ICIP.2019.8803328
    56. Yan, L., Fu, J., Wang, C., Ye, Z., Chen, H., & Ling, H. (2021). Enhanced network optimized generative adversarial network for image enhancement. Multimedia Tools and Applications, 80, 14363-14381.
      doi:10.1007/s11042-020-10310-z

    З моменту появи алгоритмів глибокого навчання, алгоритми на основі згорткових нейронних мереж (CNN) суттєво просунулися в покращенні зображення при слабкому освітленні. Однак вони все ще стикаються з серйозною проблемою: алгоритм покращення зображення при низькій освітленості на основі CNN має надмірну обчислювальну складність і потребує достатньої пам’яті. Хоча точність алгоритму покращується, ефективність обчислень знижується. У цій статті представлено легку мережу для низького освітлення та запропоновано покращення зображення. Для ознайомлення подані основи використовуваної технології. Базуючись на принципі MobileNetV2, ми використали генеративні змагальні мережі з покращеними механізмами уваги, як базовий алгоритм. Потім були побудовані три порівняльні алгоритми для експериментів. Результати експерименту підтверджують, що запропонована мережа потребує менше параметрів алгоритму, гарантуючи ефект покращення зображення в умовах слабкого освітлення.

    Ключові слова: машинне навчання, покращення зображення, змагальні мережі, роздільна глибинна згортка

Free download this article 

© Ukrainian Journal of Physical Optics ©