
Citation: | DENG Jiawei, YU Zhenming, PANG Guangyao. Colour Variation Minimization Retinex Decomposition and Enhancement with a Multi-Branch Decomposition Network[J]. Chinese Journal of Electronics, 2023, 32(4): 908-919. DOI: 10.23919/cje.2021.00.350 |
Currently, information technology is developing rapidly. As the carrier of visual information, image has the advantages of intuitive information and rich content, and has become an important means to obtain, process and transmit information. In a light-rich environment, we obtain images with clear content and detailed structure. However, in low-light/uneven-light scenes such as night, cloudy, and backlit, image suffers from low contrast, noise interference, and colour distortion. One or more problems have been solved with the advancement of image capture equipment and image acquisition technology, but the expensive equipment and complex auxiliary methods have greatly increased the hardware cost. In daily life, image can be collected under very low-light conditions such as mobile devices pictures [1] and urban road monitoring. They show varying degrees of degradation, which seriously affects their further use. Dark image enhancement [2] is an important part of image restoration technology. The technique restores detail in low-light areas by changing the brightness of the source images. On the one hand, this technology can help consumers optimize image details and highlight the more attractive parts of the image during photography; on the other hand, this technique plays an important role in the image preprocessing stage. It can provide better image to advanced visual tasks such as video surveillance [3] and intelligent driving [4]. Therefore, it is important to develop practical technology for large-scale image data enhancement.
To improve low-quality images, low-light image enhancement technology must address four major problems: low contrast, high noise, colour distortion and uneven brightness. Among the traditional methods, histogram equalization (HE) [5] is the most representative. The grayscale histogram of the dark images is distributed over a small range. HE extends the original grayscale images to an approximately uniform distribution over the entire area, improving image contrast [6]. This algorithm also causes overexposure when it improves the image contrast. Retinex theory states that colour texture and light are constant feature and variable feature, respectively. The theory suggests that adjusting variable features to the retina can distinguish constant features and minimize the loss of constant features [7]. Homomorphic filtering establishes a reflection-illumination model in the frequency domain. The model treats the illumination components in the low-frequency domain with approximately uniform treatment, and improves the reflectivity contrast in the high-frequency domain. However, noise problems remain in the enhanced images [8]. These traditional methods focus on brightness enhancement and contrast improvement to restore the details of shadowed image blocks. The enhancement ability is substantial for some specific images, but these approaches cannot meet the needs of large-scale image processing.
Data-driven deep learning model is a hot area for dark image enhancement technique. It builds a black-box model that illuminates low-light images on the basis of large-scale data and restores shadow detail. According to the different algorithm requirements, deep learning methods can be divided into high-reference image enhancement, low-reference image enhancement, and non-reference image enhancement. High-reference image enhancement is a mainstream low-light image enhancement method based on deep learning. High-reference image enhancement is the earliest studied deep learning method. The method inputs the whole process of bright image matching, constraint, dark image feature extraction and enhancement [9]. The enhancement lighting is close to the reference lighting, but the training data relies on strictly matched image pairs. Low-reference image enhancement reduces the requirement of image pairs to a certain extent. The input images can help unpaired training to learn information through a global-local discriminator structure [10]. However, we need to rigorously select unpaired reference images and investigate how to correlate common features of the reference images. Non-reference image enhancement is the most popular research direction in the field of low-light image enhancement in the past year. Some scholars have attempted to completely abandon the reference images. For example, maximum entropy theory [11] proposes that the maximum channel of the enhanced images is consistent with that of the original images. The theory is applied to the Retinex model to decompose the constant reflectivity. The estimated reflectance is the enhanced images obtained by this method, which does not consider the influence of real illumination. ZeroDCE [12] merges the dark images and the bright images to enhance the image brightness by drawing a nonlinear depth estimation curve. However, the method still needs to extract detailed lighting information from carefully selected bright images to draw a brightness curve. Therefore, decomposition and enhancement models face difficulties in predicting detailed reflection and natural brightness without reference images.
To address the above problem and improve the practical value of the non-reference dark images, this paper proposes a colour variation minimization retinex decomposition and enhancement with a multi-branch decomposition network (CvmD-net) based on retinex decomposition theory. First, an input constant feature prior mechanism (ICFP) is proposed to compensate for the lack of paired images. We use ICFP to extract constant features of the original images and optimize the decomposition training process. Second, to estimate the reflectance and illumination in the training process, the interaction between the two maps causes instability in the reflectance estimation. Therefore, we design two subnetworks with different structures and generate better image components. The reflectance subnet is used to learn the colour and detail characteristics of the original images. The illumination subnet is used to learn the initial exposure and structure of dark images. Finally, we propose colour variation minimization on the basis of structure consistency and unify the constant attribute between the input image and the estimated reflectivity. We refine the following innovative work:
1) To solve the dependence of the deep learning-based retinex algorithm on bright reference images, we first propose an input constant feature prior mechanism. This mechanism optimizes model decomposition and enhancement by extracting colour features and structural features from input low-quality images, and achieves an optimization effect that is not worse than that of the paired real reference images. The mechanism successfully replaces the reference images, greatly reducing the difficulty of constructing low-light image datasets.
2) To address reflectance estimation overfitting in the current retinex model during the process of training image decomposition, this paper designs a multi-branch decomposition network. The model is divided into a reflectance subnet and an illumination subnet. The reflectance subnet uses a U-net structure to estimate the reflectivity, and the illumination subnet uses a shallow convolutional neural network to estimate the illumination. The network structure reduces the negative effects caused by the interaction between reflectance and illumination, and estimates the reflectance with better quality.
3) To overcome noise interference in the enhanced images, this paper proposes a colour variation minimization loss function that uses the pixel intensity difference between the input-output images R, G, and B channels to adjust the predicted reflectance. This function effectively suppresses the amplified noise in the reflectance estimation and reduces colour distortion.
Scholars at home and abroad have long paid attention to dark image enhancement technology and have been applied to many scenes. The vigorous development of artificial intelligence disciplines has led to the development of traditional image restoration and data-driven image restoration.
Image contrast enhancement HE design an inverse equilibrium function for gray histogram. This function extends the original grayscale interval to the full range
Illumination estimation Retinex model is the primary method of brightness enhancement. Inspired by the human retina, it believes that brightness changes with the environment, and reflectivity is inherent property of object. The model reconstructs the image light level by adjusting the illumination. For example, Fu et al. [16] propose a new logarithmic transformation prior model to restore better original reflectivity. Given the weighted gradient term of the target stimulus signal, the model adopted an alternate minimization scheme to estimate the reflectance and illumination simultaneously. This method can suppress part of the noise, but the brightness is very poor in dark areas. Based on retinex theory, Guo et al. [17] propose a lighting estimation method based on reflection optimization (LIME). It designs different optimization strategies to iteratively predict illuminance. The structure prior uses the augmented Lagrange method to transform the optimized illumination into an optimization problem. This method considers only the brightness factor and can enhance the shadow details, but it requires cumbersome adjustment of the parameters to find the optimal value for a single image. Li et al. [18] add noise cancellation to the retinex model. This is the first attempt to estimate the measurement noise. Instead of logarithmic transformation, ADM based on the augmented Lagrange multiple was proposed to optimize the model. This method combines the image enhancement with denoising to over-smooth the texture of the image structure.
Homomorphic filtering This method enhances the high-frequency image information based on the frequency domain transformation of the acquired images, and transforms the enhanced result to preserve the spatial domain. For example, Tiwari et al. [19] apply homomorphic filtering to the medical image domain to sharpen local structure. It uses the average difference between the enhanced brightness and the original brightness to optimize the final illumination. Luan [20] study subimage segmentation and homomorphic filter cut-off frequency selection. They divide variance corresponding to different sensitivity regions into two subimages. The two subimages are subjected to independent homomorphic filtering with the cut-off frequency as the boundary. This method has strong applicability for restoring strong light areas of an image, but it also enlarges the useless dark area structures.
High-reference image enhancement Data-driven image recovery learns how to remove darkness from the trained images. On the basis of deep learning, researchers create different directions. For example, Gharbi et al. [9] propose that the two-sided network architecture seeks a balance between performance improvement and execution speed. This method improves the accuracy of prediction by increasing the color dimension, and provides two paths to extract global and local features. Wei et al. [21] are the first to implement retinex theory using deep learning network. This method uses a two-stage network to simulate image decomposition and brightness enhancement. The primary network biases to predict reflectance, and selects the predicted reflectance-weighted illumination. The two-stage network focuses on enhanced illumination with reflectivity guidance. Wang et al. [22] develop an encoder-decoder network to estimate the global illumination. The network splices the input images and global illumination into a large-core convolutional neural network to reconstruct image details. Chen et al. [23] choose two standard U-net networks to process full-resolution images and compare different denoising enhancement schemes through the controlled variable method. The input images collect the original sensor data to make a new dataset that contains short-exposure dark images and corresponding long-exposure matching images. To reduce colour distortion, Wang et al. [24] introduce adversarial optimization in the Retinex decomposition model. In this method, they fuse the feature of the reflectance map, the CRM correction image and the input image predicted by the decomposition network. The decomposition reflectance of the reference image enhance the result of the adversarial constraints. These methods can extract many reference features from paired data, but reference images are difficult to obtain in practice. Some models fuse different exposure images to produce enhanced images with more detail [1]–[3]. The output image can enrich the texture of dark area. The source image is preprocessed to produce a set of matching images. The optimization of different exposure image has obvious influence on the output image.
Low-reference image enhancement These methods consider the connection between dark images and arbitrary light images. For example, Jiang et al. [10] propose a dual-discriminator adversarial network to balance global and local features. This method lowers the mapping guidelines for matching image. A self-regularized perception loss for network training is used to normalize unpaired data extracted from the input images. Ignatov et al. [25] transform image enhancement into an image style transformation problem. A cyclic consistency confrontation network that maps the enhanced images back to the source image spaces are proposed to relax the need for paired images. Chen et al. [26] improve the bidirectional generative adversarial network with a more advanced global feature enhancement architecture. The model generator selects global features of the enhanced U-net to increase the advanced information and uses the adaptive weighting strategy A-WGAN to accelerate network training convergence. A generative adversarial network is introduced to generate enhanced images, which reduce the threshold of training data appropriately. However, compared with the paired image enhancement methods, the model volume is increased, and more computing resources are consumed.
Non-reference image enhancement Scholars have just begun to study non-reference images based on deep learning, while traditional method has matured to focus on single-frame image enhancement. For example, Zhang et al. [11] propose the maximum entropy constraint of the fusion of HE and retinex. This constraint also trains the depth model and improves brightness when there is only a single image. The constrained reflectance is used as the enhancement result. However, the model works only on the specific scale and format pictures. Guo et al. [12] convert image enhancement into the image depth curve estimation problem. The special image curve is trained by a convolutional neural network. The method fits a high-level pixel intelligence curve within the input dynamic range for pixel-level adjustments. Although this method does not require a reference image, the brightness must be increased by capturing bright images in the dataset.
The above high-reference image enhancement and low-reference image enhancement methods have achieved good results on low-resolution images, but they are difficult to generalize for practical applications. At present, it is relatively lacking on non-reference low-light image enhancement research. Moreover, we need to research further improvements in the qualitative and quantitative aspects.
Classical retinex theory is inspired by the human visual perception system. It suggests that the reflection of natural light should be taken into account in object imaging, as shown in Fig.1. The retinex theory can be expressed as
S=R⋅I | (1) |
where
Inspired by MUDE [27], this paper proposes a colour variation minimization retinex decomposition and enhancement with a multi-branch decomposition network to minimize colour variation. As shown in Fig.2, The CvmD-net includes four major modules: 1) Input constant feature prior mechanism (ICFP). To replace the reference images, we extract the average structural features of the input images based on structural consistency and map them to the decomposition model for optimization. 2) Multi-branch decomposition network (MD-net). We separate the image decomposition and propose a multi-branch decomposition model. The model predicts reflectance and illumination from the difference network. 3) Colour variation minimization. We propose a colour variation minimization function to constrain the minimum difference between the R, G, and B channels of reflectance. 4) Initial illumination equalization adjustment. We extract the average brightness from the equalized images and adjust the initial illumination to match the average brightness.
The proposed model is divided into two stages: original image decomposition and brightness enhancement. The decomposition expression
SD=LDrecon+Lic+Lci+Lr | (2) |
The reconstruction loss
SR=LRrecon+Lbc+Lbi | (3) |
The deep learning algorithm based on retinex decomposition theory [21] trains the decomposition network with paired low-/normal-light images during the image decomposition phase. The Low-decomposition and high-decomposition do not have feature intersections, but are executed alternately in sequence. We predict the colour feature and structural feature of the reflectance by the low-decomposition. The feature information is few when hidden in the dark. The high-decomposition guides the low-decomposition to restore the correct constant feature. When there is not normal-light image guidance, we attempt to obtain prior feature from the input image and constrain the predicted reflectivity. Since the constancy characteristic is the only property that determines the transaction, we disperse only a part of the darkness to restore the overall image content. According to (1), we assume initial illumination
We propose ICFP based on initial illumination
Rir=‖SIil+0.0001‖1 | (4) |
where
Inspred by Zhu et al. [29], we average the three RGB channel histograms of dark image to highlight the colour and structure. According to the equalized R, G and B channels and the maximum value of the three input channels, the decomposition network simulates the high decomposition process, and extracts the constant features of
Lic=λic∑i,j∈in,sy‖Ri⋅Ij−Sij‖1 | (5) |
where
The deep learning decomposition [21] model uses a single network to predict reflectance and illumination simultaneously. The two components share the extracted image features, and their interaction hinders the best decomposition results. A large amount of spatial noises are enhanced and retained in the reflectance component. Therefore, this paper proposes a multi-branch decomposition network (MD-net) to predict the reflectance and illumination separately. The network is divided into the reflectance subnetwork (Re-net) based on the U-net [30] and the illumination subnetwork (Il-net) based on the simple network [31].
We choose the reconstruction loss [32] as the basis function, which is based on retinex decomposition theory. The
LDrecon=∑i=in,sy‖Ri⋅Ii−Si‖1 | (6) |
where
Excessive structural information from the illumination map makes the enhanced image worse. The illumination must ensure the integrity of the global structure while smoothing the detailed information as much as possible. Here, we choose the illumination smoothness loss [21]:
Lci=λci‖∇Iin⋅G‖ | (7) |
G=exp(−λ⋅∇f(Sin)) | (8) |
where
The entire MD-net consists of two branch networks with different structures [33]. The low-light image first input a 3 × 3 convolutional layer to extract low-level image features; then, the standard U-net uses the features to map the reflectance and the convolutional neural network with 4 layers uses the features to map the illumination. The illumination structure smoothly selects the weighted illumination gradient of the reflectance gradient to keep the overall structure of the illumination consistent with the reflectance structure. Finally, the single-layer convolutional layer maps the eigenspace of U-net and estimates the reflectivity. The last layer of the shallow convolutional neural network is set up with 1 × 1 convolution kernel to map initial illumination.
The greyscale world theory [34] states that the average intensity of the three channels of the RGB images tends to the same greyscale. Inspired by this theory, we hypothesize that the difference in intensity between RGB primary colour channels is independent of luminance intensity. Based on differences in primary color channels, difference mapping can constrain the reflectivity of the observed image. The reflectance guided by this hypothesis can suppress the spatial noise to approximate the original level [35].
To preserve the colour fidelity and minimize the noise while predicting the reflectance, we propose a colour variation minimization loss. Based on the input constant feature prior, the function shows that the reflected RGB channel area tends to the average intensity. The expression is:
fr=‖ΔCR+S2omax{S2i,λr}‖1 | (9) |
ΔCR=∑a,b∈R,G,B(Δlab−Δhab)2 | (10) |
S2o=∑a,b∈R,G,BΔl2ab | (11) |
S2i=∑a,b∈R,G,BΔh2ab | (12) |
where
Based on the colour change minimization function, this paper selects the colour change of the input image
Lr=λrfr(Rir,Rin) | (13) |
where
In the brightness adjustment stage, this paper uses the colour variation minimization function by selecting the colour variation of the input image
Lbc=λbcfr(Sin,Rin⋅Io) | (14) |
where
The initial illumination brightness predicted by the multi-branch decomposition network is low and has many dark areas. To light up these areas, the common deep learning methods refer to the bright real image. This type of method overcomes the concurrency problem caused by dim light and is better than traditional method. The latest non-reference Retinex model [11] ignores initial lighting optimization. For this problem, the observed mean luminance value is selected as the reference brightness according to the ICFP mechanism. We constrain The light optimized by the brightness enhancement network (Ba-net) to approximate the natural light.
Considering that the images are full resolution in practical applications, the brightness enhancement network adopts the U-net structure. We reduce the scaling operation and perform only 2 downsamplings to simplify the network. The reconstruction function of the illumination from the Ba-net and the reflectance from the initial image is as follows:
LRrecon=‖Rin⋅Io−f(Sin)‖1 | (15) |
where
The redundant structure information in
Lbi=λbifs(Io,Rin) | (16) |
where
As shown in Fig.3, (b) and (d) are the reflection and illumination predicted by MD-net respectively, and (c) is the final illumination lit by Ba-net. The brightness of
We collate the research of similar algorithms in recent years, and select representative results to compare with the work in this paper. This article uses multiple evaluation criteria to assess performance.
1) Datasets
To verify the effect of the proposed method, we use LOL [21] for paired image enhancement experiments and ZeroDCE [12] for non-reference image enhancement experiments respectively. The evaluation images come from the widely used datasets DICM [36], LIME [17], LOL [21], MEF [37], SRIE [16], and VIP. We collect Skynet images at night to test the advantages of our method in the field of video surveillance.
LOL dataset The first dataset that contains the real images was organized by Zhang et al. [21] and made public on the Internet. All the images are 600 × 400. They change the exposure of the captured image by adjusting the parameters of the camera and combine the results into pairs. We select the fifteen of the 500 real images to evaluate the network. To expand the amount of data, 1000 synthetic image pairs are selected from RAISE [38].
ZeroDCE dataset The latest non-reference low-light image enhancement algorithm ZeroDCE [12] constructed a dataset according to its own characteristics. All 512 × 512 images are derived from part 1 of the public dataset SICE [39] and 360 multi-exposure sequences. The 2002 total images include the dark images, the normal images and the overexposure images.
Skynet image dataset To verify the superiority of our work in real-image application, we collect the 1512 night surveillance images from the Skynet system to establish a skynet image dataset. The sample images of the skynet image dataset are shown in Fig.4. For convenience during training, all images are uniformly adjusted to 600 × 400.
2) Evaluation metrics
To comprehensively validate our method from a quantitative perspective, We consider three different evaluation benchmarks to measure the comparative experiments, namely, LOE [40], PSNR [41], SSIM [42].
We use luminance order error (LOE) to measure the difference between enhanced and real light intensity. LOE is expressed as
LOE=1m×nm∑i=1n∑j=1RD(i,j) | (17) |
where
In the evaluation method without subjective judgment, we choose the peak signal-to-noise ratio (PSNR) to calculate the pixel difference after the original image is enhanced. The PSNR can highlight the effect of the image noise.
PSNR=10⋅log10(MAX2MSE) | (18) |
where
Structural similarity (SSIM) extends the pixel difference to the surrounding area and calculates the structural characteristic and the brightness change in image blocks.
SSIM=Lαxy⋅Cβxy⋅Sγxy | (19) |
where
3) Parameter settings
Our experiment is performed on an NVIDIA GTX 1080 GPU using the TensorFlow framework [43]. The batch size is 16, the patch size is 48,
To be clear about the true impact of our focused work on the imagery. This article discusses network loss and two-level network parameters.
1) Colour variation minimization loss
We show the results of training with the three loss of network architectures in Fig.5.
2) The effect of parameter setting changes
We evaluate the reflection estimation of parameters
1) Assessments on baseline datasets
This paper studies representative retinex algorithms in recent years and analyzes their excellent performance on LoL and ZeroDCE datasets. In the comparison, SRIE, LIME and Robust are traditional image decomposition-enhancement methods. Retinex-net and GLAD-net are deep learning methods without the ability to suppress noise. They train enhanced model by means of the high-reference normal-light image. This paper studies enhancement technology without reference image and need only low-light image. We select several different scene images from DICM [36] and LIME [17] and subjectively compare the enhanced image described above. In Fig.7 and Fig.8, (c) and (f) have a large deviation for the colours, and (c) and (e) is excessive for the brightness and noise. The local information is blocked with the white light. (d) is better than (c) and (4) for the brightness, but it has a few areas with overexposure remain. HE and GLAD-net cause serious colour variation. SRIE achieves insufficient enhancement of the original image brightness. LIME and Retinex-net are relatively complete to restore the image structure and texture, but overexposure occurs. RetinexDIP [44] focuses on enhancing the texture of the structure. Due to parameter limitations, the light changes of the enhanced image are the most deliberate. This paper adopts the non-reference image to enhance the image colour recovery the best. Compared with (e) and (g), (h) is closer to the natural image. The edges of the image texture obtained by our method are smoother, which may be the inevitable reduction of edge differences while suppressing noise.
2) Evaluation on surveillance images
We use the low-resolution images to conduct the above experiments, while many high-resolution images exist in real scenes. We use a real dataset based on the Skynet images to train our algorithm, and several low-light images with a scale of 3392 × 2072 are collected from video surveillance as the test data. As shown in Fig.9, our work shows good generalization ability in practical applications.
3) Numerical analysis
We perform many experiments on baseline datasets, and use mainstream metrics to perform quantitative comparisons on the multiple public test sets. Table 1 lists the values of the three baseline evaluation test comparison methods. The data in the table quantifies the luminance fidelity, pixel-level distortion, and structural similarity of the enhanced observation images. Among them, we change the text colour to highlight the best value on each benchmark. The bold font in the table indicates the best result. Our method is the highest for The SSIM and LOE. The result indicates that our method focuses on restoring the structure and texture characteristics and the enhanced illumination is more in accordance with the changing law of the natural light.
Method | SRIE | LIME | Robust | Retinex-net | GLAD-net | Zhangetal. | ZeroDCE | RetinexDIP | Ours |
PSNR↑ | 14.41 | 16.07 | 15.06 | 12.05 | 14.63 | 17.27 | 16.22 | 17.21 | 17.25 |
SSIM↑ | 0.54 | 0.59 | 0.62 | 0.51 | 0.66 | 0.75 | 0.61 | 0.72 | 0.76 |
LOE↓ | 268.33 | 936.34 | 355.22 | 570.06 | 237.62 | 878.18 | 494.78 | 550.38 | 230.4 |
Fig.10 shows the change in loss as iteration increases, Retinex-Net and ours are both decomposition and enhancement models. We record the change in decomposition loss and enhanced loss. It is obvious that the loss of each method drops significantly at the beginning, and our decomposition loss is less volatile than other methods at a gentle time, with stable iterations.
In this paper, we propose a two-stage framework CvmD-net which is independent of reference images, and optimize MD-net to estimate reflectance and illuminance by extracting invariant features of original images. The two sub-decomposition networks share the image preprocessing layer and use different frames to extract high-level features. Additionally, we propose a colour variation minimization regularization condition that uses low-light images to constrain the reflectance estimation. The paper also proposes an input constant feature prior mechanism that takes synthesized high-contrast/brightness information to guide the network decomposition and brightness adjustment. Extensive experiments have proven the superiority of our method in the application of uneven lighting.
In the future work, we may begin to recover image details under extremely dark conditions and extend our algorithm to low-light video enhancement.
[1] |
J. Huang, P. F. Zhu, M. R. Geng, et al., “Range scaling global U-Net for perceptual image enhancement on mobile devices,” in Proceedings of the European Conference on Computer Vision, Munich, Germany, pp.230–242, 2018.
|
[2] |
K. Singh, A. Seth, H. S. Sandhu, et al., “A comprehensive review of convolutional neural network based image enhancement techniques,” in 2019 IEEE International Conference on System, Computation, Automation and Networking (ICSCAN), Pondicherry, India, pp.1–6, 2019.
|
[3] |
C. Chen, Q. F. Chen, M. Do, et al., “Seeing motion in the dark,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), pp.3184–3193, 2019.
|
[4] |
G. Z. Tiron and M. S. Poboroniuc, “Neural network based traffic sign recognition for autonomous driving,” in 2019 International Conference on Electromechanical and Energy Systems (SIELMEN), Craiova, Romania, pp.1–5, 2019.
|
[5] |
Y. T. Kim, “Contrast enhancement using brightness preserving bi-histogram equalization,” IEEE Transactions on Consumer Electronics, vol.43, no.1, pp.1–8, 1997. DOI: 10.1109/30.580378
|
[6] |
M. Abdullah-Al-Wadud, H. Kabir, M. A. A. Dewan, et al., “A dynamic histogram equalization for image contrast enhancement,” in 2007 Digest of Technical Papers International Conference on Consumer Electronics, Las Vegas, NV, USA, pp.1–2, 2007.
|
[7] |
D. J. Jobson, Z. Rahman, and G. A. Woodell, “Properties and performance of a center/surround retinex,” IEEE Transactions on Image Processing, vol.6, no.3, pp.451–462, 1997. DOI: 10.1109/83.557356
|
[8] |
R. Bhadu, R. Sharma, S. K. Soni, et al., “Sparse representation and homomorphic filter for capsule endoscopy image enhancement,” in 2017 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India, pp.1178–1182, 2017.
|
[9] |
M. Gharbi, J. W. Chen, J. T. Barron, et al., “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics, vol.36, no.4, article no.articleno.118, 2017. DOI: 10.1145/3072959.3073592
|
[10] |
Y. F. Jiang, X. Y. Gong, D. Liu, et al., “EnlightenGAN: Deep light enhancement without paired supervision,” IEEE Transactions on Image Processing, vol.30, pp.2340–2349, 2021. DOI: 10.1109/TIP.2021.3051462
|
[11] |
Y. Zhang, X. G. Di, B. Zhang, et al., Self-supervised image enhancement network: Training with low light images only, arXiv preprint arXiv: 2002.11300, 2020, doi: 10.48550/arXiv.2002.11300.
|
[12] |
C. L. Guo, C. Y. Li, J. C. Guo, et al., “Zero-reference deep curve estimation for low-light image enhancement,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp.1777–1786, 2020.
|
[13] |
S. K. Naik and C. A. Murthy, “Hue-preserving color image enhancement without gamut problem,” IEEE Transactions on Image Processing, vol.12, no.12, pp.1591–1598, 2003. DOI: 10.1109/TIP.2003.819231
|
[14] |
T. Celik and T. Tjahjadi, “Contextual and variational contrast enhancement,” IEEE Transactions on Image Processing, vol.20, no.12, pp.3431–3441, 2011. DOI: 10.1109/TIP.2011.2157513
|
[15] |
C. Lee, C. Lee, and C. S. Kim, “Contrast enhancement based on layered difference representation of 2D histograms,” IEEE Transactions on Image Processing, vol.22, no.12, pp.5372–5384, 2013. DOI: 10.1109/TIP.2013.2284059
|
[16] |
X. Y. Fu, D. L. Zeng, Y. Huang, et al., “A weighted variational model for simultaneous reflectance and illumination estimation,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp.2782–2790, 2016.
|
[17] |
X. J. Guo, Y. Li, and H. B. Ling, “LIME: Low-light image enhancement via illumination map estimation,” IEEE Transactions on Image Processing, vol.26, no.2, pp.982–993, 2017. DOI: 10.1109/TIP.2016.2639450
|
[18] |
M. D. Li, J. Y. Liu, W. H. Yang, et al., “Structure-revealing low-light image enhancement via robust retinex model,” IEEE Transactions on Image Processing, vol.27, no.6, pp.2828–2841, 2018. DOI: 10.1109/TIP.2018.2810539
|
[19] |
M. Tiwari and B. Gupta, “Brightness preserving contrast enhancement of medical images using adaptive gamma correction and homomorphic filtering,” in 2016 IEEE Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), Bhopal, India, pp.1–4, 2016.
|
[20] |
R. P. Luan, “An enhancement algorithm for non-uniform illumination image based on two homomorphic filters,” in 2016 2nd Workshop on Advanced Research and Technology in Industry Applications (WARTIA 2016), Dalian, China, pp.1213–1216, 2016.
|
[21] |
C. Wei, W. J. Wang, W. H. Yang, et al., “Deep retinex decomposition for low-light enhancement,” in British Machine Vision Conference 2018, Newcastle, UK, pp.155–167, 2018.
|
[22] |
W. J. Wang, C. Wei, W. H. Yang, et al., “GLADNet: low-light enhancement network with global awareness,” in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi'an, China, pp.751–755, 2018.
|
[23] |
C. Chen, Q. F. Chen, J. Xu, et al., “Learning to see in the dark,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.3291–3300, 2018.
|
[24] |
J. Y. Wang, W. M. Tan, X. J. Niu, et al., “RDGAN: Retinex decomposition based adversarial learning for low-light enhancement,” in 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China, pp.1186–1191, 2019.
|
[25] |
A. Ignatov, N. Kobyshev, R. Timofte, et al., “WESPE: Weakly supervised photo enhancer for digital cameras,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, pp.691–700, 2018.
|
[26] |
Y. S. Chen, Y. C. Wang, M. H. Kao, et al., “Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, pp.6306–6314, 2018.
|
[27] |
F. L. Luo, Z. H. Zou, J. M. Liu, et al., “Dimensionality reduction and classification of hyperspectral image via multistructure unified discriminative embedding,” IEEE Transactions on Geoscience and Remote Sensing, vol.60, article no.5517916, 2022. DOI: 10.1109/TGRS.2021.3128764
|
[28] |
G. Q. Qi, L. Chang, Y. Q. Luo, et al., “A precise multi-exposure image fusion method based on low-level features,” Sensors, vol.20, no.6, article no.1597, 2020. DOI: 10.3390/s20061597
|
[29] |
Z. Q. Zhu, H. Y. Wei, G. Hu, et al., “A novel fast single image dehazing algorithm based on artificial multiexposure image fusion,” IEEE Transactions on Instrumentation and Measurement, vol.70, article no.5001523, 2021. DOI: 10.1109/TIM.2020.3024335
|
[30] |
O. Ronneberger, P. Fischer, and T. Brox, “U-Net: convolutional networks for biomedical image segmentation,” in 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Munich, Germany, pp.234–241, 2015.
|
[31] |
C. Y. Li, J. C. Guo, F. Porikli, et al., “LightenNet: A convolutional neural network for weakly illuminated image enhancement,” Pattern Recognition Letters, vol.104, pp.15–22, 2018. DOI: 10.1016/j.patrec.2018.01.010
|
[32] |
B. L. Cai, X. M. Xu, K. L. Guo, et al., “A joint intrinsic-extrinsic prior model for Retinex,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp.4020–4029, 2017.
|
[33] |
F. L. Luo, H. Huang, Z. Z. Ma, et al., “Semisupervised sparse manifold discriminative analysis for feature extraction of hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol.54, no.10, pp.6197–6211, 2016. DOI: 10.1109/TGRS.2016.2583219
|
[34] |
G. Buchsbaum, “A spatial processor model for object colour perception,” Journal of the Franklin Institute, vol.310, no.1, pp.1–26, 1980. DOI: 10.1016/0016-0032(80)90058-7
|
[35] |
Z. Q. Zhu, H. P. Yin, Y. Chai, et al., “A novel multi-modality image fusion method based on image decomposition and sparse representation,” Information Sciences, vol.432, pp.516–529, 2018. DOI: 10.1016/j.ins.2017.09.010
|
[36] |
C. Lee, C. Lee, and C. S. Kim, “Contrast enhancement based on layered difference representation,” in 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA, pp.965–968, 2012.
|
[37] |
K. D. Ma, K. Zeng, and Z. Wang, “Perceptual quality assessment for multi-exposure image fusion,” IEEE Transactions on Image Processing, vol.24, no.11, pp.3345–3356, 2015. DOI: 10.1109/TIP.2015.2442920
|
[38] |
D. T. Dang-Nguyen, C. Pasquini, V. Conotter, et al., “RAISE: A raw images dataset for digital image forensics,” in Proceedings of the 6th ACM Multimedia Systems Conference, New York, NY, USA, pp.219–224, 2015.
|
[39] |
J. R. Cai, S. H. Gu, and L. Zhang, “Learning a deep single image contrast enhancer from multi-exposure images,” IEEE Transactions on Image Processing, vol.27, no.4, pp.2049–2062, 2018. DOI: 10.1109/TIP.2018.2794218
|
[40] |
Z. Q. Ying, G. Li, and W. Gao, A bio-inspired multi-exposure fusion framework for low-light image enhancement, arXiv preprint arXiv: 1711.00591, 2017, doi: 10.48550/arXiv.1711.00591.
|
[41] |
Y. Tai, J. Yang, X. M. Liu, et al., “MemNet: A persistent memory network for image restoration,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp.4549–4557, 2017.
|
[42] |
Z. Wang, A. C. Bovik, H. R. Sheikh, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol.13, no.4, pp.600–612, 2004. DOI: 10.1109/TIP.2003.819861
|
[43] |
M. Abadi, P. Barham, J. M. Chen, et al., “TensorFlow: a system for large-scale machine learning,” in Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, GA, USA, pp.265–283, 2016.
|
[44] |
Z. J. Zhao, B. S. Xiong, L. Wang, et al., “RetinexDIP: A unified deep framework for low-light image enhancement,” IEEE Transactions on Circuits and Systems for Video Technology, vol.32, no.3, pp.1076–1088, 2022. DOI: 10.1109/TCSVT.2021.3073371
|
1. | Xu, J., Rao, Y., Zhou, J. et al. Transferable Unintentional Action Localization with Language-guided Intention Translation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. DOI:10.1109/TPAMI.2025.3538675 |
2. | Abhishek, K.L., Geetanjali, R., Sushitha, S. et al. Anti-theft System Using Structural Similarity. EAI/Springer Innovations in Communication and Computing, 2024. DOI:10.1007/978-3-031-54696-9_11 |
Method | SRIE | LIME | Robust | Retinex-net | GLAD-net | Zhangetal. | ZeroDCE | RetinexDIP | Ours |
PSNR↑ | 14.41 | 16.07 | 15.06 | 12.05 | 14.63 | 17.27 | 16.22 | 17.21 | 17.25 |
SSIM↑ | 0.54 | 0.59 | 0.62 | 0.51 | 0.66 | 0.75 | 0.61 | 0.72 | 0.76 |
LOE↓ | 268.33 | 936.34 | 355.22 | 570.06 | 237.62 | 878.18 | 494.78 | 550.38 | 230.4 |