Remote Sens. 2021, 13, 1619 5 of 23
training set, data enhancement processing was carried out to the data set to better extract
the features of apples belonging to different labeled categories and avoid the over-fitting of
the model obtained from training.
Table 2. Detailed information of images in test set.
Test Set Sunny Cloudy Total
Number of images 100 100 200
Graspable apple 482 525 1007
Ungraspable apple 766 563 1329
Due to the uncertain factors, such as illumination angle and weather, resulting in the
light environment of image acquisition is extremely complex; in order to improve the gen-
eralization ability of apple targets detection model, several image enhancement methods
were utilized for the 1014 images of training set respectively based on MATLAB (version
2016, the MathWorks Inc., Natick, MA, USA) software and its related image processing
functions. The image enhancement methods include image brightness enhancement and
reduction, horizontal mirroring, vertical mirroring, multi-angle rotation (90
◦
, 180
◦
, 270
◦
)
etc. In addition, considering the noise generated by the image acquisition equipment in the
process of image acquisition and the blur of the captured images caused by the shaking
of the equipment or the branches, Gaussian noise with variance of 0.02 was added to the
images, and the motion blur processing was carried out. Detailed procedures of image
enhancement methods are illustrated in the following.
Image brightness enhancement and reduction: Firstly, the original image is converted
to HSV space by using ‘rgb2hsv’ function; secondly, the V component (brightness com-
ponent) of the image is multiplied by different coefficients; finally, the synthesized HSV
space image is converted to RGB space by using ‘hsv2rgb’ function, realizing the brightness
enhancement and reduction of the image. In the study, three brightness intensities can be
generated utilizing brightness enhancement, including (H + S + 1.2
×
V),
(H + S + 1.4 × V)
and (H + S + 1.6
×
V); two brightness intensities can be generated using brightness reduc-
tion, including (H + S + 0.6 × V) and (H + S + 0.8 × V).
Image mirroring (horizontal and vertical mirror) was implemented using the Matlab
function ‘imwarp’. The horizontal mirroring was implemented by transforming the left and
right sides of the image centering on the vertical line of the image. The vertical mirroring
was implemented by transforming the upper and lower sides of the image centering on the
horizontal centerline of the image.
For image rotation, the Matlab function ‘imrotate’ was used to rotate the raw image,
and 90
◦
, 180
◦
, and 270
◦
of rotation were achieved by changing the function parameter
‘angle’, respectively. The transformed images can improve the detection performance of
the model by correctly identifying the apples of different orientations.
Four kinds of motion blur processing were employed to make the convolutional
network model have strong adaptability with the blurred images. A predetermined two-
dimensional filter was created using the Matlab function ‘fspecial’. LEN (length, represents
pixels of linear motion of camera) and THETA (
θ
, represents the angular degree in a
counter-clockwise direction) of the motion filter were set as (6, 30), (6,
−
30), (7, 45) and
(7, −45),
respectively. Then, the Matlab function ‘imfilter’ was used to blur the image with
the generated filter.
Furthermore, the addition of Gaussian noise with variance of 0.02 to the raw images
was implemented using Matlab function ‘imnoise’.
The final training sets consist of 16,224 images used as the final training set data for
training of apple targets recognition model, including 15,210 enhanced images and 1014
raw images. The detailed distribution of training set data is shown in Figure 3. There was
no overlap between the training set and the test set.