A Robust Method for Adaptive Center Extraction of Linear Structured Light Stripe

，，，，

1.College of Mechanical and Electrical Engineering，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China；2.State-Owned Jinjiang Machine Factory，Chengdu 610043，P.R.China（Received 15 May 2020；revised 30 June 2020；accepted 25 July 2020）

Abstract: In the non-contact measurement using the linear structured light（LSL），the extraction precision of the light stripe center directly affects the measurement accuracy of the whole detection system. To solve the problem that general algorithms cannot accurately extract the center of the light stripe with the uneven width and unstable greyvalue distribution，an adaptive optimization method is proposed. In this method，the stripe region is firstly segmented，and the widths of the laser stripe are calculated by boundary detection. The initial stripe center points are computed by the quadratic weighted grayscale centroid method based on the self-adaptive stripe width. After that，these center points are optimized according to the determined slope threshold. The sub-pixel coordinates of these center points are recalculated. Detailed analysis is also performed in line with the proposed evaluation index of the extraction algorithm.The experimental results show that the mean square error of extracted center points is only 0.1 pixel，meaning that the accuracy of laser stripe center extraction is improved significantly by the method. Furthermore，the method can run effectively at a relatively low computational time cost，and can demonstrate great robustness as well.Key words:laser stripe；image processing；center extraction；self-adaptive；evaluation index

0 Introduction

In optical non-contact measurement，due to its high precision，good real-time and easy to realize，the linear structure light（LSL）mode is widely used in many fields，such as automated welding［1］，dimensional measurement［2］，3D reconstruction of object surfaces［3］，surface quality inspection［4］，food detection［5-6］， etc. Since the laser stripe photographed by the CCD camera has the width of a few or tens pixels，whether or not the stripe center can be accurately extracted will largely determine the measurement reliability of the entire visual inspection system. Therefore，it is necessary to consider the accuracy，stability and efficiency of the laser stripe extraction algorithm in the implementation process for real-time vision measurement based on the linear structure light.

The laser stripe center extraction algorithm can be divided into two categories:traditional algorithm and improved algorithm. The traditional methods mainly include skeleton refinement method，geometric center method， extreme value method，etc.［7］These algorithms generally rely on the geometric contour or grayscale maximum value of the stripe to obtain the position of the center points.They are simple to implement，fast to extract，but poor in accuracy and sensitive to noise. Besides，the direction template method，the Steger method and the grayscale centroid method，which have relatively more accurate extraction results but poor realtime，fall into the category of conventional methods as well. Using the direction template method to obtain the stripe center can significantly suppress the influence of noise in the extraction process［8］. However，the efficiency suffers due to a large number of template-based mathematical calculations，or the accuracy is not satisfactory because of a basis on only four direction templates，that is，vertical，horizontal，left 45° and right 45°. The Steger method［9］，which is designed to extract the sub-pixel center points by performing convolution on every image pixel with the Hessian matrix. During this process，the eigenvalues and eigenvectors of each Hessian matrix need to be solved. Although its accuracy is as high as sub-pixel level，the computational complexity is also significantly higher，resulting in a long calculation time. In order to improve the efficiency of the Steger method，the graphics processing unit（GPU）can be applied for the stripe center inspection，which is used to remarkably shorten the processing time of line detection and center extraction［10］. Methods based on parabolic，Gaussian’spline interpolation or template matching are even worse than the easily computable threshold operation，while the grayscale centroid method supplies best results in practical application［11］. In addition，the grayscale centroid method has become one of the commonly used methods for laser stipe center extraction due to its high-accuracy，efficiency and easy to implement［12］. In the study of improved grayscale centroid method，the dynamic least square method is used to fit the center points determined by the grayscale centroid method［13］. Meanwhile，a rectangular frame at each initial center point is defined，two sides of which are parallel to the tangent vector and the normal vector at the center point respectively. The length of the rectangular side parallel to the tangent vector is obtained by the radius of curvature of the point. Finally，the exact sub-pixel positions of the center points are calculated along the normal vector direction inside the rectangle. A robust laser stripe extraction method based on the grayscale centroid method is proposed for 3D reconstruction［14］. The center points are extracted preliminarily by the thinning algorithm，of which the normal directions are also calculated by using the Hessian matrix. Then the final accurate center points can be obtained along this direction. This improved method can be run in real time and has a better performance than the simplified Hessian matrix method.

In addition to the above improved methods，there are also many other measures for the center extraction of the laser stripes. By analyzing the characteristics of laser fringes，a stripe center extraction method of multi-Gaussian signal fitting based on wavelet transform is proposed［15］. Different expectations and variances are used to obtain the best fitting result of each laser stripe cross-section and the position of the stripe center point is determined based on the symmetry center of the fitted Gaussian signal.Based on an ideal light intensity model，the center point of each cross-section of the light stripe is calculated according to the conservation laws of Legendre moment［16-17］. Meanwhile，a smoothing spline algorithm is applied to eliminate the noise.The boundary-connection is used to determine the light stripe region，and the improved split-and-Merge approach is used to extract the laser stripe center-line on the detected object［18］. The experimental results show that the proposed method is fast，robust and accurate. To solve the problem that the laser stripe center is difficult to be accurately extracted due to the strong specular reflection on the metal surface，a collar-wise laser stripe extraction method based on Gaussian filter and sub-pixel interpolation is put forward［19］，which is suitable for real-time online detection of 3D morphology in the industrial environment.

In practical applications，the light intensity distribution of the laser stripe，which is projected by the ordinary laser projector，is often uneven. The width of each cross-section of the laser stripe changes obviously. In addition，the reflection condition of the detected objects is usually quite different. Therefore，it is easy to lead to some problems such as inaccurate extraction of the laser stripe center，obvious defect of broken line and so on. As can be concluded from the above analysis，few researches discussed this problem. What’s more，the optimization effect，accuracy and reliability of the above methods need to be improved. In this paper，an adaptive optimization algorithm for extracting the laser stripe center is proposed. Firstly，the region of interest（ROI）of the stripe image is determined. The initial center points of the stripe，which are extracted by using the quadratic weighted grayscale centroid method based on self-adaptive width，are recalculated according to the computed slope threshold. Finally，the sub-pixel center points with high accuracy can be achieved by performing the average-value optimization method. The proposed method and other traditional methods are analyzed and compared in a detail and comprehensive way，which is on the basis of the well-rounded evaluation index proposed in this paper.

1 Image Preprocessing

The detailed flowchart of optimization algorithm in this paper is shown in Fig.1. Firstly，the laser stripe image captured by a CCD industrial camera is sequentially subjected to operations such as median filtering，threshold segmentation，as well as morphological closing operation.And ROI of the image is determined after analyzing the features of it.Then the width of each cross-section of laser stripe is obtained using the edge detection method. Meanwhile，initial central points are extracted by applying adaptive width quadratic weighted center of gravity method. Finally，the accurate sub-pixel coordinates of all central points are acquired，using the average optimization algorithm.

Fig.1 Flowchart of the proposed optimization method

1.1 Characteristics of laser stripe

Under ideal condition，the light-intensity in the cross section of a laser stripe exhibits a Gaussian distribution. However，under practical circumstance，due to the interference of the field environment，the quality of the laser，the material of the measured object surface and other factors，the actually acquired image of light stripe is shown in Fig.2（a）. And the grey-value distribution in a representative cross-section is shown in Fig.2（b）. It can be seen that，the closer the pixels are to the light stripe center，the faster their grey-values increase，and there are multiple maximum grey-values at the center. The greyvalue on both sides of the stripe is almost zero. Generally，the distribution of the grey-value is similar to a Gaussian distribution.

Fig.2 Laser stripe and grey-value distribution in a cross-section

1.2 Preprocessing method

After completing the median filtering operation，image binarizationa，a global adaptive image segmentation algorithm is adopted to obtain the binary image of a laser stripe in this paper. The algorithm， using the maximum inter-class variance method，i.e.，Otsu method，calculates the optimal threshold of image segmentation，which maximizes the difference between foreground and background of an image. The greater the inter-class variance between the two parts is，the greater the difference of them will be. When the largest inter-class variance is obtained，the probability of misclassifying the foreground and background of an image is the smallest. The equation for calculating the inter-class variance is

wherew0is the proportion of foreground pixel points in an image，u0the average grey-value of the foreground in an image，w1the ratio of background pixel points in an image，u1the average grey-value of the background in an image，anduthe total average grey-value of an image，which can be expressed as

Denoted bytis the segmentation threshold of the image foreground and background，which is the optimal segmentation threshold of an image when makinggin Eq.（1）reaches the maximum. The laser image is segmented，according to the optimal segmentation thresholdt，by using the global adaptive binarization segmentation algorithm，shown as

wheref（x，y）is the grey-value of a pixel point with image coordinate（x，y），andG（x，y）the grey-value of a pixel point after image segmentation.

In order to bridge the narrow discontinuity and slender gully of the stripe，eliminate the small holes，and smooth the contour of the binary stripe at the same time，the morphological closure operation is applied in this paper. Morphological closure operation is an image processing technique that combines two morphological operations（morphological dilation and morphological corrosion）. Its mathematical model is expressed as

whereAis the image to be processed andBthe structural element of morphological operation.

Morphological dilation operation can be expressed as

Morphological corrosion operation can then be expressed as

Morphological closure operation is performed on the image of the binary laser stripe，and the result image is shown in Fig.3.

Fig.3 Binary laser stripe (enlarged)

The method to determine the laser stripe region（i.e.，ROI）is to retrieve the grey-valueG（x，y）of the pixels of the binary light stripe column by column. The pixels are scanned from the left end of the stripe until the first pixel with the grey-valueG（x，y）=255 is detected，and the pixels are scanned from the right until the last one with the grey-valueG（x，y）=255 is detected in the last column of the stripe. These two obtained pixels’coordinates are marked as（xmin，ymin）and（xmax，ymax）respectively. Therefore，the set of pixel points within the ROI region can be expressed as

It can be seen from the processed laser stripe image that the width varies along the length of the stripe. For this reason，the optimization algorithm in this paper obtains the width of each cross-section of the stripe by detecting the upper and lower boundary of it，and extracts the center of each cross-section within its width range. Inside ROI of image，the binary laser stripe is scanned column by column to obtain the upper and lower boundary，and the width of each cross-section of the stripe（denoted bywi）is computed，as shown in Fig.4.

Fig.4 Schematic diagram of width calculation

Then the pixel set of theith column required for calculating the center point’s coordinate is expressed as

whereyi，upandyi，downare respectively the upper and the lower boundaryycoordinates of the stripe，wi=yi，down-yi，up.

2 Light Stripe Center Extraction

2.1 Adaptive extraction for sub-pixel center of laser stripe

Considering the inconsistent stripe width along the extension direction and small grey-value difference at the stripe center，the quadratic weighted centroid method based on the adaptive width is adopted in this paper to extract sub-pixel center points of the laser stripe. The computation principle of the method is shown in Fig.5. The widthwiof each crosssection of the laser stripe and the pixel setG（i，y）of theith column have been obtained from the above parts. The coordinates of each center point in every column of the laser stripe which are denoted byPiin the schematic diagram are calculated by using the method. The dots in the graph are the extracted center points of the stripe，which are all at sub-pixel level.

Fig.5 Principle for center extraction using self-adaptive quadratic weighted centroid method

The equation for calculating the coordinate of the center point of each cross-section is written as

whereyi，upis theycoordinate of the upper boundary of each cross-section of the laser stripe（under image coordinate system），withe width of theith cross-section of the stripe，G（i，y）the grey-value of the pixel whose image coordinate isyin theith cross-section of the stripe，andy′centerthe initial center coordinate of each cross-section of the laser stripe.

The distribution of the center points preliminarily extracted by Eq.（9）is shown in Fig.6. It shows that the fluctuation of these center points is relatively apparent.

Fig.6 Distribution of extracted center points

2.2 Optimization for center points

Some of these initial center points deviate from the center position of the laser stripe due to bad quality of the laser projector. In order to ensure the highaccuracy of the fitted center line，the optimization for these center points is performed in this paper.The initial center points are optimized in two steps.

The first step is calculating the meany-coordinate of center points within a certain region，the region moves one pixel every time and its size is fixed.The calculation process can be expressed as follows

whereyc′ente（rm）is the coordinate of the initially extracted center point in the（m+1）th column of the laser stripe，athe domain size for calculating the avera gey-coordinate of the center points，which is generally an odd number（a≥1），the coordinate of the initially extracted center point in thexth column of the laser stripe，andthe coordinate of the center point in the（m+1）th column after optimization.

The second optimization step is to judge one center point whether has a large deviation according to a slop threshold，if so，the coordinate of the point need to be re-calculated. The slope threshold is determined by calculating the approximate slope of the center line fitted in the above part. Final center points’coordinates are obtained by

wherek0is the slope threshold of the laser stripe，the value of which is determined according to the tilt angle of the laser stripe and the selection rule of the coordinate axis，and the optimized structure can not be affected by the selection ofk0value.kis the slope of center points within a certain region，and this region also moves one pixel every time and its size is fixed.bis a constant which represents the step size of re-calculating the point’s coordinate，and it needs to be confirmed in the experimentis the final coordinate of the center point in the（m+1）th column of the laser stripe.

According to Eq.（9），the initial coordinates of center points are obtained， and Eq.（10） and Eq.（11）are used to adjust the coordinates of some center points whose deviations are a bit larger.

After the above optimization operation，the distribution of the extracted center points of the laser stripe is shown in Fig.6. It can be seen that the fluctuation of center points decreases compared with that before optimization. Therefore，the effect of the proposed optimization operation in this paper is obvious.

3 Experiments and Discussion

The visual measurement system based on LSL set up in this paper is shown in Fig.7. In the experiment，VS 2013 is taken as the development platform，and the programming language C++ is utilized to perform the proposed optimization method，wherein，some data structures and common image process algorithms from OpenCV are also used. A high-precision CCD industrial camera， with 1 292 pixel×964 pixel resolution and 3.75 μm×3.75 μm pixel size，is used. The megapixel lens with 8 mm focal length is selected. The laser maximum power is 30 mW and the wavelength of the laser light is 650 nm（red light）.

In order to verify the extraction effect of the proposed method， skeleton refinement method，grayscale centroid method and Hessian matrix method are compared.

Fig.7 Experimental measurement system based on LSL

3.1 Computational complexity

Time complexity and space complexity are important evaluation systems and evaluation criteria for computational complexity.

Time complexity，one of the key technologies of visual detection，whether or not the laser stripe extraction method is high-efficient is an important aspect to be considered. In order to determine whether these extraction methods can be applied in real time，the runtime of them must be calculated.

In this experiment，the computer for the image processing has a CPU of Intel i5-4210U 1.7 GHz and a RAM of 4 GB. For each laser image，the center extraction program runs 10 times，and the average time of different center extraction methods is shown in Table 1.

Table 1 Aaverage time of different center extraction methods

As shown in Table 1，the Hessian matrix method is the most time-consuming，which cannot run in real time，for a large amount of convolution is needed in extracting the center points of the laser stripe.Using the skeleton refinement method and the grayscale centroid method to determine the position of center points also takes a bit longer time. On the contrary，the proposed optimization method simplifies the process of extracting center points. Therefore，the proposed method not only ensures the extraction accuracy，but also makes the running time shorter. Namely，it needs relatively low computational time cost to finish the extraction process.

For space complexity，according to the image acquisition equipment and computer configuration in this paper，the maximum memory for the algorithm of proposed method is 68 MB，the output image resolution is 1 292 pixel×964 pixel，and the algorithm program size is 425 KB.

3.2 Extraction accuracy

Fig.8 shows that the laser stipe center line extracted by the proposed method is at the actual center position of the laser stripe. It proves that the optimization method can effectively adjust center points’positions to the center of the laser stripe when these points deviate from the center significantly.

Fig.8 Center line of light stripe extracted by the proposed method

To test whether the center point extraction of one algorithm is accurate or not，the standard deviationσ，which is defined as the mean square error of the distance between extracted center points and the fitted straight line，needs to be calculated as

wherediis the distance from thei-th center point to the fitted line，the average distance between center points and the fitted line，andnthe number of extracted center points.diis obtained from the calculation formula of the distance between the point and the line，sodican then be calculated by

wherexiandyiare thex-coordinate andy-coordinate of theith center point，respectively.A，BandCare the coefficients of the linear equationAx+By+C=0.

The purpose of the algorithm itself is to extract clearer center points and fit centerline of laser stripe.Compared with original laser stripe，the optimized centerline is smoother and straighter，which is more in line with the actual situation，and there will be no over-optimization. The distributions of the center points extracted by the proposed optimization method and the other traditional methods are shown in Fig.9. Obviously，the center points extracted by the proposed method have the smallest volatility，while the grayscale centroid method has noticeable burr and broken-line problem. In addition，the skeleton refinement method and the Hessian matrix method demonstrate the characteristic of step-like variation，with poor accuracy and stability.The standard deviationsσof each algorithm are listed in Table 2. Compared with the traditional methods，the standard deviation of the proposed method is the smallest，which means that its extraction precision of laser stripe center points is the best.

Fig.9 Distribution of center points extracted by each method

Table 2 Standard deviation of distance from extracted center points to fitting line

3.3 Repeatability error

The repeatability error（Er）of an algorithm refers to the deviation of the center points extracted at different moments，which reflects the ability of the algorithm to resist random noise at different time.Ercan be represented as

wherepis the total number of laser stripe images taken by CCD camera at different moments.

The errorErjcan be obtained by

whereyj，iis the coordinate of theith center point of thejth laser image，andnthe number of extracted center points of the laser stripe.

As for laser stripe images captured at different time，the repeatability errorErof four different extraction methods are calculated，as shown in Table 3. Compared with the traditional methods，the proposed method retains the minimum repeatability error. So it can extract laser stripe center points with the best accuracy and reliability.

Table 3 Repeatability error of extracting center points at different time

3.4 Robustness

In order to verify the robustness of the proposed method，parameters of the measurement system and the experiment environment are changed.Firstly，the laser power is adjusted to the maximum，medium and minimum，and the center points of the stripe are extracted by this method at different power. These extracted points are fitted to a straight line by performing the least square method. The fitted center line is shown in Fig.10. It can be seen that the obtained center line does not deviate from the actual central position with the change of the laser power. The error values in Table 4 fluctuate to some extent，but the error is small，which can ensure high precision. Therefore，these data show that the proposed method can provide a good result and stable performance.

Fig.10 Center-lines extracted by the proposed method at different laser power

Table 4 Extraction error of the proposed method at different laser power

Then，LSL is projected onto the surface of three different objects，that is aluminum plate（metal），CD and ordinary cardboard. Fig.11 shows that the proposed method can accurately extract these center lines，which means that the obtained center lines are all in the actual center positions of these stripes. Although the error data in Table 5 fluctuate with the variety of the measured objects，the overall values are small. The main reason for this is that the proposed method increases the computational weight of the grey-value of pixels at the center when calculating the coordinates of the center points of the laser stripe，and optimizes the positions of initial center points. Therefore，the proposed method can reduce the effects of environmental parameter changes and noise disturbances in the process of extracting the stripe center.

Fig.11 Center-lines extracted by proposed method on different objects

Table 5 Extraction error of the proposed method on different detected objects

4 Conclusions

An adaptive optimization method for laser stripe center point extraction is proposed. The method helps solve some problems that usually happen in actual industrial site，such as large deviation of extracted center points of the laser stripe，obvious broken-line defect of the extracted center line and problems caused by variable width，and uneven grey-value distribution of the laser stripe and other factors.The laser image is preprocessed，and ROI is determined. Based on obtaining the quadratic weighted barycenter of the laser stripe，an adaptive method is used in this paper. Then the positions of initially extracted center points are optimized according to the obtained slope threshold of the laser stripe. The proposed method can run in real time and provide a better performance than the traditional methods. The first experimental result shows that the standard de-

viation of distance from extracted center points to the fitted line is only 0.13 pixel，improved by about 30% compared with other extraction methods，which illustrates that the accuracy of the proposed optimization method is relatively high. Additionally，the proposed method also reduces the extraction repeatability error by around 50%. Another experimental result demonstrates that this method keeps great robustness on the condition that some environment parameters may change.