Research on Visual Detection Algorithm for Groove Feature Sizes by Means of Structured Light Projection

，*，，

1.College of Mechanical and Electrical Engineering，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China；2.Nanjing Atekon Automation Technology Co.，Ltd.，Nanjing 210012，P.R.China

Abstract:The visual inspection is an economical and effective method for welding.For measuring the feature sizes of grooves，a method based on line structured light is presented.Firstly，an adaptive algorithm to extract the subpixel centerline of structured light stripes is introduced to deal with the uneven width and grayscale distributions of laser stripes，which is based on the quadratic weighted grayscale centroid.By means of region-of-interest（ROI）division and image difference，an image preprocessing algorithm is developed for filtering noise and improving image quality.Furthermore，to acquire geometrical dimensions of various grooves and groove types precisely，the subpixel feature point extraction algorithm of grooves is designed.Finally，experimental results of feature size measuring show that the absolute error of measurement is 0.031—0.176 mm，and the relative error of measurement is 0.2%—3.6%.

Key words：groove measurement；line structured light；centerline extraction；feature point extraction

0 Introduction

Welding is widely used in metal processing.The machine vision based on line structured light has the advantages of non-contact，high precision and quite high speed，and it is regarded as the most promising technology to detect the weld seams.The application of structured light in weld inspection originated in 1970s has made a series of achieve?ments in recent years［1］.Han et al.［2］developed a structural light vision sensor with a narrow band fil?ter，using the feature extraction algorithm of the fill?ing weld bead and the capping weld bead for evaluat?ing the weld quality.In order to realize real-time seam tracking of multi-layer and multi-pass weld?ing，Zeng et al.［3］introduced a method that com?bines directional and structural light image to detect weld edge precisely.Based on structured light vi?sion，Wang et al.［4］adopted ensemble learning mod?els to recognize weld seam types，which include BPAdaboost and KNN-Adaboost.

The mature commercial products of weld track?ing and quality evaluation based on laser vision ap?peared in the United States，Denmark and Germa?ny［5］.The Laser Tracking produced by ASEA Ro?botics in Denmark can accomplish the localization and tracking of different kinds of weld seams such as butt and corner joints，with the accuracy up to 0.4 mm［6］.The TH6D from Scansonic can com?plete more precise seam tracking by multiple light beams［7］.In 2012，the monitoring system of weld?ing by means of laser vision came on the scene at Southern Methodist University，which can measure the 3D contour of weld joints and evaluate welding quality in real time［8］.

Achieving geometrical dimensions of welding grooves and recognizing groove types are important contents to weld inspection，which are key to the path planning of welding robots and the selection of material and welding parameters.According to the shape，the welding grooves are mainly classified in?to five types：Square groove，lap groove，single bevel groove，V-groove and U-groove［9］.

The centerline extraction of laser stripes and the feature point detection of grooves are two impor?tant parts of groove geometrical dimension measure?ment.The traditional methods for acquiring the la?ser stripe center are geometrical center extraction and energy center extraction［10］.The former has slow speed and low precision，in which the skeleton thinning algorithm is a typical example.In the lat?ter，the gray centroid algorithm performs excellent?ly for accuracy and poorly in anti-jamming capabili?ty，and the algorithm based on Hessian matrix takes much time［11］.The algorithm for centerline extrac?tion introduced in this paper adapts to the uneven width of laser stripe excellently，and has good per?formance in accuracy，which is an improvement to the gray centroid algorithm.In addition，the applica?tion of linear convolution and image difference effec?tively abolishes the noise.

Template matching and the method based on image sharpness are two classic methods to detect the feature points of grooves，which cannot achieve corner points accurately because these methods fo?cus on the local features of laser images［12］.Corner location and groove type recognition are combined to extract groove feature points，in which the rough feature point positioning is accomplished by corner location，and the geometric relationship between corner points and fitting centerlines is used for calcu?lating sub-pixel coordinates.

A method for measuring geometrical dimen?sions of various grooves is present in this paper.In section 1，the inspection system is presented.Then the algorithm of extracting the centerline of struc?tured light stripes is introduced in section 2.For di?mension measurement and groove type identifica?tion，the feature point extraction is crucial，which is detailedly described in section 3.Experiments of evaluating the performance of the algorithms men?tioned above are introduced in section 4.Finally，conclusions are drawn in section 5.

1 Inspection System Description

The weld groove inspection platform is shown in Fig.1.It is composed of three parts：A line-struc?tured light vision sensor，an industrial computer and a single axis robot.A line laser and an industrial camera are used to set up a line-structured light vi?sion sensor.The workpiece is held on a table driven by a single axis robot.The industrial camera is a MER-132-30GC with 16 mm focal length and can capture the image with pixel number of 1 292×964.The power and central wavelength of the laser are 100 mW and 650 nm，respectively.

Fig.1 Hardware system for visual inspection

With the movement of the single axis robot，the sensor acquires 50 stripe images on different lon?gitudinal sections of grooves to reduce random er?rors and increase the measurement reliability.The detection system gets 50 sets of measurements for each groove and the average is considered as the output value of geometrical sizes.The CPU frequen?cy and RAM size of the industrial computer are 1.7 GHz and 4.0 GB，respectively.These algo?rithms introduced in this paper are written in C++language on Visual Studio 2013 and the software toolkit for image processing is OpenCV 2.4.9.

2 Image Processing Algorithm for Extracting Centerline of Struc?tured Light Stripes

To measure the feature size of grooves，it is a prerequisite to achieve the centerline of laser stripes.Traditional algorithms for extracting the laser stripe center include the skeleton thinning algorithm，the gray centroid algorithm and the algorithm based on Hessian matrix［13］.As shown in Fig.2，the width and gray distributions of laser stripes are uneven.An adaptive optimization algorithm for extracting the sub-pixel center of structured light stripes is pro?posed.According to the adaptive width quadratic weighted grayscale centroid algorithm，the initial stripe center points are obtained，after that the accu?rate center coordinates are achieved through making analysis of the slope threshold.Compared with the above three traditional algorithms，the accuracy of the proposed algorithm is improved by 28.1%，58.2% and 12.9%，respectively.

Fig.2 Image of a laser stripe

2.1 Adaptive width quadratic weighted gray?scale centroid algorithm

（1）Edge detection of laser stripes

As shown in Fig.3，image pixels are scanned column by column from left to right and top to bot?tom until the first pixelPl(xl，yl) with a gray value of 255 is searched at left side or the last pixelPr(xr，yr) to the right.The pixels at the laser stripe are expressed by

Fig.3 Image scanning

The width varies along the length of laser stripes，which can be computed by the longitudinal coordinate of upper and lower edges.The pixels in theith section are expressed by

wherefi(x，y) is the set of pixels at theith section；yup(i) andydown(i) are the longitudinal coordinates of the upper and lower edges at theith section，re?spectively；wiis the width of theith section that could be calculated by

（2）Adaptive extracting initial center points of laser stripes

For the gray centroid algorithm，noise has a serious negative impact on laser centerline extrac?tion.As shown in Fig.4，the gray values in the cen?tral region of the stripe are high and change slight?ly，so the adaptive width quadratic weighted gray?scale centroid algorithm is adopted to achieve the initial center pointwhich is at theith section.Through the upper and lower edges of laser stripes，the noise is excluded from the computa?tion，thus the performance of the algorithm will be upgraded to offer better capabilities for anti-interfer?ence.

Fig.4 Schematic diagram of adaptive width quadratic weighted grayscale centroid algorithm

They-coordinate of an initial center point，，can be calculated by

2.2 Optimized algorithm based on discrete analysis of center points

In order to improve the extraction accuracy of stripe center points and debase the influence of noise，an algorithm that optimizes the initial center points is designed based on making analysis of the slope threshold，in which the least square method is used to fit the experimental data.The optimization process of the initial center points is divided into two steps.

（1）Mean optimization

The average of the initial center points in each window is computed as the optimized center pointand the window centered on the ini?tial pointslides from left to right with the size ofa×a.The value ofcan be calculated by

whereiandmare the sequence numbers of section；they-coordinates of the initial center points at theith andmth sections，respective?ly.is they-coordinate of；andais the side length of the window.

（2）Analyzing dispersion and relocating center points

According to the border pointsPlandPrre?ferred in Section 2.1，the slope thresholdkTis used to evaluate the dispersion of initial center points.It could be calculated by

wherexrandxlare thex-coordinates ofPrandPl，andyrandylare they-coordinates ofPrandPl，re?spectively.

When the window slides along the optimized center points from left to right，the slopek1between the center pointscould be calculated by

whereis they-coordinate of，ithe se?quence number of sections，andbthe difference in thex-coordinate of two sections.

The algorithm determines whether the disper?sion of the optimized center points exceeds standard by means of analyzingkTandk1.The relocation ofis necessary，ifk1is 10 times larger thankT.They-coordinate of the final center pointcould be cal?culated by

By all the above processes，the sub-pixel cen?ter point at theith section is.

3 Algorithm of Image Preprocess?ing and Feature Point Extraction for Laser Stripe on Grooves

As shown in Fig.5，the image preprocessing is completed firstly.Then after centerline extraction，feature points are extracted.Finally the feature sizes of grooves are calculated.

Fig.5 Flow chart of laser image processing

3.1 Image preprocessing

Image interferences near the laser stripe are ac?node noise and fuzzy flare as result of specular reflec?tion and diffuse reflection on metallic surface.With the increasing of laser power，this phenomenon be?comes more obvious.In addition，small speckle noise and solitary points are produced by the system hardware，such as laser devices and industrial cam?eras，because of the changes of anti-jamming capa?bility.

（1）Image enhancement

The automatic measurement of feature sizes must own satisfactory capability because it aims at five types of grooves.As shown in Fig.6，through the feature analysis of laser stripes on different kinds of grooves，the laser stripe is split into three parts：The left，middle and right laser regions.As a re?sult，the computation load for the pretreatment pro?cess is reduced significantly.The template matching is one way to accomplish ROI division，but it needs a long time［14］.In this paper，the laser image is scanned from top to bottom，and the row with the maximum gray value is thought as the baseline.The pixels with the gray value less than 100 are searched from left to right along the baseline，and the pixels nearest to the laser stripe are the horizontal border points of ROI.The vertical borders of ROI are de?termined by the number of pixels with gray value of 255 at every row.

Fig.6 Division of laser stripes

The traditional image enhancement approaches are not good solutions to the problem of low con?trast between the object and background in laser im?ages.These approaches include histogram equaliza?tion，logarithmic transformation，gamma correc?tion，Laplace operator and Wiener filtration［15］.An image enhancement algorithm based on linear convo?lution and image difference is proposed to improve the quality of the laser stripe in noise background.

There are elongated laser stripes extending hor?izontally in the left or right region where anisotropic scattered facula and acnode noise are found principal?ly.For the above reason，laser images are enhanced in the horizontal direction.The convolution window should be longer than the stripe with the width of 30—40 pixels in the experiment platform.The win?dow size is set to 41×1.is the revolution of the convolution windowM1

There is a small horizontal laser stripe with the width of about 30 pixels in the middle region，and the size of the convolution window is set to 31×1.is the revolution of the convolution windowM2

The width of interference stripes is about 5 pix?els.As shown in Eq.（11），the convolution window isM3，whose size is set to 1×5.

The difference image between laser images convoluted byM1andM3or betweenM2andM3is the result of image enhancement.As shown in Fig.7（a），the laser stripes in red boxes are interference stripes，which are obvious after the convolution op?eration byM3as shown in Fig.7（b）.The difference image between laser images convoluted byM2andM3is shown in Fig.7（c），and the noise and interfer?ence stripes are suppressed considerably.

Fig.7 Process of image enhancement

（2）Background filtering

In order to filter image background and extract intact stripes，the morphological opening is com?bined with connection area removal in this paper.For the binary image obtained by Otsu method，the larger grain noise is eliminated by morphological op?eration.Afterwards，connection areas are searched to delete residual noise according to area-size fea?tures.Finally，stripe edges are smoothed by closing operation of mathematical morphology.

3.2 Sub?pixel feature point extraction algo?rithm of grooves

After laser stripe centerlines are gained，it is necessary to implement the type recognition and ac?curate corner extraction by the characteristics of vari?ous grooves.

As shown in Fig.8，the pointsAc，Bc，Cc，Dc，Ec，F(xiàn)care general corner points because these points exist in every kind of groove，and the special corner points are pointsGc，Hc，Mc，Ncbecause these points only appear on V-groove and U-groove.The characteristics of various grooves are summa?rized in Table 1 and the methods to find corner points are shown in Table 2.For example，in single bevel groove，pointsBcandCchave the samex-co?ordinate，but thex-coordinate of pointDcdiffers from that of pointEc.

Table 2 Methods for extracting corner points

Fig.8 Characteristics of various types of grooves

Table 1 Characteristics of various grooves

The general corner points are obtained by im?age scanning，after that the groove types can be rec?ognized initially，then the special corner points of grooves are extracted in turn.By means of the slope and coordinate difference between corner points，the groove type recognition can be realized precisely.

After the groove type identification and rough corner positioning，it is necessary to carry out the sub-pixel coordinate extraction of groove feature points.According to the algorithm proposed in Sec?tion 2，laser stripe centerlines are achieved precise?ly，then the least-square method is used for fitting straight linel.

As shown in Fig.9，the sub-pixel coordinates of feature points are calculated by the straight-line intersection method and the minimum distance method.For extracting sub-pixel feature points，the first method finds intersection points between two lines and the last method calculates the foot point of an initial corner point to line.For example，the pointBfis the intersection point between the fitting centerlinesl1andl2in V-groove，and the sub-pixel coordinate ofBfis computed by slope and intercept.In square groove，the pointBfis the foot point ofBctol1，which can be calculated according to the coor?dinate ofBc，the slope and intercept ofl1.

Fig.9 Schematic diagram of extracting sub-pixel feature points

4 Experimental Results and Analy?sis

4.1 Experimental comparisons and precision analysis of different center extraction algo?rithms

Different algorithms are adopted to verify the effectiveness of the optimized centerline extraction algorithm.After completing the preprocessing of la?ser images，the skeleton algorithm，the gray cen?troid algorithm，the algorithm based on Hessian ma?trix and the algorithm introduced in this paper are used to achieve the center points of six laser images for each algorithm.The experimental data are ana?lyzed by some evaluation indexes，such as standard deviation and repeatability.

The precision of the above algorithms is esti?mated by the standard deviation of the distance be?tween center points and the fitting line.The ability of the algorithm to resist random noise is evaluated by repeatability error.Under the same working con?dition，six images of laser stripes are captured，but the center points extracted from the first image are regarded as the reference.

The experiment results of different center ex?traction algorithms are shown in Table 3.As can be seen from Table 3，the proposed algorithm achieves the greatest operating efficiency，the maximum accu?racy，and the best repeatability.

Table 3 Accuracy of the center point extraction by different algorithms

4.2 Experimental results and analysis of groove feature point extraction algorithms

Harris［16］，Shi-Tomasi［17］and the algorithm based on sharpness［18］are applied for detecting the corner point of V-groove in the input image，which only contains the centerline of laser stripes.The pa?rameters of three algorithms are shown in Table 4.

Table 4 Parameters of different feature point extraction algorithms

The Euclid distance between the actual corner points obtained by manual marking and the detected corner points are calculated.If the distance is less than 5 pixels，the corner extraction is a success，otherwise a failure.In order to evaluate the effect of corner detec?tion in different algorithms for the laser strip center?line，the error detection rate，the miss detection rate，the position error and running time are served as in?dexes.The position error is the average distance be?tween correct corner points and manually marked cor?ner points.The results of different corner detection al?gorithms for V-groove are shown in Table 5.

Table 5 Experimental results of different corner detection algorithms

The proposed corner detection algorithm is su?perior to the other three algorithms.The error detec?tion rate and miss detection rate are zero，the corner position error is only 0.50 pixel and the running time is 31 ms.In conclusion，the algorithm has high accu?racy and good real-time performance in corner detec?tion.

4.3 Experimental results of feature size mea?suring of grooves

The feature sizes of various grooves are shown in Table 6.

Table 6 Feature sizes of various grooves to be measured

In order to evaluate the performance of the method for feature size measuring of grooves，the measurement accuracy is analyzed.The accuracy re?fers to the degree of numerical deviation between the output values and the truth values.In general，the absolute error and the relative error are used to estimate the accuracy of the system.The method of accuracy test is as follows.

（1）As shown in Fig.10，the ROMER Abso?lute Arm 7325 is applied to obtain the real feature sizes of various grooves，and its measurement accu?racy is ±0.051 mm at the range of 2.5 m.The true feature sizes are shown in Table 7.

Table 7 True values of feature sizes

Fig.10 Approach for obtaining truth values of feature sizes

（2）The sensor captures 50 laser images on dif?ferent longitudinal sections of grooves and calculates 50 data values，and the average of these data is thought as the output values.

（3）The measurement values of each image and the output values are used to calculate the abso?lute error and relative error respectively.

As shown in Fig.11，the vertical and horizontal coordinates are the absolute error of each image and the sequence number of images，respectively.The absolute error of the detection system is 0.031—0.176 mm，and the relative error is 0.2%—3.6%，indicating the effectiveness of the proposed algo?rithm.The above results are shown in Fig.11 and Table 8.

Fig.11 Scatter plots of the system absolute error on differ?ent grooves

Table 8 The relative error of feature sizes %

The main reasons for the above error perfor?mance are as follows：

（1）The image quality of various grooves is dif?ferent，and the line structured light stripe some?times cannot accurately display the complete mor?phology of grooves.It is easy for laser stripes on the square groove，the lap groove and the single bevel groove to be covered，so the output values differ from the truth values to a greater degree.

（2）Groove samples cannot be processed at high precision.That leads to the difference between section geometry sizes and it has bad impact on the consistency of measurement results.

（3）The stability of hardware affects the mea?surement accuracy of geometrical dimensions，such as the change in laser power.

（4）This paper adopts the image difference and feature point extraction algorithms，which may af?fect the measurement accuracy of geometrical di?mension.The anti-interference performance of these algorithms is not very good and some implementa?tion steps need to be further improved.

5 Conclusions

A method based on machine vision for detect?ing groove feature sizes is proposed.The algorithms of sub-pixel centerline extraction，image preprocess?ing，corner extraction are introduced in detail.In particular，some conclusions are reached as follows：

（1）The centerline extraction algorithm includes the adaptive width quadratic weighted grayscale cen?troid algorithm and optimized relocation algorithm.The effect of centerline extraction is wonderful.

（2）The image preprocessing combines image enhancement and background filtering，and the im?age quality of laser stripes is greatly improved.

（3）The corner extraction algorithm fuses type recognition and corner extraction，so the sub-pixel coordinates of corners can be achieved precisely.

（4）For the system used to measure groove fea?ture sizes，the absolute error is 0.031—0.176 mm，and the relative error is 0.2%—3.6%.It shows that the algorithms of the system are quite effective and practicable.

According to the high accuracy of the detection system，the algorithms for measuring the groove feature size in this paper are proved a success.