A Flexible Calibration Method of Laser Light-Sectioning System for Online 3D Measurement

，，，

1.School of Mechanical，Electrical and Information Engineering，Putian University，Putian 351100，P.R.China；2.School of Science，Nanjing University of Science and Technology，Nanjing 210094，P.R.China；3.College of Aerospace，Nanjing University of Aeronautics and Astronautics，Nanjing 210016，P.R.China；4.School of Civil Engineering，Southeast University，Nanjing 210096，P.R.China

（Received 20 May 2021；revised 2 August 2021；accepted 29 October 2021）

Abstract:A flexible calibration method based on a front-coated flat mirror is proposed for a laser light-sectioning three-dimensional（3D）measurement system.Since the calibration target and its mirror image are spatially separated and can be recorded in an image by a camera，the proposed method requires only a single composite image that contains a non-planar checkerboard pattern，a laser strip projected on the target and their mirror images to complete the calibration of the camera and the laser plane in one step.Levenberg-Marquardt（LM）algorithm is used to optimize the system parameters，and the measurement accuracy and speed are improved to enable online 3D inspection.Static and dynamic online 3D measurements are carried out on a cup and a triple stepped shaft，respectively，to validate the proposed method.The shaft has two steps with the depth of（0.5±0.01）mm and（2±0.01）mm to be measured online when the shaft is rotated and translated at the same time.The measurement results can be output at a frequency of 7 to 11 readings per second with standard deviations of 0.040 mm and 0.051 mm.The experimental results verify the effectiveness and flexibility of the proposed method.

Key words：laser light-sectioning；system calibration；front-coated flat mirror；online three-dimentional（3D）measurement；Levenberg-Marquardt（LM）algorithm

0 Introduction

Structured light projection profilometry is widely applied to three-dimensional（3D）measurements in industry，specifically in online quality inspection［1-2］.According to the projected patterns，it can be divided into four categories［3］：Point-structured，line-structured，grating-structured，and coded-structured patterns.Among these methods，the linestructured and grating-structured projections are the two most commonly used methods due to their relative simplicity and ease of implementation［4-5］.The grating-structured method（fringe projection profilometry）usually employs a phase-shifting algorithm to reconstruct the 3D shape，so it is difficult to apply it to dynamic measurements［6-8］.In contrast，the light-structured（light-sectioning）［9-11］method is more attractive because 3D measurements can be achieved directly from a single image.Furthermore，due to the advantages of adjustable laser power，strong anti-interference，and simple evaluation equipment，the laser light-sectioning method is more suitable for industrial applications.

There are two key technologies in the light-sectioning 3D measurements：Laser strip center extraction and system calibration.Extracting the laser strip center is an image processing technique and can be implemented by many kinds of sub-pixel location algorithms［12-14］.System calibration，including the camera parameters and the laser plane equation，plays an important role in the translation of image coordinates to world coordinates.

To a large extent，the calibration process of the system strongly determines the 3D measurement accuracy.According to different calibration targets，it can be divided into three-，two-，one-and zero-dimensional（self-calibration）target methods for camera calibration.The 3D target method takes a 3D object of high-precision manufacturing as a calibration object，and two-dimensional method usually takes a checkerboard as a calibration target［15］.Onedimensional target can adopt circle dots［16］and other methods to obtain system parameters.All the above calibration methods need high-precision calibration objects.To increase flexibility，Zhang［17］proposed a calibration method using no less than three images of a planar target in free-form attitudes.For the laser plane calculation，Huynh et al.［18］adopted crossratio invariance to generate more control points.Furthermore，Zhou and Zhang［19］combined the work of Zhang and Huynh and developed a complete system calibration method.Yu et al.［20］proposed a planeconstraint calibration method using a one-mirror galvanometer element.These are all the continuation work of Zhang’s calibration method，and the use of the cross-ratio makes the algorithm rather complicated.

In order to simplify the calibration process or to save cameras in large-scale measurements，flat mirrors have been introduced in recent years for the calibration of 3D measurement systems［21-27］.Usually，a flat chessboard and a flat mirror are adopted to calibrate the camera’s intrinsic and/or extrinsic parameters［21，23］.Xu et al.［23］calibrated a multi-camera system using a flat mirror.The system has four cameras forming two stereo vision pairs，which have no overlapping fields of view（FOV）.A plane mirror was used to overlap the FOV of the two camera pairs and the camera group parameters were calibrated with the aid of the plane chessboard.Sometimes two mirrors were used for large-scale FOV or multiview measurement［24-26］.Genovese et al.［24］calibrated a stereo vision system based on a active phase target and two planar mirrors，and achieved large scale 3D measurements using digital image correlation（DIC）method.Yin et al.［25］used two plane mirrors with an angle of about 120° to obtain images of the measured object simultaneously from three different directions（i.e.，by a real camera and two virtual cameras），and the panoramic 3D shape of the object was achieved.Zhu et al.［26］used two mutually perpendicular flat mirrors，forming a 45° dual-reflector with respect to the object to be measured，to eliminate the effect of rigid body displacement on strain measurements.The plane mirror in Ref.［27］is somewhat unusual in that its feature points（a chessboard grid）were lithographically printed to calibrate the external parameters of the camera.By multiple moves of the white paper on which the laser strip was projected（as the strip is nearly straight），the multiple composite images including the real and virtual images of the strip were acquired，and the laser plane equations were calibrated using the stereo vision method based on the obtained multiple composite images.

This paper presents a calibration method of a laser light-sectioning system based on the assistance of a front-coated flat mirror.Different from previous methods，this method adopts a curved chessboard pattern and only needs a single composite image including the pattern，the laser strip and their mirror images to calibrate the parameters of the measurement system in one step.In addition，on the basis of previous work，we improve the accuracy and speed of laser stripe centerline inspection，and realize online 3D measurement.Once the mirror is placed near the calibration target with a laser strip projected onto it，the camera can capture both the target，the laser strip and their mirror images in a single image.With analysis of the pin-hole imaging model，it can be found that there are superfluous equations to complete the system calibration.Through image processing technology，all parameters of the laser lightsectioning system can be obtained using the captured composite image.Compared with the existing calibration methods，the proposed technique needs only one image，so it seems simple and flexible.Levenberg-Marquardt（LM）［28］iteration algorithm is used to optimize the system parameters.Mean while，to accelerate the speed of measurement，F(xiàn)OV is reduced to focus on the laser strip，avoiding unnecessary search processes.Online static 3D measurement has been carried out on a cup with a flower on it，while online dynamic measurement is carried out on a triple stepped shaft which has two steps with the depth of（0.5±0.01）mm and（2±0.01）mm.The shaft is driven to rotate and translate at the same time，and the standard deviation of the depth measurements are 0.040 mm and 0.051 mm.Since the least squares algorithm is used for online measurement，the time-consuming of each evaluation of the steps is somewhat unstable，and the measurement results are output at a frequency of 7 to 11 readings per second.The experimental results show that the proposed calibration method is flexible，effective and robust.

1 Principle

1.1 Laser light-sectioning system

A laser light-sectioning system is shown in Fig.1.A CCD camera connected to a computer captures the target image.A laser beam is expanded by a cylinder lens while another aspherical lens straightens it into a vertical light plane，which is projected onto the target surface and forms a light strip.A front-coated plane mirror is placed near the target at an appropriate angle，so that the target and its image are imaged simultaneously on the CCD camera.At the same time，the spatial separation of the target and its mirror image is required in the captured composite image.By projecting the laser plane onto an arbitrary non-planar object with feature points on it，the internal and external parameters of the camera and the laser plane equation can then be calibrated based on a pin-hole imaging model.

Fig.1 Schematic diagram of calibration of a laser light-sectioning system

1.2 Calibration of system

In Fig.1，xw，yw，zware the world coordinates andxc，yc，zcthe camera coordinates.For most cameras，a pin-hole imaging mode can accurately describe the relationship between the image coordinates and the world coordinates of a point on the surface of the target as follows

wheresis the scale factor，andm=[u v1]Ta column matrix of image coordinates of a point in pixel.RandTare the rotation matrix and the translation vector from the world coordinate system to the camera coordinate system，respectively.BothRandTare the external parameters of the camera.M=[xwywzw1]Tis a column matrix of the world coordinates of a point on the target.Ais the camera intrinsic matrix，which can be expressed as

wherefx=f/dx，fy=f/dy.fis the focal length anddx，dyare the physical dimensions of a pixel inxandydirections，respectively；αis the first-order distortion coefficient of the camera；andu0，v0are the image coordinates of the principal point in pixel.Once the parameters of the camera（internal and external）and the laser plane equation are calibrated，the 3D information of the measured object can be obtained by Eq.（1）.

The intrinsic matrix can be obtained in advance by using Zhang’s method［17］.The origin of the world coordinates is set on the mirror，and letxwaxis be perpendicular to the mirror.A non-planar checkerboard pattern is used as a calibration target，and the world coordinates of a pointPon it are denoted asMP=[xwywzw1]T，while the world coordinates of its mirror imageP′are denoted asM′P=[x′wy′wz′w1]T.Therefore，we havexw=－x′w，yw=y′w，zw=z′w.The relationship between the image physical coordinates and world coordinates can be written as

wherex=(u－u0)/fx，y=(v－v0)/fy，x′=(u′－u0)/fx，y′=(v′－v0)/fy.Multiplying the left side of Eq.（3）by the inverse ofRgives

Here，assume

From the two equations in Eq.（4），the following two equations are established.

Eliminating the extraneous variabless1ands2in Eqs.（6a，6b）yields the following equation

wheren=[nxnynz]Tis a rotation vector corresponding to the rotation matrixR，which has nine parameters but only three variables are independent.The relationship betweennandRcan be obtained by Rodrigues transformation.Initially，nis set to ben=[0 0 0]T.By minimizing the above function with respect to the rotation vector iteratively using LM algorithm，the rotation matrixRcan be determined finally.Eq.（3b）can be transformed to the following equation

Initially，due to the component inz-axis is the largest，tis normalized and set to bet=[0 0 1]T.Based on the true distancel1of two points and the reconstructed distancel2，the translation vector is obtained byT=l1/l2×t，and all parameters in Eq.（9）are obtained.

The laser plane depends on the 3D points on the laser strip projected onto the calibration target.The intrinsic parameters and distortion parameters of the camera may be calibrated beforehand.According to Eq.（3），the corresponding mirror image points of the laser strip need to be evaluated.Since a 3D point on the calibration target is symmetrical to its mirror image point about the plat mirror，the connection lines of these couple points are parallel to each other.After perspective imaging，all the connection lines intersect at a point called a vanishing point［21，27］.Therefore，the corresponding image points can be determined based on the vanishing point.Interpolation algorithms can be used to determine the sub-pixel location.

The general form of a spatial plane equation can be written asaxw＋byw＋czw＋1=0.Suppose that the world coordinates of an arbitrary point on the laser strip is(xwi，ywi，zwi)，i=1，2，3，…，n，as shown in Eq.（10）.Oncensets of 3D points on the laser strip are obtained，the parameters in the laser plane equation can be determined using the least squares method.To optimize the least square solution，more points on the laser strip can be used.

2 Measurement Results and Discussion

2.1 System calibration

Fig.2 shows the experimental setup of the laser light-sectioning system.The type of the camera is Ueye CP-3370 with a resolution of 2 048 pixel×2 048 pixel.The pixel pitch of the camera is 5.5μm，and a KOWA LM35SC lens is configured with a focal length of 35 mm.The laser model is MW-BL-450/50 MW equipped with a cylinder and an aspherical lens to convert the laser beam into a vertical laser plane.The power is adjustable from 0 to 50 mW，and the wavelength is 450 nm（blue light）.A leadscrew sliding table driven by a programmed CM40L controller is used for subsequent online measurement.The planar mirror is front coated to prevent the secondary reflection during imaging.

Fig.2 Setup of a laser light-sectioning system

Theoretically，any object with markers on it can be used to calibrate the light-sectioning system.In this paper，a 12×9 non-planar chessboard pattern is adopted as a calibration target，of which the size of a chess grid is 6.5 mm×6.5 mm，and is utilized to estimate the coefficient of the translation vector.The chessboard grid is designed and printed by a computer，then curled randomly to obtain a non-planar chessboard pattern.

Fig.3（a）shows the non-planar chessboard pattern，the laser strip（free-form line），and their mirror images acquired by the CCD camera.The left part is real image while the right is the virtual one.The laser strip and the chessboard grid should not intersect in the composite image to avoid difficulties in extracting the centerline of the laser strip.Fig.3（b）shows the inner corner points and their mirror image points marked with green dots，which can be used as image features.They are symmetric to the mirror plane，i.e.，point 1 corresponds to point 1′.The connecting lines between the point pairs are parallel to each other and can be used to determine the vanishing point.

Fig.3 Calibration target and its mirror image

The calibration results of CCD camera are shown in Table 1.The blue free-form line in Fig.3（b）is the laser strip used for calibrating the laser plane.A gradient threshold approach is used to detect the fringe edge，and a gravity center method is adopted for its centerline determination.Sub-pixel position of centerline can be located using an interpolation algorithm.Based on the vanishing point determined，the equation of the laser plane may be calibrated by Eqs.（3，10）.The calibration results of the laser plane are shown in Table 2.

Table 1 Calibration results of camera

Table 2 Calibration results of laser plane

2.2 Experimental results and discussion

To validate the proposed calibration method，the 3D shape of a cup with a flower on its surface is measured.The cup is placed on a lead screw sliding table which is driven by a programmed CM40L controller to translate at a speed of 1.8 mm/s.We keep the laser stationary，and let the sliding table translate in the direction perpendicular to the laser plane so that the laser can scan the cup.The 3D point clouds of each laser strip modulated by the cup shape are extracted online，and then the cup shape is reconstructed.The frame rate of the CCD camera is set to be 10 f/s.Fig.4 shows a self-assembled leadscrew sliding table with a programmed CM40L controller.The translation speed is adjustable from 0.01 mm/s to 8 mm/s.A rotator is also installed on the sliding table to allow the measured object to rotate while translating for subsequent online dynamic measurements.

Fig.5 shows the target measured and the corresponding modulated laser stripe.Fig.6 shows the measurement results.The translation speed of the cup directly affects the spatial resolution of the reconstructed 3D shape.The slower the translation speed is，the higher the spatial resolution of the measurement results is.Of course，a faster extracting speed of the 3D point cloud of the laser strips is required accordingly.This is a bit of challenging for the software part of the measurement system.The measurement results of the cup verify that the proposed calibration method is effective for 3D evaluation.

Fig.4 Leadscrew sliding table with a programmed CM40L controller

Fig.5 Target measured online using the proposed laser light-sectioning system

Fig.6 Measurement results

To verify the stability and online evaluation speed of the system，a dynamic 3D measurement is carried out on a triple stepped shaft.The shaft is assembled on the sliding table，and is driven to rotate and translate simultaneously.Fig.7（a）shows a triple stepped shaft with two steps at depths of（0.5±0.01）mm and（2±0.01）mm.Fig.7（b）shows the corresponding interrupted laser strip projected onto the shaft，from which the 0.5 mm-step is faintly visible.Since the laser power used in the experiment is relatively low，the lights need to be switched off when acquiring the images in order to obtain a clear laser stripe.

Fig.7 Target for online dynamic 3D measurement

The shaft rotates at an angular velocity of 20 r/min，while translating at a velocity of 2 mm/s.The sliding table with the triple stepped shaft mounted is placed on a horizontal laboratory table.The laser plane is adjusted to be horizontal and coplanar with the axis of the rotary shaft，and the translation direction of the sliding table is perpendicular to the optical axis of the laser as far as possible.According to the above geometric arrangement，the spatial distance between the parallel laser strips can be obtained by calculating the world coordinates of the laser strip centerline.According to the spatial geometry，the distance is the step depth.Moreover，to accelerate the speed of measurement，F(xiàn)OV is reduced to focus on the laser strip when its world coordinates are extracted.

Fig.8 shows a 3D plot of the laser strip reconstructed.Fig.9 shows the distribution of the online measured depth over time for the 2 mm-step with a standard deviation of 0.051 mm，while for the 0.5 mm-step，the standard deviation is 0.040 mm.The fluctuation of the measurement results may relate partially to the micro-vibration caused by the rotation and translation.Fig.10 is a screenshot showing the output of the measurement results during the online dynamic measurement.Three are three column data in the screenshot.The left column is the online output of depth of the 0.5 mm-step，the middle column is that of the 2 mm-step and the right column shows the frequency of outputting the measurement results.It can be seen from Fig.10 that the frequency varies from 7 to 11，due to the use of least square method that results in unstable time-consuming of each evaluation of the step depth.

Fig.8 3D plot of the laser strip reconstructed

Fig.9 Online dynamic measurement results of the 2 mmstep over time

Fig.10 Screenshot of online dynamic measurement results of the triple stepped shaft

3 Conclusions

The system calibration is always an important content in the field of optical 3D measurement.In this paper，a flexible calibration method of the camera and laser plane in a light-sectioning system based on front-coated planar mirror-assisted imaging is proposed.Since the calibration target and its mirror image are spatially separated and can be captured in a singe image，the proposed method requires only a composite image containing a non-planar checkerboard pattern，the laser strip projected on the pattern，and their mirror images to complete the calibration of the camera and laser plane in one step.Compared with existing calibration approaches，the proposed method seems relatively simple and flexible.To validate the proposed calibration method，the profile of a cup with a flower on it is measured online.In addition，dynamic 3D measurement is carried out on a triple stepped shaft which has two steps with the depth of（0.5±0.01）mm and（2±0.01）mm.The shaft is scanned by a stationary laser plane when it rotates at an angular velocity of 20 r/min and simultaneously translates at a velocity of 2 mm/s.The online measurement results show that the standard deviation of the measurements of the two steps are 0.040 mm and 0.051 mm，and the evaluation results can be outputted at a frequency of 7 to 11 readings per second.The experimental results show that the proposed calibration method is not only flexible but also effective and robust.

In short，the proposed technique can be used to calibrate the laser light-sectioning system for industrial online 3D inspection quickly and effectively.We believe that with the development of calibration technology，high-speed and high-precision 3D imaging will have a great development space.