documentation:vision:camera_calibration [robotwiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Camera Calibration ======

To calibrate a camera some calibration pattern is required, traditionally this is a checkerboard which can be generated with the help of the code in the following [[https://docs.opencv.org/4.x/da/d0d/tutorial_camera_calibration_pattern.html | link]].

===== OpenCV =====

This approach is based on the opencv documentation for camera calibration in the following [[https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html | link]]. The repository in the following [[https://git.cs.lth.se/robotlab/camera_calibration | link]] implements camera calibration with as well as picture taking with realsense cameras. This repository was written by Pontus Rosqvist, Josefin Gustafsson, Marcus Nagy and Martin Lyrå.

The code for camera calibration expects images of checkerboards taken with the camera for which we need to compute the instrinsic camera parameters. The size of the checkerboard needs to be supplied since the opencv function [[https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga93efa9b0aa890de240ca32b11253dd4a | findChessboardCorners]] needs to know how many points it is looking for and the entire chessboard needs to be visible in every image. The height and width of the chessboard which needs to be specified is not the height and width of the chessboard in squares but rather the heigth and width of the inner corners (points between squares). For example a normal chessboard is 8 by 8 if you count the squares but if we count the inner corners it is 7 by 7 so we would supply 7 by 7 to the function.

Ideally the calibration algorithm is supplied many different images of the checkeboard in many different positions and angles in all areas of the image to ensure a good estimate but it is quite sensitive to noise so it is important to visually inspect the images to verify that the corners can be estimated correctly and accurately.

==== Taking Pictures With a RealSense Camera ====

Image saving with a realsense camera has been implemented in [[https://git.cs.lth.se/robotlab/camera_calibration/-/blob/main/take_pictures.py | take_pictures.py]]. At the moment the L500 and D400 models of realsense cameras are supported but it is easy to add more cameras by adding more resolutions. This script only takes one argument, the folder the images should be saved in, if the folder does not exist it is made and if it does exist previous images that might exist in that directory are overwritten. If no name for a folder is supplied the images are saved in the current working directory.

If the pipeline does not receive a frame from the camera within 5 seconds the following error will be thrown

<code>RuntimeError: Frame didn't arrive within 5000</code>

In which case one should check that the camera works and sends frames to the computer with realsense-viewer.

It is very important to check if the camera is recognized as a USB2 device or USB3 device since USB2 supports a smaller sets of resolution than USB3. If a resolution that is not possible is requested (for example if the camera is detected as a USB2 device and the expected resolution is too big) the pipeline throws the following error:

<code>Runtime error: Couldn't resolve requests</code>

In which case one should check what resolution the frames from the camera have with realsense-viewer to verify that the expected resolution is correct.

After the images have been taken duplicates should be removed as well as blurry pictures. Even if it might be possible for opencv to find the checkeboard in a blurry images this will introduce noise to the estimate of the intrinsic camera parameters which is not desirable.

==== Computing the Instrinsic Parameters ====

The intrinsic camera parameters for a specific camera can be computed from a set of images of checkerboards taken by that camera, this is implemented in [[https://git.cs.lth.se/robotlab/camera_calibration/-/blob/main/calibrate.py | calibrate.py]]. This script takes the following command line arguments:

^ Argument Name  ^ Description                                                                                                                                        ^
| -height        | Height of rectangle in checkerboard (the amount of intersection corners per row in the y-axis).                                                    |
| -width         | Width of rectangle in checkerboard (the amount of intersection corners per row in the x-axis).                                                     |
| -size          | Length of checkerboard squares in millimeters.                                                                                                     |
| -f             | name of folder where images are stored. Camera parameters are also saved in this folder. True or False.                                            |
| -pause         | Show each detected checkerboard and display for visual inspection. Press c to continue to the next picture and q to stop execution. True or False. |

The script prints the result for each image wether it was able to detect a checkerboard or not, if the wrong height and width are supplied the checkerboard might still be detected in 1 in 10 images depending on how incorrect the values are. If the values are correct, however, the checkerboard should be detected in all but the blurriest pictures as long as the whole checkerboard is in the image.

The camera parameters are saved in a text file called 'params' in the same folder as the images. A function which parses this text file is also implemented in [[https://git.cs.lth.se/robotlab/camera_calibration/-/blob/main/calibrate.py | calibrate.py]], the function is called parse_camera_parameters and takes the path to the file where the camera parameters are saved as its only argument. The parsing function returns two tuples

^ Output ^ Name    ^ Contains                          ^
| 1      | calib   | $f_x$, $f_y$, $c_x$, $c_y$        |
| 2      | distort | $k_1$, $k_2$, $p_1$, $p_2$, $k_3$ |

The first tuple, calib, contains the intrinstic camera matrix where $f_x$ and $f_y$ are the focal lengths in the $x$- and $y$-direction and ($c_x$, $c_y$) is the location of the pixel center. The distortion parameters $k_1$, $k_2$, $p_1$, $p_2$ and $k_3$ are parameters which estimate how much straight lines captured by the camera are bent. This bending effect can be removed with the help of opencv's function [[https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga69f2545a8b62a6b0fc2ee060dc30559d | undistort]].

===== SkiROS =====