To calibrate a camera some calibration pattern is required, traditionally this is a checkerboard which can be generated with the help of the code in the following [[https://docs.opencv.org/4.x/da/d0d/tutorial_camera_calibration_pattern.html | link]].
This approach is based on the opencv documentation for camera calibration in the following [[https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html | link]]. The repository in the following [[https://git.cs.lth.se/robotlab/camera_calibration | link]] implements camera calibration with as well as picture taking with realsense cameras. This repository was written by Pontus Rosqvist, Josefin Gustafsson, Marcus Nagy and Martin Lyrå.
The code for camera calibration expects images of checkerboards taken with the camera for which we need to compute the instrinsic camera parameters. The size of the checkerboard needs to be supplied since the opencv function [[https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga93efa9b0aa890de240ca32b11253dd4a | findChessboardCorners]] needs to know how many points it is looking for and the entire chessboard needs to be visible in every image. The height and width of the chessboard which needs to be specified is not the height and width of the chessboard in squares but rather the heigth and width of the inner corners (points between squares). For example a normal chessboard is 8 by 8 if you count the squares but if we count the inner corners it is 7 by 7 so we would supply 7 by 7 to the function.
Ideally the calibration algorithm is supplied many different images of the checkeboard in many different positions and angles in all areas of the image to ensure a good estimate but it is quite sensitive to noise so it is important to visually inspect the images to verify that the corners can be estimated correctly and accurately.
Image saving with a realsense camera has been implemented in [[https://git.cs.lth.se/robotlab/camera_calibration/-/blob/main/take_pictures.py | take_pictures.py]]. At the moment the L500 and D400 models of realsense cameras are supported but it is easy to add more cameras by adding more resolutions. This script only takes one argument, the folder the images should be saved in, if the folder does not exist it is made and if it does exist previous images that might exist in that directory are overwritten. If no name for a folder is supplied the images are saved in the current working directory.
If the pipeline does not receive a frame from the camera within 5 seconds the following error will be thrown
<code>RuntimeError: Frame didn't arrive within 5000</code>
In which case one should check that the camera works and sends frames to the computer with realsense-viewer.
It is very important to check if the camera is recognized as a USB2 device or USB3 device since USB2 supports a smaller sets of resolution than USB3. If a resolution that is not possible is requested (for example if the camera is detected as a USB2 device and the expected resolution is too big) the pipeline throws the following error:
<code>Runtime error: Couldn't resolve requests</code>
In which case one should check what resolution the frames from the camera have with realsense-viewer to verify that the expected resolution is correct.