[人工智能]opencv4计算单目相机内外参矩阵与畸变校正

代码见
https://learnopencv.com/camera-calibration-using-opencv
相关知识点请参考《视觉SLAM十四讲从理论到实践》，附链接如下
https://pan.baidu.com/s/154vSodYtNG_mGOxvFJDiQQ
提取码: qkw8
文末的代码在上面源码的基础上进行了部分修改，改正几个错误并添加了使用相机内参进行畸变校正的代码。我用的是双目相机所以代码中有裁剪图像的部分，此处自行修改。遇到什么问题可评论或私信我。

源码修改与解读

首先在一个文件夹中保存若干张带有棋盘格的图片，越多越好，然后将std::string path设置为图片保存的路径。
将int CHECKERBOARD修改为你的棋盘格的角点的数量，比如我的棋盘格有7行9列，角点就应该是6,8。
源码大致思路如下：

• 首先定义2个二维数组objpoints和imgpoints，他们的每个元素分别存放棋盘格的三维世界坐标和他们对应的二维像素坐标，比如有12张图片，角点以6,8为例，那尺寸就是12x48。
• 循环读取一张图片并尝试定位图片中棋盘格的角点;
• 用所有图片计算出的角点坐标计算内外参矩阵和畸变参数;
• 我在后面加上根据内参矩阵对原图进行畸变校正。

用到的几个函数分析

代码中用到的函数主要在calib3d.hpp文件夹中，代码中也都有详细的注释，我再简要翻译一下。

findChessboardCorners

找到棋盘格中的角点。
参数：
image输入 带棋盘格的图像，应为灰度图
patternSize输入 棋盘格的角点的数量
corners输出 每个角点的像素坐标
flags输入 一些设置

cornerSubPix

亚像素级上精确定位坐标。没看懂，原文和部分翻译如下：
The function iterates to find the sub-pixel accurate location of corners or radial saddle
points as described in @cite forstner1987fast, and as shown on the figure below.
(该函数迭代以查找角点或径向鞍点的亚像素精确位置。)

Sub-pixel accurate corner locator is based on the observation that every vector from the center $q$
to a point $p$ located within a neighborhood of $q$ is orthogonal to the image gradient at $p$
subject to image and measurement noise. Consider the expression:
(亚像素精确的角点定位器基于以下观测，从中心q到位于q邻域内的点p的每个向量与p处受图像和测量噪声影响的图像梯度正交。考虑表达式：)
$${?}_i={D{I}_{p_i}}^T?(q?p_i)$$
where $D{I}_{p_i}$ is an image gradient at one of the points $p_i$ in a neighborhood of $q$ . The
value of $q$ is to be found so that ${?}_i$ is minimized. A system of equations may be set up
with ${?}_i$ set to zero:
$$\sum _i(D{I}_{p_i}?{D{I}_{p_i}}^T)?q?\sum _i(D{I}_{p_i}?{D{I}_{p_i}}^T?p_i)$$
where the gradients are summed within a neighborhood (“search window”) of $q$ . Calling the first
gradient term $G$ and the second gradient term $b$ gives:
$$q=G^{?1}?b$$
The algorithm sets the center of the neighborhood window at this new center $q$ and then iterates
until the center stays within a set threshold.
参数：
image输入 图片
corners输入 角点坐标
winSize输入 搜索窗口边长的一半。比如如果设置窗口为(5,5)，那么将使用$(5?2+1)\times (5?2+1)=11\times 11$的搜索窗口
zeroZone设置死区范围以避免极点。一般设为(-1,-1)即可
criteria

drawChessboardCorners

将标出角点的数据写到一个Mat里，后面可以用imshow显示。

calibrateCamera

根据输入的图片和角点坐标计算内外参矩阵。原文中的详细说明：
The function estimates the intrinsic camera parameters and extrinsic parameters for each of the views. The algorithm is based on @cite Zhang2000 and @cite BouguetMCT . The coordinates of 3D object points and their corresponding 2D projections in each view must be specified. That may be achieved by using an object with known geometry and easily detectable feature points. Such an object is called a calibration rig or calibration pattern, and OpenCV has built-in support for a chessboard as a calibration rig (see @ref findChessboardCorners). Currently, initialization of intrinsic parameters (when @ref CALIB_USE_INTRINSIC_GUESS is not set) is only implemented for planar calibration patterns (where Z-coordinates of the object points must be all zeros). 3D calibration rigs can also be used as long as initial cameraMatrix is provided.
参数：
objectPoints输入 所有角点的三维世界坐标
imagePoints输入 所有角点的二维像素坐标
imageSize输入 图像尺寸
cameraMatrix输出 3x3的相机内参矩阵
distCoeffs输出 相机畸变参数，5个参数
rvecs输出 包含3x3的旋转矩阵的向量，向量长度等于棋盘格图片数量(假设所有准备好的图片中均有棋盘格)
tvecs输出 包含3x1的平移矩阵的向量，向量长度等于棋盘格图片数量
return返回重映射的均方根误差。
算法步骤如下：
(1) 计算初始内参;
(2) 计算初始相机姿态;
(3) 使用Levenberg-Marquardt全局优化算法最小化重映射误差。

undistort

畸变校正。
参数：
src输入 畸变图像
dst输出 校正后图像
cameraMatrix输入 相机内参矩阵
distCoeffs输入 相机畸变参数

源码

#include <opencv2/opencv.hpp>
#include <opencv2/calib3d/calib3d.hpp>
#include <iostream>

int CHECKERBOARD[2]{6,8};

int main()
{
  std::vector<std::vector<cv::Point3f>> objpoints;
  std::vector<std::vector<cv::Point2f>> imgpoints;
  std::vector<cv::Point3f> objp;
  for(int i{0}; i<CHECKERBOARD[1]; i++)
    for(int j{0}; j<CHECKERBOARD[0]; j++)
      objp.push_back(cv::Point3f(j,i,0));

  std::vector<cv::String> images;
  std::string path = "/home/pi/Desktop/myfile/cpp/camera/build/*.png";
  cv::glob(path, images);
  cv::Mat origin, frame, gray;
  std::vector<cv::Point2f> corner_pts;
  bool success;

  for(int i{0}; i<images.size(); i++) {
    origin = cv::imread(images[i]);
    frame = origin(cv::Rect(640,0,640,480));
    cv::cvtColor(frame,gray,cv::COLOR_BGR2GRAY);
    success = cv::findChessboardCorners(gray, cv::Size(CHECKERBOARD[0], CHECKERBOARD[1]), corner_pts, cv::CALIB_CB_ADAPTIVE_THRESH | cv::CALIB_CB_FAST_CHECK | cv::CALIB_CB_NORMALIZE_IMAGE);
    if(success) {
      cv::TermCriteria criteria(cv::TermCriteria::EPS | cv::TermCriteria::MAX_ITER, 30, 0.001);
      cv::cornerSubPix(gray,corner_pts,cv::Size(11,11), cv::Size(-1,-1),criteria);
      cv::drawChessboardCorners(frame, cv::Size(CHECKERBOARD[0], CHECKERBOARD[1]), corner_pts, success);
      objpoints.push_back(objp);
      imgpoints.push_back(corner_pts);
    }
    cv::imshow("Image",frame);
    cv::waitKey(0);
  }
  cv::destroyAllWindows();
  cv::Mat cameraMatrix,distCoeffs,R,T;
  cv::calibrateCamera(objpoints, imgpoints, cv::Size(gray.rows,gray.cols), cameraMatrix, distCoeffs, R, T);

  std::cout << "cameraMatrix : " << cameraMatrix << std::endl;
  std::cout << "distCoeffs : " << distCoeffs << std::endl;
  std::cout << "Rotation vector : " << R << std::endl;
  std::cout << "Translation vector : " << T << std::endl;
  std::cout << "finished." << std::endl;

  for (int i=images.size()-1; i>=0; --i){
    origin = cv::imread(images[i]);
    frame = origin(cv::Rect(0,0,640,480));
    cv::undistort(frame, origin, cameraMatrix, distCoeffs);
    cv::imshow("Image", origin);
    cv::waitKey(0);
  }

  return 0;
}

calib.hpp中常用函数目录

opencv4.5.2版本，部分功能类似的函数被忽略。
findHomography728
RQDecomp3x3760
decomposeProjectionMatrix787
matMulDeriv807
composeRT838
projectPoints879
solvePnP1078
initCameraMatrix2D1440
findChessboardCorners1493
drawFrameAxes1627
findCirclesGrid1694
calibrateCamera1821
stereoCalibrate2065
stereoRectify2187
getOptimalNewCameraMatrix2268
calibrateHandEye2419
calibrateRobotWorldHandEye2562
convertPointsToHomogeneous2576
findFundamentalMat2652
findEssentialMat2703
decomposeEssentialMat2817
recoverPose2869
computeCorrespondEpilines2959
triangulatePoints2986
correctMatches3005
filterSpeckles3020
getValidDisparityROI3025
validateDisparity3030
reprojectImageTo3D3073
sampsonDistance3095
estimateAffine3D3145
estimateTranslation3D3192
estimateAffine2D3258
estimateAffinePartial2D3310
decomposeHomographyMat3343
filterHomographyDecompByVisibleRefpoints3367
StereoMatcher3376
undistort3589
initUndistortRectifyMap3657
getDefaultNewCameraMatrix3700
undistortPoints3744
fisheye3760