I found the issue but have no idea why it's an issue. When I comment out depth->setAlphaScaling(0);, the code runs without the error. Do you know why this is the case?
My other big question: if I don't have depth->setAlphaScaling(0);, will my intrinsic matrix be incorrect? With depth->setAlphaScaling(0);, I know that black borders get removed. So since my input frame size is 640x400, I expected the outputed frame size to be smaller than 640x400. But when I enabled depth->setAlphaScaling(0) and used:
cv::Rect roi1, roi2;
cv::stereoRectify(l_m, l_d, r_m, r_d, cv::Size(640, 400), r, t, r1, r2, p1, p2, q, cv::CALIB_ZERO_DISPARITY, 0, cv::Size(), // let OpenCV compute output size
&roi1, &roi2);
std::cout << "P1\n" << p1 << "\nP2\n" << p2 << "\n";
*f = p1.at<double>(0, 0);
*cx = p1.at<double>(0, 2);
*cy = p1.at<double>(1, 2);
//When alpha=0, roi covers entire frame (of course minus the black spots)
*out_w = roi1.width;
*out_h = roi1.height;
std::cout << "roi1.width: " << roi1.width << "\n";
std::cout << "roi1.height: " << roi1.height << "\n";,
I got:
roi1.width: 640
roi1.height: 400.
