1 对于单个相机拍摄的图像序列
很简单,将自己的图片放入一个文件夹,命令输入指向这个文件夹就好了:
1.1 Sequential & Incremental SfM pipeline
$ cd openMVG_Build/software/SfM/
$ python SfM_SequentialPipeline.py [full path image directory] [resulting directory]
$ python SfM_SequentialPipeline.py ~/home/user/data/ImageDataset_SceauxCastle/images ~/home/user/data/ImageDataset_SceauxCastle/Castle_Incremental_Reconstruction
1.2 Global SfM pipeline
$ cd openMVG_Build/software/SfM/
$ python SfM_GlobalPipeline.py [full path image directory] [resulting directory]
$ python SfM_GlobalPipeline.py ~/home/user/data/ImageDataset_SceauxCastle/images ~/home/user/data/ImageDataset_SceauxCastle/Castle_Global_Reconstruction
注意:一般拍摄图像中会有相机的参数信息,OpenMVG将尝试检索图像的像素焦距。如果没有,在 Scene Initialization stage我们可以使用-f X * 1.2 提供一个近似焦距,其中X = Max(image <Width, Height>)
1.3 DIY
在sfm过程中可能有一些额外的参数需要设置,可以分步骤,下面是自己用的.bash文件:
cd reconstruction
mkdir reconstruction_work
cd reconstruction_work
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfMInit_ImageListing.exe -i ..\images\ -d D:\Reconstruction\openmvg_v20r\share\openMVG\sensor_width_camera_database.txt -o .\matches
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeFeatures.exe -i .\matches\sfm_data.json -o .\matches
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_PairGenerator.exe -i .\matches\sfm_data.json -o .\matches\pairs.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeMatches.exe -i .\matches\sfm_data.json -p .\matches\pairs.bin -o .\matches\matches.putative.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_GeometricFilter.exe -i .\matches\sfm_data.json -m .\matches\matches.putative.bin -g f -o .\matches\matches.f.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfM.exe -s INCREMENTAL -i .\matches\sfm_data.json -M .\matches\matches.f.bin -o .\output
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeSfM_DataColor.exe -i .\output\sfm_data.bin -o .\output\sfm_data_colorized.ply
mkdir .\mvs
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_openMVG2openMVS.exe -i .\output\sfm_data.bin -d .\mvs\undistortedImages -o .\mvs\scene.mvs
本人使用的是某车载摄像头的数据,图片上没有内参信息,运行上指令会在重建部分报错:
ERROR: [sequential_SfM.cpp:110] Unable to choose an initial pair, since there is no defined intrinsic data.
INFO: [sequential_SfM.cpp:173] Cannot find a valid initial pair - stop reconstruction.
因此需要在第一步openMVG_main_SfMInit_ImageListing生成sfm_data.json文件前手动添加的额外参数:
- [-f|–focal] (value in pixels)
- [-k|–intrinsics] Kmatrix: “f;0;ppx;0;f;ppy;0;0;1”
- [-c|–camera_model] Camera model type:
- 1: Pinhole
- 2: Pinhole radial 1
- 3: Pinhole radial 3 (default)
- [-g|–group_camera_model]
- 0-> each view have it’s own camera intrinsic parameters
- 1-> (default) view can share some camera intrinsic parameters
查看图像的参数: 设置上述内参,因为是同一个相机的图像设置g为1注意内参用分号隔开:
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfMInit_ImageListing.exe -i ..\images\ -d D:\Reconstruction\openmvg_v20r\share\openMVG\sensor_width_camera_database.txt -o .\matches -k "3656.311177; 0.0; 1225.812961; 0.0; 3671.211662; 993.655164; 0.0; 0.0; 1.0" -g 1
格式输入错误运行bash参数会报错:
ERROR: [main_SfMInit_ImageListing.cpp:45]
Missing ';' character
ERROR: [main_SfMInit_ImageListing.cpp:238] Invalid K matrix input
正确格式运行应该会完成重建,数据集有问题可能无法重建:
ERROR: [sequential_SfM.cpp:534] /!\ Robust estimation failed to compute E for this pair: {4,5}
这里应该是车载数据集的视差太小了,连续帧前后估计有困难。自制数据集应尽量保证在重复区域足够多的前提下扩大视角。
openMVG的sfm pipeline成功运行后,输出三个子文件夹: 分别包含特征匹配部分、重建部分和mvs格式转换的内容 output中文件sfm_data.bin就是重建的数据
1.4 openMVS
在进行OpenMVS步骤之前还需要将上一步生成的sfm_data.bin转化成mvs格式,该步在1.3bash最后一行已经执行,这一步会产生两个输出
- 一个就是scene.mvs这个文件
- 另一个就是一个undistorted_images文件夹,里面是经过畸变校正的图像
下面使用MVS进行网格化和纹理贴图: 为了方便,直接把mvs文件夹复制到MVS的exe所在文件夹下 可以先查看mvs文件中的稀疏点云:
Viewer.exe mvs\scene.mvs
生成稠密点云:
DensifyPointCloud.exe mvs\scene.mvs
查看稠密点云:
Viewer.exe mvs\scene_dense.mvs
建立粗网格:
ReconstructMesh.exe mvs\scene_dense.mvs
查看粗网格:
Viewer.exe mvs\scene_dense_mesh.mvs
细化网格(可选,比较耗时:
RefineMesh.exe mvs\scene_dense_mesh.mvs
查看细化网格:
Viewer.exe mvs\scene_dense_mesh_refine.mvs
加纹理:
TextureMesh.exe mvs\scene_dense_mesh_refine.mvs
查看结果:
Viewer.exe mvs\scene_dense_mesh_refine_texture.mvs
1.5 汇总:
完整的流程,设置好输入输出就可以了:
#!/usr/bin/python3
"""
This script is for an easy use of OpenMVG and OpenMVS
usage: MvgMvs_Pipeline.py [-h] [--steps STEPS [STEPS ...]] [--preset PRESET]
[--0 0 [0 ...]] [--1 1 [1 ...]] [--2 2 [2 ...]]
[--3 3 [3 ...]] [--4 4 [4 ...]] [--5 5 [5 ...]]
[--6 6 [6 ...]] [--7 7 [7 ...]] [--8 8 [8 ...]]
[--9 9 [9 ...]] [--10 10 [10 ...]] [--11 11 [11 ...]]
[--12 12 [12 ...]] [--13 13 [13 ...]]
[--14 14 [14 ...]] [--15 15 [15 ...]]
[--16 16 [16 ...]] [--17 17 [17 ...]]
input_dir output_dir
Photogrammetry reconstruction with these steps:
0. Intrinsics analysis openMVG_main_SfMInit_ImageListing
1. Compute features openMVG_main_ComputeFeatures
2. Compute pairs openMVG_main_PairGenerator
3. Compute matches openMVG_main_ComputeMatches
4. Filter matches openMVG_main_GeometricFilter
5. Incremental reconstruction openMVG_main_IncrementalSfM
6. Global reconstruction openMVG_main_GlobalSfM
7. Colorize Structure openMVG_main_ComputeSfM_DataColor
8. Structure from Known Poses openMVG_main_ComputeStructureFromKnownPoses
9. Colorized robust triangulation openMVG_main_ComputeSfM_DataColor
10. Control Points Registration ui_openMVG_control_points_registration
11. Export to openMVS openMVG_main_openMVG2openMVS
12. Densify point-cloud DensifyPointCloud
13. Reconstruct the mesh ReconstructMesh
14. Refine the mesh RefineMesh
15. Texture the mesh TextureMesh
16. Estimate disparity-maps DensifyPointCloud
17. Fuse disparity-maps DensifyPointCloud
positional arguments:
input_dir the directory which contains the pictures set.
output_dir the directory which will contain the resulting files.
optional arguments:
-h, --help show this help message and exit
--steps STEPS [STEPS ...] steps to process
--preset PRESET steps list preset in
SEQUENTIAL = [0, 1, 2, 3, 4, 5, 11, 12, 13, 14, 15]
GLOBAL = [0, 1, 2, 3, 4, 6, 11, 12, 13, 14, 15]
MVG_SEQ = [0, 1, 2, 3, 4, 5, 7, 8, 9]
MVG_GLOBAL = [0, 1, 2, 3, 4, 6, 7, 8, 9]
MVS = [12, 13, 14, 15]
MVS_SGM = [16, 17]
default : SEQUENTIAL
Passthrough:
Option to be passed to command lines (remove - in front of option names)
e.g. --1 p ULTRA to use the ULTRA preset in openMVG_main_ComputeFeatures
For example, running the script
[MvgMvsPipeline.py input_dir output_dir --steps 0 1 2 3 4 5 11 12 13 15 --1 p HIGH n 8 --3 n HNSWL2]
[--steps 0 1 2 3 4 5 11 12 13 15] runs only the desired steps
[--1 p HIGH n 8] where --1 refer to openMVG_main_ComputeFeatures,
p refers to describerPreset option and set to HIGH, and n refers
to numThreads and set to 8. The second step (Compute matches),
[--3 n HNSWL2] where --3 refer to openMVG_main_ComputeMatches,
n refers to nearest_matching_method option and set to HNSWL2
"""
import os
import subprocess
import sys
import argparse
DEBUG = False
if sys.platform.startswith('win'):
PATH_DELIM = ';'
FOLDER_DELIM = '\\'
else:
PATH_DELIM = ':'
FOLDER_DELIM = '/'
os.environ['PATH'] += PATH_DELIM + os.path.dirname(os.path.abspath(__file__))
os.environ['PATH'] += PATH_DELIM + os.getcwd()
def whereis(afile):
"""
return directory in which afile is, None if not found. Look in PATH
"""
if sys.platform.startswith('win'):
cmd = "where"
else:
cmd = "which"
try:
ret = subprocess.run([cmd, afile], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, check=True)
return os.path.split(ret.stdout.decode())[0]
except subprocess.CalledProcessError:
return None
def find(afile):
"""
As whereis look only for executable on linux, this find look for all file type
"""
for d in os.environ['PATH'].split(PATH_DELIM):
if os.path.isfile(os.path.join(d, afile)):
return d
return None
OPENMVG_BIN = whereis("openMVG_main_SfMInit_ImageListing")
OPENMVS_BIN = whereis("ReconstructMesh")
CAMERA_SENSOR_DB_FILE = "sensor_width_camera_database.txt"
CAMERA_SENSOR_DB_DIRECTORY = find(CAMERA_SENSOR_DB_FILE)
if not OPENMVG_BIN:
OPENMVG_BIN = input("openMVG binary folder?\n")
if not OPENMVS_BIN:
OPENMVS_BIN = input("openMVS binary folder?\n")
if not CAMERA_SENSOR_DB_DIRECTORY:
CAMERA_SENSOR_DB_DIRECTORY = input("openMVG camera database (%s) folder?\n" % CAMERA_SENSOR_DB_FILE)
PRESET = {'SEQUENTIAL': [0, 1, 2, 3, 4, 5, 11, 12, 13, 14, 15],
'GLOBAL': [0, 1, 2, 3, 4, 6, 11, 12, 13, 14, 15],
'MVG_SEQ': [0, 1, 2, 3, 4, 5, 7, 8, 9, 11],
'MVG_GLOBAL': [0, 1, 2, 3, 4, 6, 7, 8, 9, 11],
'MVS': [12, 13, 14, 15],
'MVS_SGM': [16, 17]}
PRESET_DEFAULT = 'SEQUENTIAL'
BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8)
NO_EFFECT, BOLD, UNDERLINE, BLINK, INVERSE, HIDDEN = (0, 1, 4, 5, 7, 8)
def has_colours(stream):
'''
Return stream colours capability
'''
if not hasattr(stream, "isatty"):
return False
if not stream.isatty():
return False
try:
import curses
curses.setupterm()
return curses.tigetnum("colors") > 2
except Exception:
return False
HAS_COLOURS = has_colours(sys.stdout)
def printout(text, colour=WHITE, background=BLACK, effect=NO_EFFECT):
"""
print() with colour
"""
if HAS_COLOURS:
seq = "\x1b[%d;%d;%dm" % (effect, 30+colour, 40+background) + text + "\x1b[0m"
sys.stdout.write(seq+'\r\n')
else:
sys.stdout.write(text+'\r\n')
class ConfContainer:
"""
Container for all the config variables
"""
def __init__(self):
pass
class AStep:
""" Represents a process step to be run """
def __init__(self, info, cmd, opt):
self.info = info
self.cmd = cmd
self.opt = opt
class StepsStore:
""" List of steps with facilities to configure them """
def __init__(self):
self.steps_data = [
["Intrinsics analysis",
os.path.join(OPENMVG_BIN, "openMVG_main_SfMInit_ImageListing"),
["-i", "%input_dir%", "-o", "%matches_dir%", "-d", "%camera_file_params%"]],
["Compute features",
os.path.join(OPENMVG_BIN, "openMVG_main_ComputeFeatures"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-o", "%matches_dir%", "-m", "SIFT"]],
["Compute pairs",
os.path.join(OPENMVG_BIN, "openMVG_main_PairGenerator"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-o", "%matches_dir%"+FOLDER_DELIM+"pairs.bin"]],
["Compute matches",
os.path.join(OPENMVG_BIN, "openMVG_main_ComputeMatches"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-p", "%matches_dir%"+FOLDER_DELIM+"pairs.bin", "-o", "%matches_dir%"+FOLDER_DELIM+"matches.putative.bin", "-n", "AUTO"]],
["Filter matches",
os.path.join(OPENMVG_BIN, "openMVG_main_GeometricFilter"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%"+FOLDER_DELIM+"matches.putative.bin", "-o", "%matches_dir%"+FOLDER_DELIM+"matches.f.bin"]],
["Incremental reconstruction",
os.path.join(OPENMVG_BIN, "openMVG_main_SfM"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%", "-o", "%reconstruction_dir%", "-s", "INCREMENTAL"]],
["Global reconstruction",
os.path.join(OPENMVG_BIN, "openMVG_main_SfM"),
["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%", "-o", "%reconstruction_dir%", "-s", "GLOBAL", "-M", "%matches_dir%"+FOLDER_DELIM+"matches.e.bin"]],
["Colorize Structure",
os.path.join(OPENMVG_BIN, "openMVG_main_ComputeSfM_DataColor"),
["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"colorized.ply"]],
["Structure from Known Poses",
os.path.join(OPENMVG_BIN, "openMVG_main_ComputeStructureFromKnownPoses"),
["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-m", "%matches_dir%", "-f", "%matches_dir%"+FOLDER_DELIM+"matches.f.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"robust.bin"]],
["Colorized robust triangulation",
os.path.join(OPENMVG_BIN, "openMVG_main_ComputeSfM_DataColor"),
["-i", "%reconstruction_dir%"+FOLDER_DELIM+"robust.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"robust_colorized.ply"]],
["Control Points Registration",
os.path.join(OPENMVG_BIN, "ui_openMVG_control_points_registration"),
["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin"]],
["Export to openMVS",
os.path.join(OPENMVG_BIN, "openMVG_main_openMVG2openMVS"),
["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-o", "%mvs_dir%"+FOLDER_DELIM+"scene.mvs", "-d", "%mvs_dir%"+FOLDER_DELIM+"images"]],
["Densify point cloud",
os.path.join(OPENMVS_BIN, "DensifyPointCloud"),
["scene.mvs", "--dense-config-file", "Densify.ini", "--resolution-level", "1", "--number-views", "8", "-w", "\"%mvs_dir%\""]],
["Reconstruct the mesh",
os.path.join(OPENMVS_BIN, "ReconstructMesh"),
["scene_dense.mvs", "-w", "\"%mvs_dir%\""]],
["Refine the mesh",
os.path.join(OPENMVS_BIN, "RefineMesh"),
["scene_dense_mesh.mvs", "--scales", "1", "--gradient-step", "25.05", "-w", "\"%mvs_dir%\""]],
["Texture the mesh",
os.path.join(OPENMVS_BIN, "TextureMesh"),
["scene_dense_mesh_refine.mvs", "--decimate", "0.5", "-w", "\"%mvs_dir%\""]],
["Estimate disparity-maps",
os.path.join(OPENMVS_BIN, "DensifyPointCloud"),
["scene.mvs", "--dense-config-file", "Densify.ini", "--fusion-mode", "-1", "-w", "\"%mvs_dir%\""]],
["Fuse disparity-maps",
os.path.join(OPENMVS_BIN, "DensifyPointCloud"),
["scene.mvs", "--dense-config-file", "Densify.ini", "--fusion-mode", "-2", "-w", "\"%mvs_dir%\""]]
]
def __getitem__(self, indice):
return AStep(*self.steps_data[indice])
def length(self):
return len(self.steps_data)
def apply_conf(self, conf):
""" replace each %var% per conf.var value in steps data """
for s in self.steps_data:
o2 = []
for o in s[2]:
co = o.replace("%input_dir%", conf.input_dir)
co = co.replace("%output_dir%", conf.output_dir)
co = co.replace("%matches_dir%", conf.matches_dir)
co = co.replace("%reconstruction_dir%", conf.reconstruction_dir)
co = co.replace("%mvs_dir%", conf.mvs_dir)
co = co.replace("%camera_file_params%", conf.camera_file_params)
o2.append(co)
s[2] = o2
def replace_opt(self, idx, str_exist, str_new):
""" replace each existing str_exist with str_new per opt value in step idx data """
s = self.steps_data[idx]
o2 = []
for o in s[2]:
co = o.replace(str_exist, str_new)
o2.append(co)
s[2] = o2
CONF = ConfContainer()
STEPS = StepsStore()
PARSER = argparse.ArgumentParser(
formatter_class=argparse.RawTextHelpFormatter,
description="Photogrammetry reconstruction with these steps: \r\n" +
"\r\n".join(("\t%i. %s\t %s" % (t, STEPS[t].info, STEPS[t].cmd) for t in range(STEPS.length())))
)
PARSER.add_argument('input_dir',
help="the directory which contains the pictures set.")
PARSER.add_argument('output_dir',
help="the directory which will contain the resulting files.")
PARSER.add_argument('--steps',
type=int,
nargs="+",
help="steps to process")
PARSER.add_argument('--preset',
help="steps list preset in \r\n" +
" \r\n".join([k + " = " + str(PRESET[k]) for k in PRESET]) +
" \r\ndefault : " + PRESET_DEFAULT)
GROUP = PARSER.add_argument_group('Passthrough', description="Option to be passed to command lines (remove - in front of option names)\r\ne.g. --1 p ULTRA to use the ULTRA preset in openMVG_main_ComputeFeatures\r\nFor example, running the script as follows,\r\nMvgMvsPipeline.py input_dir output_dir --1 p HIGH n 8 --3 n ANNL2\r\nwhere --1 refer to openMVG_main_ComputeFeatures, p refers to\r\ndescriberPreset option which HIGH was chosen, and n refers to\r\nnumThreads which 8 was used. --3 refer to second step (openMVG_main_ComputeMatches),\r\nn refers to nearest_matching_method option which ANNL2 was chosen")
for n in range(STEPS.length()):
GROUP.add_argument('--'+str(n), nargs='+')
PARSER.parse_args(namespace=CONF)
def mkdir_ine(dirname):
"""Create the folder if not presents"""
if not os.path.exists(dirname):
os.mkdir(dirname)
CONF.input_dir = os.path.abspath(CONF.input_dir)
CONF.output_dir = os.path.abspath(CONF.output_dir)
if not os.path.exists(CONF.input_dir):
sys.exit("%s: path not found" % CONF.input_dir)
CONF.reconstruction_dir = os.path.join(CONF.output_dir, "sfm")
CONF.matches_dir = os.path.join(CONF.reconstruction_dir, "matches")
CONF.mvs_dir = os.path.join(CONF.output_dir, "mvs")
CONF.camera_file_params = os.path.join(CAMERA_SENSOR_DB_DIRECTORY, CAMERA_SENSOR_DB_FILE)
mkdir_ine(CONF.output_dir)
mkdir_ine(CONF.reconstruction_dir)
mkdir_ine(CONF.matches_dir)
mkdir_ine(CONF.mvs_dir)
STEPS.apply_conf(CONF)
if CONF.steps and CONF.preset:
sys.exit("Steps and preset arguments can't be set together.")
elif CONF.preset:
try:
CONF.steps = PRESET[CONF.preset]
except KeyError:
sys.exit("Unknown preset %s, choose %s" % (CONF.preset, ' or '.join([s for s in PRESET])))
elif not CONF.steps:
CONF.steps = PRESET[PRESET_DEFAULT]
print("# Using input dir: %s" % CONF.input_dir)
print("# output dir: %s" % CONF.output_dir)
print("# Steps: %s" % str(CONF.steps))
if 4 in CONF.steps:
if 6 in CONF.steps:
STEPS.replace_opt(4, FOLDER_DELIM+"matches.f.bin", FOLDER_DELIM+"matches.e.bin")
STEPS[4].opt.extend(["-g", "e"])
if 15 in CONF.steps:
if 14 not in CONF.steps:
STEPS.replace_opt(15, "scene_dense_mesh_refine.mvs", "scene_dense_mesh.mvs")
for cstep in CONF.steps:
printout("#%i. %s" % (cstep, STEPS[cstep].info), effect=INVERSE)
opt = getattr(CONF, str(cstep))
if opt:
for o in range(0, len(opt), 2):
if len(opt[o]) > 1:
opt[o] = '-' + opt[o]
opt[o] = '-' + opt[o]
else:
opt = []
for anOpt in STEPS[cstep].opt:
if anOpt in opt:
idx = STEPS[cstep].opt.index(anOpt)
if DEBUG:
print('#\tRemove ' + str(anOpt) + ' from defaults options at id ' + str(idx))
del STEPS[cstep].opt[idx:idx+2]
cmdline = [STEPS[cstep].cmd] + STEPS[cstep].opt + opt
print('Cmd: ' + ' '.join(cmdline))
if not DEBUG:
try:
pStep = subprocess.Popen(cmdline)
pStep.wait()
if pStep.returncode != 0:
break
except KeyboardInterrupt:
sys.exit('\r\nProcess canceled by user, all files remains')
else:
print('\t'.join(cmdline))
printout("# Pipeline end #", effect=INVERSE)
reference:
https://openmvg.readthedocs.io/en/latest/software/SfM/SfMInit_ImageListing/
|