PromptMoGe

Metric depth from one RGB frame and a phone LiDAR: MoGe-3 ViT-L prompted with the 256×192 LiDAR depth of an iPhone Pro / iPad Pro, plus two compressed variants that run end to end on the device. Project page: https://sergmister.github.io/PromptMoGe/ · code, usage, training and the iOS demo: https://github.com/sergmister/PromptMoGe

file	model	point map	refiner	iPad Pro 11-inch (M5), K = 1
promptmoge_l.pt	PromptMoGe-L, the teacher	any resolution	fp16	—
promptmoge_a.pt	Model A	480×640	int8 QAT	152 ms
promptmoge_b.pt	Model B	240×320	int8 QAT	129 ms
ios/models/	ready-to-run device models: shared ViT (vit/, Core ML, split across Neural Engine and GPU) and, per model (A/, B/), the prompt, neck and head Core ML models and the int8 refiner weights for the Metal engine

Each checkpoint holds only the tensors that differ from MoGe-3 (prompt stem and pyramid, neck, heads, refiner); the frozen DINOv2 backbone is loaded from Ruicheng/moge-3-vitl.

from promptmoge import load_model, infer          # pip install -e . from the GitHub repository
model = load_model("A", device="cuda")            # "L", "A" or "B"; downloads from this repository
out = infer(model, rgb_uint8, lidar_metres, lidar_confidence, refine_steps=1)
depth, points = out["depth"], out["points"]       # metres, camera space

AbsRel on ARKitScenes against laser-scan ground truth (3 held-out captures, 1 094 frames, 1 200 tokens):

	all pixels	confident pixels
raw ARKit LiDAR	0.0211	0.0192
PromptDA-L, 420×560 (1 200 tokens)	0.0149	0.0133
PromptMoGe-L	0.0137	0.0122
Model A	0.0138	0.0122
Model B	0.0142	0.0125

The iOS models can be regenerated from the checkpoints with python -m promptmoge.export.coreml and python -m promptmoge.export.refiner.

MIT license. Built on MoGe (Microsoft, MIT) and DINOv2 (Meta AI, Apache 2.0).