Zorah Lähner

Zorah Lähner

Assistant Professor (tenure track), Geometry in Machine Learning Group

Posts by Collection

news

portfolio

publications

Isometric Multi-Shape Matching

Published in Conference on Computer Vision and Pattern Recognition (CVPR), 2021

This paper is about the number 1. The number 2 is left for future work.

Download here

QuAnt: Quantum Annealing with Learnt Couplings

Published in International Conference on Learning Representations (ICLR), 2023

Modern quantum annealers can find high-quality solutions to combinatorial optimisation objectives given as quadratic unconstrained binary optimisation (QUBO) problems. Unfortunately, obtaining suitable QUBO forms in computer vision remains challenging and currently requires problem-specific analytical derivations. Moreover, such explicit formulations impose tangible constraints on solution encodings. In stark contrast to prior work, this paper proposes to learn QUBO forms from data through gradient backpropagation instead of deriving them. As a result, the solution encodings can be chosen flexibly and compactly. Furthermore, our methodology is general and virtually independent of the specifics of the target problem type. We demonstrate the advantages of learnt QUBOs on the diverse problem types of graph matching, 2D point cloud alignment and 3D rotation estimation. Our results are competitive with the previous quantum state of the art while requiring much fewer logical and physical qubits, enabling our method to scale to larger problems.

Download here

CCuantuMM: Cycle-Consistent Quantum-Hybrid Matching of Multiple Shapes

Published in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

Jointly matching multiple, non-rigidly deformed 3D shapes is a challenging, N P-hard problem. A perfect matching is necessarily cycle-consistent: Following the pairwise point correspondences along several shapes must end up at the starting vertex of the original shape. Unfortunately, existing quantum shape-matching methods do not support multiple shapes and even less cycle consistency. This paper addresses the open challenges and introduces the first quantum-hybrid approach for 3D shape multi-matching; in addition, it is also cycle-consistent. Its iterative formulation is admissible to modern adiabatic quantum hardware and scales linearly with the total number of input shapes. Both these characteristics are achieved by reducing the N -shape case to a sequence of three-shape matchings, the derivation of which is our main technical contribution. Thanks to quantum annealing, high-quality solutions with low energy are retrieved for the intermediate NP-hard objectives. On benchmark datasets, the proposed approach significantly outperforms extensions to multi-shape matching of a previous quantum-hybrid two-shape matching method and is on-par with classical multi-matching methods.

Download here

Conjugate Product Graphs for Globally Optimal 2D-3D Shape Matching

Published in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

We consider the problem of finding a continuous and non-rigid matching between a 2D contour and a 3D mesh. While such problems can be solved to global optimality by finding a shortest path in the product graph between both shapes, existing solutions heavily rely on unrealistic prior assumptions to avoid degenerate solutions (e.g. knowledge to which region of the 3D shape each point of the 2D contour is matched). To address this, we propose a novel 2D-3D shape matching formalism based on the conjugate product graph between the 2D contour and the 3D shape. Doing so allows us for the first time to consider higher-order costs, i.e. defined for edge chains, as opposed to costs defined for single edges. This offers substantially more flexibility, which we utilise to incorporate a local rigidity prior. By doing so, we effectively circumvent degenerate solutions and thereby obtain smoother and more realistic matchings, even when using only a one-dimensional feature descriptor. Overall, our method finds globally optimal and continuous 2D-3D matchings, has the same asymptotic complexity as previous solutions, produces state-of-the-art results for shape matching and is even capable of matching partial shapes.

Download here

A Network Analysis for Correspondence Learning via Linearly-Embedded Functions

Published in German Conference on Pattern Recognition (GCPR), 2023

Calculating correspondences between non-rigidly deformed shapes is the backbone of many applications in 3D computer vision and graphics. The functional map approach offers an efficient solution to this problem and has been very popular in learning frameworks due to its low-dimensional and continuous nature. However, most methods rely on the eigenfunctions of the Laplace-Beltrami operator as a basis for the underlying function spaces. While these have many advantages, they are also sensitive to non-isometric deformations and noise. Recently a method to learn the basis functions along with suitable descriptors has been proposed by Marin et al. We do an in-depth analysis of the architecture proposed, including a new training scheme to increase robustness against sampling inconsistencies and an extension to unsupervised training which still obtains results on-par with the supervised approach.

Download here

SIGMA: Scale-Invariant Global Sparse Shape Matching

Published in International Conference on Computer Vision (ICCV), 2023

We propose a novel mixed-integer programming (MIP) formulation for generating precise sparse correspondences for highly non-rigid shapes. To this end, we introduce a projected Laplace-Beltrami operator (PLBO) which combines intrinsic and extrinsic geometric information to measure the deformation quality induced by predicted correspondences. We integrate the PLBO, together with an orientation-aware regulariser, into a novel MIP formulation that can be solved to global optimality. In contrast to previous methods, our approach is provably invariant to rigid transformations and global scaling, intialisation-free, has optimality guarantees, and scales to high resolution meshes with (empirically observed) linear time. We show state-of-the-art results for sparse non-rigid matching on several challenging 3D datasets, including inconsistent meshing, as well as applications in mesh-to-point-cloud matching.

Download here

Kissing to Find a Match: Efficient Low-Rank Permutation Representation

Published in Neural Information Processing Systems (NeuRIPS), 2023

Permutation matrices play a key role in matching and assignment problems across the fields, especially in computer vision and robotics. However, memory for explicitly representing permutation matrices grows quadratically with the size of the problem, prohibiting large problem instances. In this work, we propose to tackle the curse of dimensionality of large permutation matrices by approximating them using low-rank matrix factorization, followed by a nonlinearity. To this end, we rely on the Kissing number theory to infer the minimal rank required for representing a permutation matrix of a given size, which is significantly smaller than the problem size. This leads to a drastic reduction in computation and memory costs, e.g., up to 3 orders of magnitude less memory for a problem of size n = 20000, represented using 8.4 × 105 elements in two small matrices instead of using a single huge matrix with 4 × 108 elements. The proposed representation allows for accurate representations of large permutation matrices, which in turn enables handling large problems that would have been infeasible otherwise. We demonstrate the applicability and merits of the proposed approach through a series of experiments on a range of problems that involve predicting permutation matrices, from linear and quadratic assignment to shape matching problems.

Download here

On the Direct Alignment of Latent Spaces

Published in NeuRIPS Workshop on Unifying Representations in Neural Models (UniReps), 2023

With the wide adaption of deep learning and pre-trained models rises the question of how to effectively reuse existing latent spaces for new applications. One important question is how the geometry of the latent space changes in-between different training runs of the same architecture and different architectures trained for the same task. Previous works proposed that the latent spaces for similar tasks are approximately isometric. However, in this work we show that method restricted to this assumption perform worse than when just using a linear transformation to align the latent spaces. We propose directly computing a transformation between the latent codes of different architectures which is more efficient than previous approaches and flexible wrt. to the type of transformation used. Our experiments show that aligning the latent space with a linear transformation performs best while not needing more prior knowledge.

Download here

Hybrid Functional Maps for Crease-Aware Non-Isometric Shape Matching

Published in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

Non-isometric shape correspondence remains a fundamental challenge in computer vision. Traditional methods using Laplace-Beltrami operator (LBO) eigenmodes face limitations in characterizing high-frequency extrinsic shape changes like bending and creases. We propose a novel approach of combining the non-orthogonal extrinsic basis of eigenfunctions of the elastic thin-shell hessian with the intrinsic ones of the LBO, creating a hybrid spectral space in which we construct functional maps. To this end, we present a theoretical framework to effectively integrate non-orthogonal basis functions into descriptor- and learning-based functional map methods. Our approach can be incorporated easily into existing functional map pipelines across varying applications and is able to handle complex deformations beyond isometries. We show extensive evaluations across various supervised and unsupervised settings and demonstrate significant improvements. Notably, our approach achieves up to 15% better mean geodesic error for non-isometric correspondence settings and up to 45% improvement in scenarios with topological noise.

Implicit-ARAP: Efficient Handle-Guided Deformation of High-Resolution Meshes and Neural Fields via Local Patch Meshing

Published in arXiv:2405.12895, 2024

In this work, we present the local patch mesh representation for neural signed distance fields. This technique allows to discretize local regions of the level sets of an input SDF by projecting and deforming flat patch meshes onto the level set surface, using exclusively the SDF information and its gradient. Our analysis reveals this method to be more accurate than the standard marching cubes algorithm for approximating the implicit surface. Then, we apply this representation in the setting of handle-guided deformation: we introduce two distinct pipelines, which make use of 3D neural fields to compute As-Rigid-As-Possible deformations of both high-resolution meshes and neural fields under a given set of constraints. We run a comprehensive evaluation of our method and various baselines for neural field and mesh deformation which show both pipelines achieve impressive efficiency and notable improvements in terms of quality of results and robustness. With our novel pipeline, we introduce a scalable approach to solve a well-established geometry processing problem on high-resolution meshes, and pave the way for extending other geometric tasks to the domain of implicit surfaces via local patch meshing.

3D Shape Completion with Test-Time Training

Published in ICLR Workshop on Geometry-Grounded Representation Learning and Generative Modeling (GRaM), 2024

This work addresses the problem of shape completion, i.e., the task of restoring incomplete shapes by predicting their missing parts. While previous works have often predicted the fractured and restored shape in one step, we approach the task by separately predicting the fractured and newly restored parts, but ensuring these predictions are interconnected. We use a decoder network motivated by related work on the prediction of signed distance functions (DeepSDF). In particular, our representation allows us to consider test-time-training, i.e., finetuning network parameters to match the given incomplete shape more accurately during inference. While previous works often have difficulties with artifacts around the fracture boundary, we demonstrate that our overfitting to the fractured parts leads to significant improvements in the restoration of eight different shape categories of the ShapeNet data set in terms of their chamfer distances.

Synchronous Diffusion for Unsupervised Smooth Non-Rigid 3D Shape Matching

Published in European Conference on Computer Vision (ECCV) (Accepted), 2024

Most recent unsupervised non-rigid 3D shape matching methods are based on the functional map framework due to its efficiency and superior performance. Nevertheless, respective methods struggle to obtain spatially smooth pointwise correspondences due to the lack of proper regularisation. In this work, inspired by the success of message passing on graphs, we propose a synchronous diffusion process which we use as regularisation to achieve smoothness in non-rigid 3D shape matching problems. The intuition of synchronous diffusion is that diffusing the same input function on two different shapes results in consistent outputs. Using different challenging datasets, we demonstrate that our novel regularisation can substantially improve the state-of-the-art in shape matching, especially in the presence of topological noise.

studentprojects

talks

teaching

Logik und Diskrete Strukturen

Mandatory lecture for Computer Science Bachelor students, University of Bonn, 2011

Teaching assistant with two weekly tutorials, supervised ~40 students.

Analysis of Three-Dimensional Shapes

Lecture for Computer Science Master students, Technical University of Munich, 2016

Organization of exercises, teaching assistant for weekly tutorial, supervised ~15 students.

Analysis of Three-Dimensional Shapes

Lecture for Computer Science Master students, Technical University of Munich, 2017

Organization of exercises, teaching assistant for weekly tutorial, supervised ~15 students.

Shape Analysis and Optimization

Seminar for Computer Science and Mathematics Master students, Technical University of Munich, 2018

Organization and supervisor in weekly seminar, supervised ~7 students.

Diskrete Strukturen

Mandatory Lecture for Computer Science (and related studies) Bachelor Students, Technical University of Munich, 2018

Teaching assistant for two weekly tutorials, supervised ~40 students.

Recent Advances in 3D Computer Vision

Seminar for Computer Science Master students, Technical University of Munich, 2020

Organization and supervisor in weekly seminar, supervised ~6 students.

Recent Advances in 3D Computer Vision

Seminar for Computer Science Master students, Technical University of Munich, 2020

Organization and supervisor in weekly seminar, supervised ~6 students.

Recent Advances in Machine Learning

Practical course for Computer Science and Mechanical Engineering Master students, University of Siegen, 2021

Supervision of project “Feature-Based Learning for 3D Correspondence”, supervised ~6 students.

Visual Computing

Seminar for Computer Science Bachelor and Master students, University of Siegen, 2021

Supervisor in weekly seminar, supervised 1 student.

Deep Learning

Lecture for Computer Science, Electrical Engineering and Mechanical Engineering Master students, University of Siegen, 2021

Organization and execution of exercises, around 100 students.

Einführung in Visual Computing

Lecture for Computer Science Bachelor students, University of Siegen, 2022

Organization and execution of exercises, around 30 students.