Thesis proposals

This thesis aims to develop a set of Python tools for the visualization and interactive exploration of medical volumetric data and 3D shape reconstructions. The project will leverage both MRI volumes and their corresponding 3D segmented shapes to build flexible and user-friendly packages for data inspection, rendering, and interactive exploration. The focus will be on creating integrated visualization pipelines that allow users to combine volumetric views with surface meshes and manipulate rendering parameters. Additionally, the project will include Python scripts and modules to interface with Blender, enabling high-quality 3D visualizations, animations, and automated rendering of anatomical structures. The objective is to provide a comprehensive toolkit for researchers in medical imaging.

Data

The dataset consists of real 3D MRI volumes of different anatomical structures. Each volume includes segmentation labels for anatomical structure, which will be used to generate visualizations.

Expected Outcomes

• Python visualization modules capable of rendering MRI volumes, segmented structures, and 3D surfaces.

• Integrated visualization pipelines combining volumetric and surface representations, with interactive exploration features.

• Python scripts for Blender integration to automate import and rendering 3D shapes.

References (State of the Art)

• Schroeder et al. The Visualization Toolkit (VTK). 4th edition, Kitware

• Blender Foundation. Blender Python API Documentation.

This thesis aims to generate and study a collection of 3D shapes derived from real medical volumetric acquisitions and their corresponding segmentation. Specifically, the project will focus on brain magnetic resonance imaging (MRI) data to reconstruct surface representations of relevant brain structures. From these 3D reconstructions, geometric features will be extracted using both data-driven (e.g., deep learning-based feature embeddings) and non-data-driven (e.g., handcrafted geometric descriptors such as curvature) approaches. Ultimately, these features will be employed in a practical classification scenario, such as distinguishing between healthy and pathological brain structures using standard machine learning models (e.g., Support Vector Machines).

Data

The dataset consists of real 3D MRI volumes representing both healthy and pathological brains. Each volume includes segmentation labels for relevant brain, which will be used to derive their surface mesh representations.

Expected Outcomes

• Extraction of 3D brain surfaces from clinical MRI data and creation of a curated collection of 3D shape models.

• Extraction of geometric features from the reconstructed shapes using both data-driven and non-data-driven approaches.

• Application of classification models to the extracted features to perform a binary classification task (i.e., distinguishing healthy from pathological cases).

References (State of the Art)

• Li et al., MedShapeNet: a large-scale dataset of 3D medical shapes for computer vision. Biomedizinische Technik (Biomedical Engineering)

This thesis proposes the development of a comprehensive Python package for the advanced visualization of 3D geometric shapes. The framework is designed to support diverse geometric representations, including triangle meshes and point clouds (with potential integration for implicit representations such as Signed Distance Functions). The core objective is to enable the association and plotting of scalar and vectorial feature functions directly onto the geometric elements (vertices, edges, and faces) of the shapes.

Key Features

The package will implement the following core capabilities:

• Default Shape Rendering: visualization of 3D shapes using a customizable default shader.
• Scalar Function Mapping (Heatmaps): support for single-valued functions (1-valued) represented as color heatmaps.
• Vector Function Mapping (RGB Encoding): support for 3-valued functions, directly mapping components to RGB color channels for intuitive visualization.
• High-Dimensional Function Mapping: support for N-valued functions, allowing dimensionality reduction techniques (e.g., PCA into 3 dimensions) before RGB color encoding.
• Value Normalization and Clipping: robust data normalization (to [0, 1]) with user-defined input ranges and default fallbacks to min-max scaling, including value clipping for out-of-range inputs.
• Export Functionality: saving of plots in both interactive formats (HTML) and high-quality static images.
• Extensible Backend Architecture: a unified API design to facilitate support for multiple underlying rendering backends.

Technologies

• Python: Core programming language.
• PyTorch: for integration with deep learning pipelines and efficient tensor handling.
• Plotly and Matplotlib: For interactive and static 2D/3D plotting and data visualization components.
• Polyscope: As a primary candidate or reference for the 3D rendering backend due to its focus on geometric data visualization.

Expected Outcomes

• A fully documented and tested open-source Python package for 3D shape and function visualization.
• A flexible API that decouples the data visualization logic from the underlying rendering engine.
• Demonstrations of the tool’s capabilities through the visualization of complex feature functions (e.g. geometric properties, neural network activations) on standard 3D models.

This thesis investigates the application of spectral graph properties—specifically the Fiedler vector and eigenvector centrality—as positional encodings in Transformer and Graph Neural Network (GNN) architectures for graph and mesh data. The study will examine two approaches: utilizing the absolute values of the Fiedler vector as features and employing it to order vertices, thereby defining a canonical node sequence. Additionally, eigenvector centrality will be explored as an alternative positional encoding method.

Data

Standard graph and mesh datasets from the literature, such as FAUST and ShapeNet for 3D meshes, and benchmark graph datasets like Cora and PubMed for general graph experiments.

Expected Outcomes

• Comparative analysis of positional encoding strategies based on the Fiedler vector and eigenvector centrality.
• Evaluation of their impact on tasks such as node classification, segmentation, and regression on graphs and meshes.
• Insights into the effectiveness of spectral-based encodings in enhancing the performance of GNNs and Transformers on geometric data.

References (State of the Art)

• Kreuzer et al. (2021). Rethinking Graph Transformers with Spectral Attention. NeurIPS 2021. https://openreview.net/pdf?id=huAdB-Tj4yG
• Liu, R., et al. (2023). Graph Positional and Structural Encoder. arXiv. https://arxiv.org/pdf/2307.07107
• Ajayi, O., et al. (2024). NAPE: Numbering as a Position Encoding in Graphs.
• Liang, B., et al. (2024). Centrality-guided Pre-training for Graph. OpenReview. https://openreview.net/forum?id=X8E65IxA73
• Huang, Y., et al. (2023). On the Stability of Expressive Positional Encodings for Graphs. arXiv. https://arxiv.org/abs/2310.02579

This thesis explores the use of image compression techniques, specifically JPEG, to reduce the dimensionality of matrices representing correspondences between 3D shapes. Shape matching problems often rely on large-scale matrices, such as point-to-point correspondence maps or functional maps, which can be computationally expensive to store and optimize. By interpreting these matrices as images and compressing them using JPEG, we aim to significantly reduce the number of variables in the optimization problem, while maintaining the accuracy and quality of the resulting correspondences. The approach will be evaluated both on complete correspondence matrices and on alternative matrix representations, such as functional maps.

Data

Standard 3D shape matching benchmarks, including FAUST, SCAPE, TOSCA, and SMAL. SHREC 2019 and SHREC 2020 datasets, commonly used for evaluating shape correspondence methods.

Expected Outcomes

• An implementation of a JPEG-based compression pipeline for correspondence matrices.
• A systematic comparison of optimization performance with and without matrix compression under identical settings.
• Quantitative evaluation of efficiency (memory usage, runtime) and correspondence quality (accuracy, stability).
• Insights into the trade-off between compression ratio and optimization reliability across different datasets and correspondence representations.

References (State of the Art)

• Ovsjanikov et al., Functional Maps: A Flexible Representation of Maps Between Shapes, SIGGRAPH 2012.
• Melzi et al., SHREC 2019: Matching Humans with Different Connectivity, Computers \& Graphics 2019.