A scenic mountain landscape in Rocky Mountain National Park under a vast blue sky dotted with fluffy white clouds. The sun is visible on the upper right corner, casting a bright glare. A rugged mountain slope occupies the foreground, consisting mostly of rocky terrain with sparse vegetation, leading up to a ridge line that meets the sky. In the middle distance, the mountains continue to roll with varying shades of brown and hints of green, indicating sparse grasses or low bushes. In the far distance, a series of mountain ranges extend to the horizon, suggesting a vast wilderness area. It's a clear day, and the overall feel is of open space and natural beauty typical of a high-altitude environment.

SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images

Josh Myers-Dean, Jarek Reynolds, Brian Price, Yifei Fan, Danna Gurari

European Conference on Computer Vision (ECCV) 2024

A line of photos representing samples from our dataset with subpart decompositions.

TL;DR

We introduce SPIN, a hierarchical semantic segmentation dataset with subpart annotations, along with new evaluation metrics and benchmarks for model performance across different granularity levels.

Quick Links

Abstract

Hierarchical segmentation entails creating segmentations at varying levels of granularity. We introduce the first hierarchical semantic segmentation dataset with subpart annotations for natural images, which we call SPIN (SubPartImageNet). We also introduce two novel evaluation metrics to evaluate how well algorithms capture spatial and semantic relationships across hierarchical levels. We benchmark modern models across three different tasks and analyze their strengths and weaknesses across objects, parts, and subparts.

Bibtex

                @InProceedings{Myers-Dean_2024_ECCV,
                    author    = {Myers-Dean, Josh and Reynolds, Jarek and Price, Brian and Fan, Yifei and Gurari, Danna},
                    title     = {SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images},
                    booktitle = {European Conference on Computer Vision},
                    year      = {2024},
                }