Go to CVPR 2025 Workshop VisCon homepageMemory-Modular Classification: Novel-Class Generalization with Web-Crawled Memory
Keywords: Zero-shot classification, Novel class generalization
TL;DR: We present memory-modular learner (MML) for web-assisted zero-shot classification. MML classifies novel classes by loading the relevant contents crawled from internet into the memory
Abstract: We propose a memory-modular learner for zero-shot image classification that separates knowledge memorization from reasoning.
Our model enables generalization to novel visual concepts by simply replacing the memory contents, without the need for model retraining.
Unlike traditional models that encode both world knowledge and task-specific skills into their weights during training, our model stores knowledge in the external memory of web-crawled image and text data.
At inference time, the model dynamically selects relevant content from the memory based on the input image, allowing it to adapt to arbitrary visual concepts by simply replacing the memory contents.
The key differentiator is that our learner meta-learns to perform classification tasks with web-crawled data for classifying novel visual concepts.
Experimental results demonstrate the promising performance and versatility of our approach in handling diverse classification tasks, including zero-shot/few-shot classification of unseen classes, fine-grained classification, and class-incremental classification.
Submission Number: 32
Loading