Neuro-Symbolic Visual Reasoning 🧠

Neuro-Symbolic Visual Reasoning 🧠
View Report

Project Overview

TLDR: Demo Link

Many children struggle with spatial and visual reasoning—an essential skill in STEM learning. Traditional teaching tools often fail to build deep understanding. This project introduces Spatial Quest, a neuro-symbolic AI system that combines neural perception with symbolic logic to help children learn spatial reasoning through interactive visual scenes and logical queries.

My Role

  • Designed and implemented a hybrid neuro-symbolic reasoning pipeline
  • Curated and fine-tuned datasets (CLEVR) for training and benchmarking
  • Developed visual UI for interactive scene creation and query testing
  • Encoded logical rules for compositional and nested spatial queries
  • Conducted evaluations and performance benchmarking

Key Outcomes

  • Understanding the Problem: Case studies like Ishan’s highlight common struggles with mental rotation, scaling, and perspective-taking in school-age children.

  • Hybrid AI Solution: Combines object detection with logic-based reasoning to answer spatial queries such as:

    • “How many red spheres are to the left of the tall cube?”
    • “Find objects that are (red AND metallic) OR (small BUT NOT spherical)”

  • System Workflow:

    System Diagram

    • Input: Scene + user query
    • Detect: Neural network identifies objects & attributes
    • Reason: Symbolic logic processes spatial rules
    • Answer: Outputs query result with explainable logic

  • Performance Highlights:

    • Achieved 95.2% accuracy on complex CLEVR queries
    • 3x faster reasoning than end-to-end neural models
    • Robust to noise and partial scene information

  • Interactive UI & Explainability:

    • Web-based drag-and-drop interface
    • Real-time visual explanations of logical reasoning
    • Engaging learning experience for children

  • Demo

Impact & Future Scope

This neuro-symbolic tool supports cognitive growth by making abstract spatial concepts tangible and interactive. The project demonstrates the power of explainable AI in education and opens avenues for future integration with real-world images and educational platforms.