AstroMind: Multi-Agent RAG System for Astrophotography

Project Overview

AstroMind is a personal research project exploring the intersection of multi-agent systems, RAG (Retrieval-Augmented Generation), and astrophysics workflows. As both an ML engineer and amateur astrophotographer, I built this system to automate and enhance various aspects of astronomical image acquisition, processing, and analysis.

Motivation

Astrophotography involves numerous repetitive but knowledge-intensive tasks:

• Planning imaging sessions based on object visibility, moon phase, weather
• Plate solving (determining exact coordinates from images)
• Object identification and catalog cross-referencing
• Processing pipeline selection and optimization
• Analysis of image quality and astronomical features

These tasks require domain knowledge spread across catalogs, papers, and community wisdom. AstroMind uses a multi-agent architecture with RAG to make this knowledge accessible through natural language.

Architecture

Agent System

Built on a multi-agent framework with specialized agents:

Vision Agent

• Identifies celestial objects in images
• Performs astrometry (plate solving)
• Detects artifacts and quality issues
• Suggests processing improvements

Research Agent

• Queries astronomical databases (SIMBAD, NED, etc.)
• Retrieves scientific papers and data
• Synthesizes information from multiple sources
• Generates research summaries

Processing Agent

• Optimizes image stacking parameters
• Recommends calibration procedures
• Automates routine processing tasks
• Monitors processing quality

Coordinator Agent

• Orchestrates multi-agent workflows
• Manages task decomposition
• Handles agent communication
• Ensures goal achievement

Technical Stack

Backend

• Python with FastAPI for API server
• LangChain for agent orchestration
• Custom fine-tuned vision models
• PostgreSQL for data persistence

Frontend

• Next.js with TypeScript
• Real-time agent communication via WebSockets
• Interactive visualizations with D3.js
• Image viewer with annotations

Key Features

Natural Language Queries

Users can ask complex questions in plain English:

• "What nebulae are visible from my location tonight?"
• "Process this image and identify any galaxies"
• "Compare my Orion image to professional data"

Automated Workflows

Pre-built and custom workflows for common tasks:

• End-to-end image processing pipelines
• Object catalogs and field identification
• Equipment planning and recommendations
• Weather and seeing condition analysis

Learning from Feedback

The system improves over time:

• Fine-tuning vision models on user data
• Learning user preferences
• Adapting to equipment characteristics
• Building knowledge base from interactions

Technical Challenges

Vision Model Accuracy

Astronomical images differ significantly from natural images:

• Solved by fine-tuning on astronomical datasets
• Data augmentation for various imaging conditions
• Ensemble models for robust detection
• Uncertainty quantification for predictions

Agent Coordination

Orchestrating multiple agents effectively:

• Clear task decomposition strategies
• Efficient communication protocols
• Conflict resolution mechanisms
• Monitoring and debugging tools

Database Integration

Querying diverse astronomical databases:

• Unified query interface across catalogs
• Caching strategies for performance
• Handling API rate limits
• Data normalization and validation

Results & Usage

• Successfully processes 100+ images per day
• Identifies objects with 95%+ accuracy
• Reduces manual research time by 70%
• Active use in my personal astrophotography workflow

Open Source

The project is open source on GitHub with:

• Comprehensive documentation
• Example workflows and tutorials
• Community contributions welcome
• Regular updates and improvements

Future Development

• Mobile app for field use
• Real-time sky monitoring
• Integration with telescope control
• Community sharing platform
• Support for spectroscopy data