PHANTOM - Development Guide for Claude
Living Machine Learning Framework - Production-grade document intelligence, RAG pipeline, and AI classification system.
Version: 0.0.1 (Pre-Alpha) Python: 3.11+ License: Apache-2.0 Last Updated: 2026-03-26
📋 Table of Contents
- Project Overview
- Current State Assessment
- Architecture Quick Reference
- Development Priorities
- Quality Assessment
- Frontend-Backend Integration
- Key Files Reference
- Testing Strategy
- Common Tasks
- Known Issues & TODOs
🎯 Project Overview
What is Phantom?
Phantom is a local-first AI document intelligence framework that processes unstructured documents into actionable intelligence using:
- CORTEX Engine: Semantic chunking, parallel LLM classification, insight extraction
- RAG Pipeline: FAISS vector indexing with sentence-transformers embeddings
- Multi-Interface: CLI (Typer), REST API (FastAPI), Desktop UI (Tauri + SvelteKit)
- Production-Ready: VRAM monitoring, auto-throttling, thread pool concurrency, Prometheus metrics
- Fully Reproducible: Nix flake with locked dependencies
Core Capabilities
- Document Processing: Extract insights from markdown, text, PDFs
- Vector Search: Semantic search over document embeddings (FAISS)
- Classification: Multi-threaded LLM-based document classification
- Sentiment Analysis: NLTK VADER + optional spaCy NER
- RAG Pipeline: Question-answering over knowledge base
- Resource Management: Real-time VRAM/RAM monitoring with auto-throttling
Tech Stack
- Backend: Python 3.11+, FastAPI, Pydantic 2.0
- Frontend: Tauri 2.0, SvelteKit, Svelte 5, Tailwind CSS
- ML/NLP: sentence-transformers, FAISS, NLTK, scikit-learn, tiktoken
- Inference: llama.cpp (local, OpenAI-compatible API)
- Agent: Rust (Crane build system, multi-crate workspace)
- DevOps: Nix flake, GitHub Actions (8 workflows), pre-commit hooks
- Observability: structlog, Prometheus metrics
📊 Current State Assessment
✅ Production-Ready Components
| Component | Status | Files | Test Coverage |
|---|---|---|---|
| CORTEX Processor | ✅ Complete | core/cortex.py | High |
| Semantic Chunker | ✅ Complete | core/cortex.py | High |
| FAISS Vector Store | ✅ Complete | rag/vectors.py | High |
| Sentiment Engine | ✅ Complete | analysis/sentiment.py | High |
| Embedding Generator | ✅ Complete | core/embeddings.py | Medium |
| LlamaCpp Provider | ✅ Complete | providers/llamacpp.py | Medium |
| FastAPI Server | ✅ Complete | api/app.py | High |
| Prometheus Metrics | ✅ Complete | api/app.py | High |
| Pydantic Schemas | ✅ Complete | All modules | High |
| CI/CD Pipelines | ✅ Complete | .github/workflows/ | N/A |
| Nix Environment | ✅ Complete | flake.nix | N/A |
| CLI Commands | ✅ Complete | cli/main.py | Medium |
| RAG Query API | ✅ Complete | api/app.py | High |
| Document Upload | ✅ Complete | api/app.py | High |
| Vector Indexing API | ✅ Complete | api/app.py | High |
| SSE Streaming Chat | ✅ Complete | api/app.py | Medium |
🟡 Partially Implemented Components
| Component | Status | Missing | Priority |
|---|---|---|---|
| Desktop UI | 🟡 Framework | Component polish, e2e tests | Medium |
| tools vram | 🟡 Partial | Model-specific VRAM estimates | Low |
| Cloud LLM Providers | 🟡 Stub | OpenAI, Anthropic, DeepSeek impl | Medium |
❌ Not Implemented
- Cloud LLM Providers (OpenAI, Anthropic, DeepSeek)
- Kubernetes/Helm packaging
- Full desktop app UI (marked for GTK4 migration)
- Redis semantic cache integration
- Advanced prompt workbench features
Code Quality Metrics
- Total Python LOC: 11,290 (33 source files)
- Test Files: 18 (unit, integration, e2e)
- Test Coverage: 70% minimum (enforced via pytest)
- Linting: Ruff (enforced in CI)
- Type Checking: mypy (enabled, non-strict mode)
- Security Scanning: bandit, pip-audit, cargo-audit
- Documentation: 20+ markdown files
🏗️ Architecture Quick Reference
Directory Structure
phantom/
├── src/phantom/ # Main Python source (11,290 LOC)
│ ├── core/ # CORTEX engine, embeddings, chunking
│ ├── rag/ # Vector stores (FAISS), search
│ ├── analysis/ # Sentiment, SPECTRE, viability
│ ├── pipeline/ # DAG orchestration, classification
│ ├── providers/ # LLM providers (llama.cpp base)
│ ├── cerebro/ # RAG engine + knowledge integration
│ ├── neutron/ # Compliance guardrails (SENTINEL)
│ ├── api/ # FastAPI REST server + Judge API
│ └── cli/ # Typer CLI interface
├── tests/ # 18 test files
│ ├── unit/ # Unit tests (isolated components)
│ ├── integration/ # API + CLI tests
│ └── e2e/ # End-to-end pipeline tests
├── cortex-desktop/ # Tauri + SvelteKit + Svelte 5
├── intelagent/ # Rust agent (multi-crate workspace)
│ ├── crates/security/ # Privacy & audit modules
│ ├── crates/governance/ # DAO & rewards systems
│ ├── crates/memory/ # Context & knowledge graphs
│ ├── crates/quality/ # Automated peer review gates
│ └── crates/mcp/ # MCP protocol handlers
├── docs/ # 20+ markdown documentation files
├── nix/ # NixOS module definitions
└── .github/workflows/ # CI/CD pipelines (8 workflows)