Turn topics, links, and files into AI-generated research notebooks — summarize, explore, and ask anything.
Transform your research process with AI-powered intelligence
DecipherIt is a cutting-edge AI-powered research assistant inspired by Google NotebookLM that revolutionizes how researchers, students, and professionals explore, analyze, and synthesize information from the web. The platform transforms any combination of documents, URLs, or topics into comprehensive research notebooks complete with AI-generated summaries, interactive Q&A capabilities, audio overviews, visual mindmaps, and automatically generated FAQs.
Traditional research is time-consuming and fragmented. Researchers often struggle with:
DecipherIt addresses these challenges by leveraging Bright Data's MCP Server to provide unrestricted, intelligent web access combined with advanced AI agents that can understand, synthesize, and present information in multiple formats.
🔬 Deep Research DecipherIt's AI agents conduct comprehensive research by strategically planning data collection, discovering diverse sources through Bright Data's global search capabilities, and synthesizing information from multiple perspectives. The system can research any topic from current events to academic subjects, providing thorough analysis that rivals human researchers.
🔍 Multi-Source Research Users can combine various input types in a single research project: upload documents (PDF, DOCX, PPTX, XLSX), add custom URLs for specific web content, input manual text for direct analysis, or simply enter topics for AI-driven discovery. All sources are processed and integrated into a unified research space.
🤖 AI-Powered Summaries Specialized CrewAI agents work together to create comprehensive research summaries. The Research Analyst synthesizes information from all sources, while the Content Writer crafts engaging, well-structured analyses that highlight key insights, trends, and connections across the research material.
💬 Interactive Q&A Using vector embeddings and semantic search through Qdrant database, users can ask natural language questions about their research content. The system provides contextual answers by retrieving relevant information from all processed sources, enabling deep exploration of the research material.
🎧 Audio Overviews On-demand feature that transforms research into engaging podcast-style audio content. The Podcast Script Generator agent creates conversational scripts, which are then converted to high-quality audio using LemonFox TTS with multiple AI voices, making research accessible in audio format.
❓ Smart FAQ Generation AI agents automatically analyze research content to generate relevant, insightful questions and comprehensive answers. This feature helps users understand key aspects of their research topic and provides quick access to important information.
🧠 Visual Mindmaps The Mindmap Creator agent analyzes research structure to generate interactive, hierarchical visualizations with up to 5 levels of depth. Built with react-mindmap-visualiser, these mindmaps help users understand complex topics at a glance and navigate research relationships visually.
🌐 Global Web Access Powered by Bright Data's MCP Server, DecipherIt bypasses geo-restrictions and bot detection to access content from anywhere in the world. This ensures comprehensive research coverage and access to diverse, authoritative sources that traditional scraping methods cannot reach.
Clone the repository
git clone https://github.com/mtwn105/decipher-research-agent.git
cd decipher-research-agent
Set up the frontend
cd client
pnpm install
cp .env.example .env.local
# Configure your environment variables
pnpm prisma generate
pnpm prisma migrate dev
Set up the backend
cd ../backend
uv venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv sync
cp .env.example .env
# Configure your environment variables
Start the development servers
# Terminal 1 - Frontend
cd client && pnpm dev
# Terminal 2 - Backend
cd backend && uv run uvicorn server:app --host 0.0.0.0 --port 8001
DecipherIt uses the official Bright Data MCP Server for advanced web scraping capabilities. Follow these steps to set it up:
Official MCP Server Integration:
from mcp import StdioServerParameters
from crewai_tools import MCPServerAdapter
server_params = StdioServerParameters(
command="pnpm",
args=["dlx", "@brightdata/mcp"],
env={
"API_TOKEN": os.environ["BRIGHT_DATA_API_TOKEN"],
"BROWSER_AUTH": os.environ["BRIGHT_DATA_BROWSER_AUTH"]
},
)
Multi-Agent Workflow:
# Execute multiple scraping tasks in parallel
web_scraping_tasks = []
for link in links:
web_scraping_tasks.append(
web_scraping_crew.kickoff_async(inputs={
"url": link.url,
"current_time": current_time,
})
)
web_scraping_results = await asyncio.gather(*web_scraping_tasks)
⚠️ Important: Always treat scraped web content as untrusted data. DecipherIt automatically:
DATABASE_URL="postgresql://username:password@localhost:5432/decipher"
BETTER_AUTH_SECRET="your-auth-secret"
BETTER_AUTH_URL="http://localhost:3000"
R2_ENDPOINT="https://your-account-id.r2.cloudflarestorage.com"
R2_ACCESS_KEY_ID="your-r2-access-key"
R2_SECRET_ACCESS_KEY="your-r2-secret-key"
R2_BUCKET_NAME="decipher-files"
R2_PUBLIC_URL="https://files.decipherit.xyz"
NEXT_PUBLIC_BASE_URL="http://localhost:3000"
# Database
DATABASE_URL="postgresql://username:password@localhost:5432/decipher"
# AI Services
OPENAI_API_KEY="your-openai-api-key"
OPENROUTER_API_KEY="your-openrouter-api-key"
LEMONFOX_API_KEY="your-lemonfox-api-key"
# Bright Data MCP Server
BRIGHT_DATA_API_TOKEN="your-bright-data-token"
BRIGHT_DATA_WEB_UNLOCKER_ZONE="your-web-unlocker-zone" # Optional: custom zone name
# Vector Database
QDRANT_API_URL="http://localhost:6333"
QDRANT_API_KEY="your-qdrant-api-key"
# Cloud Storage
CLOUDFLARE_ACCOUNT_ID="your-cloudflare-account-id"
CLOUDFLARE_R2_ACCESS_KEY_ID="your-r2-access-key"
CLOUDFLARE_R2_SECRET_ACCESS_KEY="your-r2-secret-key"
DecipherIt employs specialized AI agents powered by CrewAI to handle different aspects of research:
DecipherIt employs a sophisticated multi-crew architecture powered by CrewAI:
search_engine
to find relevant sources based on research topicsscrape_as_markdown
to extract clean, structured content from discovered URLsDecipherIt leverages the official Bright Data MCP Server - a powerful Model Context Protocol server that provides an all-in-one solution for public web access. This integration enables:
DecipherIt leverages two key tools from the Bright Data MCP server:
search_engine
- Search the web for relevant information and discover sourcesscrape_as_markdown
- Extract and convert web content to clean, structured Markdown formatThis powerful integration allows DecipherIt's AI agents to conduct comprehensive research across the entire web without the typical limitations of traditional scraping methods.
Using Qdrant vector database and OpenAI embeddings:
Transform your research into engaging audio content:
Transform complex research into visual hierarchical structures:
Immediate Processing:
On-Demand Generation:
Comprehensive document support using MarkItDown:
version: "3.8"
services:
frontend:
build: ./client
ports:
- "3000:3000"
environment:
- DATABASE_URL=postgresql://postgres:password@db:5432/decipher
depends_on:
- db
- backend
backend:
build: ./backend
ports:
- "8001:8001"
environment:
- DATABASE_URL=postgresql://postgres:password@db:5432/decipher
depends_on:
- db
- qdrant
db:
image: postgres:15
environment:
- POSTGRES_DB=decipher
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
volumes:
- postgres_data:/var/lib/postgresql/data
qdrant:
image: qdrant/qdrant:latest
ports:
- "6333:6333"
volumes:
- qdrant_data:/qdrant/storage
volumes:
postgres_data:
qdrant_data:
docker-compose up -d
We welcome contributions! Please see our Contributing Guide for details.
git checkout -b feature/amazing-feature
)git commit -m 'Add amazing feature'
)git push origin feature/amazing-feature
)This project is licensed under the MIT License - see the LICENSE file for details.
Amit Wani
⭐ Star this repository if you find it helpful!
Made with ❤️ by Amit Wani
{ "mcpServers": { "decipher-research-agent": { "command": "pnpm", "args": [ "dlx", "@brightdata/mcp" ] } } }
Related projects feature coming soon
Will recommend related projects based on sub-categories