TS

realtime-to-mcp

by Gillinghammer/realtime-to-mcp

0 views

No description available

typescriptopenaiAPI Integration

๐ŸŽ™๏ธ Realtime MCP Proxy

Add voice + tools to OpenAI's Realtime API in 3 lines of code.

A TypeScript library that bridges OpenAI's Realtime API with Model Context Protocol (MCP) servers, enabling voice-driven tool execution.

โœจ Features

  • ๐ŸŽฏ Ultra-Simple API - Just add your MCP config to get voice + tools working
  • ๐Ÿ”Œ WebRTC Ready - Built-in server for browser voice applications
  • ๐Ÿ› ๏ธ Universal MCP Support - Works with any MCP server (HubSpot, GitHub, custom, etc.)
  • ๐Ÿ”’ Type Safe - Full TypeScript support with comprehensive type definitions
  • โšก Production Ready - Error handling, timeouts, graceful shutdown, health checks

๐Ÿš€ Quick Start

Installation

npm install @gillinghammer/realtime-mcp-core

Basic Usage

import { WebRTCBridgeServer } from '@gillinghammer/realtime-mcp-core';

const bridge = new WebRTCBridgeServer({
  openai: {
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-4o-realtime-preview-2024-12-17',
    voice: 'alloy',
    instructions: 'You are a helpful assistant with access to external tools.',
  },
  mcp: {
    command: 'npx',
    args: ['-y', '@hubspot/mcp-server'],
    env: {
      PRIVATE_APP_ACCESS_TOKEN: process.env.HUBSPOT_TOKEN!,
    },
  },
});

await bridge.start();
console.log('๐Ÿš€ Voice AI with tools running on http://localhost:8084');

That's it! Your Realtime API now has voice-driven access to all MCP tools.

๐ŸŽฌ What You Get

The bridge provides everything needed for voice + tools:

  • GET /session - Ephemeral API keys for WebRTC connections
  • POST /mcp - MCP proxy for tool calls
  • GET /tools - OpenAI-formatted tool definitions
  • GET /demo - Live demo page to test voice interactions
  • GET /health - Health check and status

๐ŸŒŸ Examples

HubSpot CRM Integration

const bridge = new WebRTCBridgeServer({
  openai: {
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-4o-realtime-preview-2024-12-17',
    instructions: 'You are a helpful HubSpot CRM assistant...'
  },
  mcp: {
    command: 'npx',
    args: ['-y', '@hubspot/mcp-server'],
    env: { PRIVATE_APP_ACCESS_TOKEN: process.env.HUBSPOT_TOKEN! }
  }
});

Voice commands: "Show me recent contacts" โ€ข "Search for companies with 'tech'" โ€ข "Add a note to John Smith"

Hacker News Integration

const bridge = new WebRTCBridgeServer({
  openai: {
    apiKey: process.env.OPENAI_API_KEY!,
    model: 'gpt-4o-realtime-preview-2024-12-17',
    instructions: 'You are a tech news assistant...'
  },
  mcp: {
    command: 'uvx',
    args: ['mcp-hn']
  }
});

Voice commands: "What are the top stories on Hacker News?" โ€ข "Find articles about AI" โ€ข "Show trending tech discussions"

Custom MCP Server

const bridge = new WebRTCBridgeServer({
  openai: { /* ... */ },
  mcp: {
    url: 'http://localhost:3000',  // Your custom MCP server
    auth: {
      type: 'bearer',
      token: process.env.CUSTOM_TOKEN
    }
  }
});

๐ŸŒ Browser Integration

Use the bridge from any web application:

<button onclick="startVoiceChat()">Start Voice Chat</button>
<script>
async function startVoiceChat() {
  // Get ephemeral API key from your bridge server
  const session = await fetch('http://localhost:8084/session').then(r => r.json());
  
  // Set up WebRTC connection to OpenAI
  const pc = new RTCPeerConnection();
  const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
  pc.addTrack(stream.getTracks()[0]);
  
  // Connect to OpenAI Realtime API
  const offer = await pc.createOffer();
  await pc.setLocalDescription(offer);
  
  const response = await fetch(
    `https://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-12-17`,
    {
      method: "POST",
      body: offer.sdp,
      headers: {
        Authorization: `Bearer ${session.client_secret.value}`,
        "Content-Type": "application/sdp"
      }
    }
  );
  
  const answerSdp = await response.text();
  await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
  
  // Now you can talk to AI and it has access to your MCP tools!
}
</script>

๐Ÿ“ Examples

๐ŸŽ™๏ธ Unified Voice Demo

One interface for all MCP providers - Switch between HubSpot, Hacker News, Airbnb, and custom servers with a single click.

Features:

  • ๐Ÿข HubSpot CRM - "Show me recent contacts" โ€ข "Search for companies"
  • ๐Ÿ“ฐ Hacker News - "What's trending in tech?" โ€ข "Find AI articles"
  • ๐Ÿ  Airbnb Search - "Find places in Tokyo" โ€ข "Search vacation rentals"
  • โž• Easy to extend with any MCP server

Setup:

cd examples/voice-demo
cp .env.example .env  # Add your API keys
npm install && npm run dev
# Open http://localhost:8085

๐Ÿ”ง Configuration

interface WebRTCBridgeConfig {
  openai: {
    apiKey: string;              // OpenAI API key
    model: string;               // Model (e.g., 'gpt-4o-realtime-preview-2024-12-17')
    voice?: string;              // Voice (alloy, echo, sage, etc.)
    instructions?: string;       // System instructions
  };
  mcp: {
    // Option 1: Start MCP server automatically
    command?: string;            // Command (e.g., 'npx', 'uvx')
    args?: string[];             // Arguments (e.g., ['-y', '@hubspot/mcp-server'])
    env?: Record<string, string>; // Environment variables
    timeout?: number;            // Request timeout (default: 10000ms)
    
    // Option 2: Connect to existing MCP server
    url?: string;                // MCP server URL
    auth?: {                     // Authentication
      type: 'bearer';
      token: string;
    };
  };
  server?: {
    port?: number;               // Server port (default: 8084)
    host?: string;               // Server host (default: 'localhost')  
    cors?: boolean;              // Enable CORS (default: true)
  };
}

๐Ÿค Contributing

We welcome contributions! Please see our Contributing Guide.

๐Ÿ“„ License

MIT License - see LICENSE for details.


Made with โค๏ธ for the voice AI community

Install

{
  "mcpServers": {
    "realtime-to-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@hubspot/mcp-server"
      ]
    }
  }
}
For more configuration details, refer to the content on the left

Related

Related projects feature coming soon

Will recommend related projects based on sub-categories