Sales Voice Agent

Overview

An AI-powered voice agent for sales qualification calls that integrates with CRM systems, handles objections naturally, and books meetings while maintaining conversation context.

Problem

Sales development teams were making hundreds of outbound calls with low connection rates and even lower qualification rates. When they did connect, SDRs spent time on repetitive qualification questions. We needed to:

Handle initial outreach calls at scale
Qualify leads using standardized criteria
Integrate with existing CRM workflows
Book meetings directly on rep calendars

What I Built

Architecture

Loading diagram...

Tool Calling System

The agent can execute actions mid-conversation:

Available Tools:

lookup_contact - Fetch prospect info from CRM
check_calendar - Find available meeting slots
book_meeting - Schedule meeting with prospect
update_crm - Log call outcome and notes
send_email - Send follow-up email

Example Flow:

User: "I'm interested but next week is crazy. Maybe the week after?"

Agent thinks: Need to check calendar for two weeks out
→ Calls check_calendar(start="2024-02-05", end="2024-02-09")
→ Receives: ["2024-02-06 10:00", "2024-02-07 14:00", "2024-02-08 11:00"]

Agent: "I have a few spots the week of February 5th. Would Tuesday at 10am or Wednesday at 2pm work better for you?"

User: "Wednesday works"

Agent thinks: Book the Wednesday slot
→ Calls book_meeting(time="2024-02-07 14:00", duration=30)
→ Receives: {success: true, meeting_id: "mtg_123"}

Agent: "You're all set for Wednesday, February 7th at 2pm. You'll receive a calendar invite shortly..."

Conversation State Machine

Loading diagram...

Reliability & Operations

Tool Call Reliability

Pattern	Implementation
Timeout	5s timeout on all tool calls
Retry	3 attempts with exponential backoff
Circuit Breaker	Open after 5 failures in 1 minute
Fallback	Graceful degradation messaging

Example Failure Handling

typescript

// If CRM lookup fails
"I apologize, I'm having a brief technical issue accessing your 
information. Could you remind me what company you're with?"

// If calendar check fails  
"I'm having trouble checking available times right now. 
Could I have someone reach out to you via email with some options?"

// If booking fails
"The booking didn't go through on my end. Let me make a note 
and our team will send you a confirmation within the hour."

Monitoring

Call metrics: Duration, outcome, qualification rate
Tool metrics: Success rate, latency by tool type
Quality metrics: Sentiment analysis, objection frequency
Cost tracking: Per-call cost breakdown

Logging

Every tool call logged with context:

json

{
  "call_id": "call_abc",
  "event": "tool.call",
  "tool": "book_meeting",
  "args": {"time": "2024-02-07T14:00:00Z", "duration": 30},
  "latency_ms": 850,
  "success": true,
  "result": {"meeting_id": "mtg_123"},
  "conversation_turn": 12
}

Tech Stack

Voice: Twilio, LiveKit
STT: Deepgram
LLM: OpenAI GPT-4o (function calling)
TTS: ElevenLabs
CRM: HubSpot, Salesforce adapters
Calendar: Google Calendar, Outlook
Backend: Node.js, TypeScript
Queue: Redis + Bull
Database: PostgreSQL

What I Owned

Designed the tool calling architecture
Built CRM integration adapters
Implemented conversation state machine
Created reliability patterns for external calls
Set up call quality monitoring

Results

40% of calls result in booked meetings (vs 15% with human SDRs)
Average call duration: 4 minutes
99.2% tool call success rate
SDRs focus on high-value activities