AI / ML research · 2025
Call Center Agent Evaluation (AUTO KPI)
A multimodal pipeline scoring call-center calls — Whisper transcription + diarization, RoBERTa sentiment, HuBERT tonal emotion, and FAISS RAG over rulebooks with a local Mistral 7B.
- WhisperX
- RoBERTa
- HuBERT
- FAISS
- Mistral 7B
- LangChain
- Gradio
The problem
Objectively evaluate call-center agents at scale — across what is said and how it is said.
My role
Researcher & engineer (MS project).
What I built
An end-to-end multimodal pipeline: Whisper/WhisperX transcription with speaker diarization, RoBERTa sentiment, HuBERT tonal-emotion detection, and a FAISS-backed RAG layer over evaluation rulebooks. A locally quantized Mistral 7B (llama-cpp-python) served via Gradio extracts KPIs — empathy, tone, compliance, problem resolution — with qualitative coaching feedback.
Architecture & stack
- Whisper/WhisperX transcription + speaker diarization
- RoBERTa sentiment analysis + HuBERT tonal-emotion detection
- FAISS RAG over evaluation rulebooks (LangChain)
- Local quantized Mistral 7B (llama-cpp-python) served via Gradio