AI Insights for Meetings

📤 Producer (Generated)

0 segments | 00:00

📝

Click "Simulate Meeting" to start

🤖 AI Notes

Disconnected | 0 notes

🤖

Waiting for AI-generated notes...

Notes will appear as segments are processed

PoC Architecture

🔄 Data Flow

┌──────────┐     ┌─────────┐     ┌───────────────┐     ┌────────────┐     ┌──────────┐
│    UI    │────▶│ PubSub  │────▶│  Flink Job    │────▶│ Centrifugo │────▶│    UI    │
│ (Browser)│     │ (Ingest)│     │  (Stateful)   │     │ (WebSocket)│     │  (Notes) │
└──────────┘     └─────────┘     └───────┬───────┘     └────────────┘     └──────────┘
                                         │
                                         ▼
                                  ┌─────────────┐
                                  │   Ollama    │
                                  │ (qwen2:0.5b)│
                                  └─────────────┘

Transcript → PubSub (ingest) → Flink (stateful) → LLM → Centrifugo (WebSocket) → UI

🧩 Components

All services deployed in Kubernetes (GKE) via Helm charts.

🖥️ UI + Backend

Node.js/TypeScript

Generates mock transcripts, publishes to PubSub
Receives AI notes via Centrifugo WebSocket

📬 PubSub

Google Cloud Pub/Sub

Broadcasts to consumers at scale
Fully managed, scalable. Kafka also works but needs more Ops

⚡ Flink

Stateful streaming engine, self-hosted in Kubernetes

Keyed by meeting_id, stores recent chunks in state
Processes immediately (no windowing = lower latency)
Tried Dataflow — felt higher latency

🤖 Ollama

Flask wrapper + qwen2:0.5b (352MB)

Small LLM: fast (~500ms), free, good for POC
Wrapper allows prompt changes without redeploying Flink

📡 Centrifugo

Real-time messaging server

WebSocket delivery, per-meeting channels
Fast (~100ms) vs polling. Scales to many clients

⏱️ Latency Breakdown (observations)

Ingest (UI → PubSub) ~100ms

Queue (PubSub → Flink) ~100-200ms

                LLM inference
                
                ~500-1000ms

Return (Centrifugo → UI) ~100-200ms

Total E2E ~800-1500ms

⚠️ Approximate numbers. Small nodes used for PoC.

📋 TODO / Future Work

Pending

Real-time delivery via Centrifugo - test scaling

WebSocket-based delivery with per-meeting channels. Need to test at scale.

Pending

Better note quality, robust self-hosted SLM service

Current notes are basic headlines. Need smarter summarization.

Pending

Load test (at least hundreds of meetings in parallel)

Verify scalability with concurrent meetings.

Future

Action item detection → Jira

Detect "I will do X" and auto-create tickets.

Future

Poll/vote suggestions

Detect disagreement, suggest creating a poll.

Future

GPU for larger model

Use gemma2:2b or phi3:mini for better quality.

🛠️ Tech Stack

Kubernetes (GKE) Apache Flink Google Pub/Sub Centrifugo Ollama Java 17 TypeScript Python/Flask Helm Charts