Call Fusion

Brokerages pay real money for leads and then lose most of them, because following up means a person sitting there dialing a list. If the call doesn’t happen in the first hour it’s usually dead, and if it doesn’t end with something in the calendar the money’s gone anyway.

By early 2023 all the pieces to do that call in software existed separately. Speech recognition you could stream, a model that could hold a thread, text to speech that didn’t sound like a robot. What nobody had done was put them on a live phone line and make it fast enough that the person on the other end wouldn’t notice.

It came down to two services. One was the real-time voice engine, which took live audio off a Twilio call, transcribed it, worked out a reply against whatever the client had put in its knowledge base, and streamed speech back. The other was the boring half nobody sees: dialing on a schedule, retries, voicemails, SMS, CRM sync, billing, recordings.

All of it came down to latency. Three seconds of silence on a phone call and people assume the line dropped. Getting under two seconds end to end meant letting the parts of a call share memory instead of passing messages between themselves, streaming every stage so the agent was already saying the first sentence while the model wrote the second, and caching retrieval so it stopped recomputing the same thing every turn.

Turn-taking was the other hard one. Early versions either talked over people or froze completely when someone said “yeah”. What fixed it was tracking who was actually speaking, throwing away the reply in flight the moment someone genuinely interrupted, and learning to ignore the five or six words people say just to show they’re still listening.

Every client got their own agent out of configuration, no new code. They could edit the prompts, pick the voice, keep the knowledge base in a spreadsheet, and choose what the agent was allowed to do mid-call: book a meeting, send a text, hand off to a person, update the CRM. Twelve models across five providers sat behind one routing layer, so the cheap ones did classification and the good ones did the talking.

Three co-founders, equal split. I owned engineering. We applied to UofT Hatchery as first-years and got in from a pool of more than 140 teams, most of them graduate students, then finished in the top five of its $105K competition. After that came NextAI’s bootcamp, the RippleX founder fellowship, and about a year in ventureLAB’s accelerator. We got offered investment and turned it down, the company ran on revenue instead.

Clients were mostly mortgage and real-estate brokerages, with recruiting firms, a cannabis retailer, a college, and a political campaign in the mix. At peak the platform was running thousands of calls a week.

The company was acquired in September 2025. I negotiated my own package and stayed on as Lead Engineer for Voice AI, which mostly meant rebuilding the pipeline so it could survive real production traffic. First audio came back in about 800 ms, the opening line was generated ahead of time so a call started talking almost immediately, and there was an agent builder good enough that someone who couldn’t code could set one up in a browser. I also added auth on every endpoint, a Canadian-French agent, and a dashboard for call analytics. It went out to 8,000+ mortgage brokers through a national broker network. I handed it off in July 2026 with a written handoff doc and nothing left open.

Keeping the voice engine as one service was right and I’d do it again. Everyone told us to split it up, but the latency budget didn’t allow for it, so I wrote that reasoning into the README to stop someone undoing it later out of habit.

Selling to brokerages taught me that a demo tells you almost nothing. What matters is call four hundred, at 7:50pm, when the lead’s kid picks up instead of the lead. Most of what I worked on after the first few months was for calls like that one.