Few companies budgeted for major AI spend this year, yet in engineering, AI is already reaching 8% to 10% of labor costs. The pressure to cut costs is real.
Open weight models from Chinese labs may solve part of the problem. They now trail the U.S. frontier by only three to six months, and many cost about 90% less, as Larridin CTO Ameya Kanitkar said in his interview with Techstrong TV.
But in Larridin's data, they account for just 10% to 15% of enterprise AI usage. What will it take for them to go further?
Mike Vizard of Techstrong TV asked Larridin co-founder and CTO Ameya Kanitkar for specifics.
Barriers to Success
Kanitkar laid out three barriers to success in cutting costs using the new models:
- Trust. Using a "Chinese model" usually means running an open weight model on American servers, through inference providers such as Fireworks or large cloud providers. This is a big step toward settling security questions, but open questions remain.
- Overpromising. A two-point gap in performance on a leaderboard can be much wider in real work. In one Larridin test, a task that cost $30 on a frontier model should have cost about $3 on a cheaper GLM model. It actually cost $10 to $15. Kanitkar's action item: Run your own evaluations.
- The harness. Engineers work through tools such as Cursor, Claude Code or Codex. Most companies lack the infrastructure to change the model underneath those tools without disrupting how people work, so they keep using the foundation models.
However, efforts in all of these areas need to be guided by an agentic focus. In Larridin's customer data, agentic workloads account for more than 90% of token costs, yet come from roughly 5% of users. So cheaper models must be used by agents to make a real difference.
How to Fix It
Kanitkar's top recommendation is an intelligent AI router between your people, your agents, and the models.
Teams keep the tools they've mastered, while the router controls costs and swaps in better or cheaper models behind the scenes. This allows employees to focus on their work.
Behind the scenes, options are also widening. Mistral, NVIDIA and Thinking Machines are all pushing Western open weight models, which Kanitkar calls a Goldilocks zone: cheaper, with less political risk, and good enough for most tasks.
He also pointed to Coinbase, which he said post-trained a GLM model on its own data for fraud detection. Kanitkar, who worked at Coinbase, noted it has the strictest safety constraints he's seen. Vetting still takes real effort, though, and he expects many enterprises to rely on third parties to certify open weight models.
Conclusion
Cheaper models only pay off if you can see where your AI spend goes and switch models without disrupting your teams. See the full interview on Techstrong TV. To get a demo of Larridin, contact us.
Quotes
- "We have more choice than ever, and arguably the models might be becoming more of a commodity, but it takes time, skills, labor, and cost to implement them." (Mike Vizard)
- "These models are good, but they're not necessarily as good as the benchmarks say. So you kind of have to take it with a grain of salt." (Ameya Kanitkar)
- "Not every task needs a frontier model." (Ameya Kanitkar)
- "Like 5% are agentic and 95% are the rest, but those 5% probably take up 90% of the costs." (Ameya Kanitkar)
- "You need to be ready for the world where models are going to change. And most enterprises are not ready for it." (Ameya Kanitkar)