This AI startup raises $45 million to make sure AI models don’t hallucinate or leak data

Thanks to the popularity of ChatPGT, companies everywhere are injecting generative AI into their products. But Vikram Chatterji believes there’s one big thing holding back companies from fully embracing AI and implementing tools in production at scale: companies don’t really know how well the tools will work once they’re released.

“If you don’t solve this, you’re stuck in near-production terrain,” said Chatterji, CEO of Galileo, a startup focused on evaluating AI models.

Chatterji wants his company to be the one to fix it. Galileo works with customers like Hewlett Packard, Comcast and Twilio to test its AI tools, making sure they aren’t mind-bending (talking AI to make up answers) or don’t provide private data. Tuesday, the company said Forbes has raised a $45 million Series B funding round, bringing its total investment to $68 million since it was founded three years ago.

The startup has four products: Fine Tune, which helps corporate customers train existing models to fit their needs, such as incorporating proprietary data; Evaluate, which allows companies to quickly decide whether a model is ready to ship or still has issues to resolve; Note, for real-time maintenance and testing once the model is released; and Protect, to mitigate threats like rapid injection, where users try to manipulate the model to go off the rails or reveal sensitive information. In July, Galileo released Luna, its own language model for evaluating the performance of other models, including response integrity, data privacy, and bias detection.

The announcement comes as companies around the world race to bolster their AI offerings. But that comes with the drawbacks of AI models making stupid mistakes, like Google’s Gemini telling users to make a cola pizza, or sensitive leaks, like Samsung’s internal source code shared on ChatGPT last year past, which led to the electronics giant banning employees from using the chatbot. Galileo is not the only AI assessment startup on the market. Last week, rival Braintrust, whose clients include Stripe and Notion, said it raised a $36 million Series A led by Andreessen Horowitz.

“In the long run, there will be an expectation that, for example, Siri will not say bad things about Apple.”

Chatterji and co-founder Yash Sheth are former Google colleagues who joined the tech giant a week apart in 2013. Chatterji was an AI product manager who most recently worked on Google’s BERT, a model of AI implemented to help understand a user’s intentions while using. your search engine. Sheth, now Galileo’s COO, was an engineer working on Google’s speech recognition tools. Atindriyo Sanyal, CTO of Galileo and another co-founder, previously held AI engineering roles at Uber and Apple.

Galileo’s round was led by Scale Venture Partners, with participation from Premji Invest, an Indian venture capital firm. Other backers include Citibank and data warehousing firm Databricks, as well as Clem Delangue, founder of AI startup Hugging Face.

Scale Venture Partners partner Andy Vitus said assessment tools are so important to corporations because he believes AI models will soon be seen as brand ambassadors, especially as companies begin to deploy agents of AIs that perform tasks for customers, such as booking rides or placing them. orders Companies also need to protect their AI models from saying anything negative about their brands or referencing past scandals.

“In the long run, there will be an expectation that, for example, Siri will not say bad things about Apple,” said Vitus, who will take a seat on Galileo’s board of directors as part of the investment. “Just as employees have managers, these agents need supervision.”

Jim Nottingham, HP’s senior vice president of applied computing solutions, said the company uses Galileo services internally for its own AI tools, but also bundles them together for customers using AI Studio HP, a suite of subscription tools that help developers build AI products.

The $45 million funding round is modest compared to some of the amazing rounds other AI companies have announced recently. Earlier this month, OpenAI said it raised $6.6 billion, the largest venture capital round ever. Elon Musk’s xAI isn’t far behind with a $6 billion raise earlier this year. Even lesser-known companies, like next-generation chip startup Groq, raised $640 million this summer.

But Galileo isn’t training massive frontier models that need armies of GPUs (the company didn’t train or fine-tune Luna from scratch), nor is it building new chip designs with manufacturing costs. As a result, Chatterji said the company doesn’t need nearly as much cash to operate. “You should only take the amount of money you really need,” he said. “And build a business that is sustainable in the long term.”

MORE FORBES

ForbesNotion And Stripe relies on this year-old startup to make sure its AI tools workFor Alex KonradForbesAI/R group, which helps companies deploy AI tools, plans 2025 IPOFor Ricard NievaForbesThe billionaire founder of Duolingo is all into AIFor Ricard Nieva