Artificial intelligence is all the rage, especially text generation artificial intelligence, also known as large language models (models similar to ChatGPT). In a recent survey of approximately 1,000 enterprise organizations, 67.2% said they believe the adoption of large language models (LLMs) will be a top priority by early 2024.
But obstacles remain. According to the same survey, a lack of customization and flexibility, coupled with an inability to protect company knowledge and intellectual property rights, was and still is preventing many businesses from deploying LL.M.s into production.
This got Varun Vummadi and Esha Manideep Dinne thinking: What would solutions to enterprise LLM adoption challenges look like? To find one, they founded Giga ML, a startup that built a platform that allows companies to deploy LL.M.s on-premises — ostensibly reducing costs while preserving privacy.
“Data privacy and customized LL.M.s are the biggest challenges companies face when adopting LL.M.s to solve problems,” Vummadi told TechCrunch in an email interview. “Giga ML solves both of these challenges.”
Giga ML offers its own set of LLMs (the “X1 Series”) for tasks such as generating code and answering common customer questions (such as “When will my order arrive?”). The startup claims that these models built on Meta’s Llama 2 outperform the popular LLM on some benchmarks, particularly the conversational MT-Bench test set. But it’s hard to say what the quality of X1 is; reporters tried the online demonstration of Giga ML but encountered technical problems. (No matter what prompt I enter, the application times out.)
Even the model of Giga ML yes While excellent in some respects, can they really make a splash in a sea of open source, offline LL.M.s?
While talking to Vummadi, I got the sense that Giga ML is not trying to create the best performing LLM, but to build tools to allow businesses to fine-tune their LLM locally without having to rely on third-party resources and platforms.
“Giga ML’s mission is to help enterprises deploy LLM securely and efficiently on their own on-premises infrastructure or virtual private cloud,” said Vummadi. “Giga ML is handled through an easy-to-use API, simplifying the process of training, fine-tuning and running LLM, eliminating any associated hassle.”
Vummadi highlighted the privacy benefits of the offline mode of operation—advantages that may be persuasive to some businesses.
Predibase, a low-code AI development platform, found that less than a quarter of businesses are willing to use a business LLM because of concerns about sharing sensitive or proprietary data with suppliers. Nearly 77% of survey respondents said they do not use or do not plan to use a business LL.M. other than to produce prototypes, citing issues such as privacy, cost and lack of customization.
“IT managers at the C-suite level find Giga ML’s offering valuable because of LLM’s secure on-premises deployment, customizable models tailored to their specific use cases, and fast inference to ensure data compliance and the highest efficiency.” Vumadi said.
Giga ML has raised approximately $3.74 million in venture capital to date from Nexus Venture Partners, Y Combinator, Liquid 2 Ventures, 8vdx and several other companies, and plans to expand its two-person team and strengthen product development in the short term. A portion of the capital will also be used to support Giga ML’s customer base, That currently includes unnamed “enterprise” companies in finance and health care, Vummadi said.
