16 Changes to the Way Enterprises Are Building and Buying Generative AI – Andreessen Horowitz
Generative AI took the consumer landscape by storm in 2023, reaching over a billion dollars of consumer spend1 in record time. In 2024, we believe the revenue opportunity will be multiples larger in the enterprise.
Last year, while consumers spent hours chatting with new AI companions or making images and videos with diffusion models, most enterprise engagement with genAI seemed limited to a handful of obvious use cases and shipping GPT-wrapper products as new SKUs. Some naysayers doubted that genAI could scale into the enterprise at all. Arent we stuck with the same 3 use cases? Can these startups actually make any money? Isnt this all hype?
Over the past couple months, weve spoken with dozens of Fortune 500 and top enterprise leaders,2 and surveyed 70 more, to understand how theyre using, buying, and budgeting for generative AI. We were shocked by how significantly the resourcing and attitudes toward genAI had changed over the last 6 months. Though these leaders still have some reservations about deploying generative AI, theyre also nearly tripling their budgets, expanding the number of use cases that are deployed on smaller open-source models, and transitioning more workloads from early experimentation into production.
This is a massive opportunity for founders. We believe that AI startups who 1) build for enterprises AI-centric strategic initiatives while anticipating their pain points, and 2) move from a services-heavy approach to building scalable products will capture this new wave of investment and carve out significant market share.
As always, building and selling any product for the enterprise requires a deep understanding of customers budgets, concerns, and roadmaps. To clue founders into how enterprise leaders are making decisions about deploying generative AIand to give AI executives a handle on how other leaders in the space are approaching the same problems they haveweve outlined 16 top-of-mind considerations about resourcing, models, and use cases from our recent conversations with those leaders below.
In 2023, the average spend across foundation model APIs, self-hosting, and fine-tuning models was $7M across the dozens of companies we spoke to. Moreover, nearly every single enterprise we spoke with saw promising early results of genAI experiments and planned to increase their spend anywhere from 2x to 5x in 2024 to support deploying more workloads to production.
Last year, much of enterprise genAI spend unsurprisingly came from innovation budgets and other typically one-time pools of funding. In 2024, however, many leaders are reallocating that spend to more permanent software line items; fewer than a quarter reported that genAI spend will come from innovation budgets this year. On a much smaller scale, weve also started to see some leaders deploying their genAI budget against headcount savings, particularly in customer service. We see this as a harbinger of significantly higher future spend on genAI if the trend continues. One company cited saving ~$6 for each call served by their LLM-powered customer servicefor a total of ~90% cost savingsas a reason to increase their investment in genAI eightfold. Heres the overall breakdown of how orgs are allocating their LLM spend:
Enterprise leaders are currently mostly measuring ROI by increased productivity generated by AI. While they are relying on NPS and customer satisfaction as good proxy metrics, theyre also looking for more tangible ways to measure returns, such as revenue generation, savings, efficiency, and accuracy gains, depending on their use case. In the near term, leaders are still rolling out this tech and figuring out the best metrics to use to quantify returns, but over the next 2 to 3 years ROI will be increasingly important. While leaders are figuring out the answer to this question, many are taking it on faith when their employees say theyre making better use of their time.
Simply having an API to a model provider isnt enough to build and deploy generative AI solutions at scale. It takes highly specialized talent to implement, maintain, and scale the requisite computing infrastructure. Implementation alone accounted for one of the biggest areas of AI spend in 2023 and was, in some cases, the largest. One executive mentioned that LLMs are probably a quarter of the cost of building use cases, with development costs accounting for the majority of the budget. In order to help enterprises get up and running on their models, foundation model providers offered and are still providing professional services, typically related to custom model development. We estimate that this made up a sizable portion of revenue for these companies in 2023 and, in addition to performance, is one of the key reasons enterprises selected certain model providers. Because its so difficult to get the right genAI talent in the enterprise, startups who offer tooling to make it easier to bring genAI development in house will likely see faster adoption.
Just over 6 months ago, the vast majority of enterprises were experimenting with 1 model (usually OpenAIs) or 2 at most. When we talked to enterprise leaders today, theyre are all testingand in some cases, even using in productionmultiple models, which allows them to 1) tailor to use cases based on performance, size, and cost, 2) avoid lock-in, and 3) quickly tap into advancements in a rapidly moving field. This third point was especially important to leaders, since the model leaderboard is dynamic and companies are excited to incorporate both current state-of-the-art models and open-source models to get the best results.
Well likely see even more models proliferate. In the table below drawn from survey data, enterprise leaders reported a number of models in testing, which is a leading indicator of the models that will be used to push workloads to production. For production use cases, OpenAI still has dominant market share, as expected.
This is one of the most surprising changes in the landscape over the past 6 months. We estimate the market share in 2023 was 80%90% closed source, with the majority of share going to OpenAI. However, 46% of survey respondents mentioned that they prefer or strongly prefer open source models going into 2024. In interviews, nearly 60% of AI leaders noted that they were interested in increasing open source usage or switching when fine-tuned open source models roughly matched performance of closed-source models. In 2024 and onwards, then, enterprises expect a significant shift of usage towards open source, with some expressly targeting a 50/50 splitup from the 80% closed/20% open split in 2023.
Control (security of proprietary data and understanding why models produce certain outputs) and customization (ability to effectively fine-tune for a given use case) far outweighed cost as the primary reasons to adopt open source. We were surprised that cost wasnt top of mind, but it reflects the leaderships current conviction that the excess value created by generative AI will likely far outweigh its price. As one executive explained: getting an accurate answer is worth the money.
Enterprises still arent comfortable sharing their proprietary data with closed-source model providers out of regulatory or data security concernsand unsurprisingly, companies whose IP is central to their business model are especially conservative. While some leaders addressed this concern by hosting open source models themselves, others noted that they were prioritizing models with virtual private cloud (VPC) integrations.
In 2023, there was a lot of discussion around building custom models like BloombergGPT. In 2024, enterprises are still interested in customizing models, but with the rise of high-quality open source models, most are opting not to train their own LLM from scratch and instead use retrieval-augmented generation (RAG) or fine-tune an open source model for their specific needs.
In 2023, many enterprises bought models through their existing cloud service provider (CSP) for security reasonsleaders were more concerned about closed-source models mishandling their data than their CSPsand to avoid lengthy procurement processes. This is still the case in 2024, which means that the correlation between CSP and preferred model is fairly high: Azure users generally preferred OpenAI, while Amazon users preferred Anthropic or Cohere. As we can see in the chart below, of the 72% of enterprises who use an API to access their model, over half used the model hosted by their CSP. (Note that over a quarter of respondents did self-host, likely in order to run open source models.)
While leaders cited reasoning capability, reliability, and ease of access (e.g., on their CSP) as the top reasons for adopting a given model, leaders also gravitated toward models with other differentiated features. Multiple leaders cited the prior 200K context window as a key reason for adopting Anthropic, for instance, while others adopted Cohere because of their early-to-market, easy-to-use fine-tuning offering.
While large swathes of the tech community focus on comparing model performance to public benchmarks, enterprise leaders are more focused on comparing the performance of fine-tuned open-source models and fine-tuned closed-source models against their own internal sets of benchmarks. Interestingly, despite closed-source models typically performing better on external benchmarking tests, enterprise leaders still gave open-source models relatively high NPS (and in some cases higher) because theyre easier to fine-tune to specific use cases. One company found that after fine-tuning, Mistral and Llama perform almost as well as OpenAI but at much lower cost. By these standards, model performance is converging even more quickly than we anticipated, which gives leaders a broader range of very capable models to choose from.
Most enterprises are designing their applications so that switching between models requires little more than an API change. Some companies are even pre-testing prompts so the change happens literally at the flick of a switch, while others have built model gardens from which they can deploy models to different apps as needed. Companies are taking this approach in part because theyve learned some hard lessons from the cloud era about the need to reduce dependency on providers, and in part because the market is evolving at such a fast clip that it feels unwise to commit to a single vendor.
Enterprises are overwhelmingly focused on building applications in house, citing the lack of battle-tested, category-killing enterprise AI applications as one of the drivers. After all, there arent Magic Quadrants for apps like this (yet!). The foundation models have also made it easier than ever for enterprises to build their own AI apps by offering APIs. Enterprises are now building their own versions of familiar use casessuch as customer support and internal chatbotswhile also experimenting with more novel use cases, like writing CPG recipes, narrowing the field for molecule discovery, and making sales recommendations. Much has been written about the limited differentiation of GPT wrappers, or startups building a familiar interface (e.g., chatbot) for a well-known output of an LLM (e.g., summarizing documents); one reason we believe these will struggle is that AI further reduced the barrier to building similar applications in-house. However, the jury is still out on whether this will shift when more enterprise-focused AI apps come to market. While one leader noted that though they were building many use cases in house, theyre optimistic there will be new tools coming up and would prefer to use the best out there. Others believe that genAI is an increasingly strategic tool that allows companies to bring certain functionalities in-house instead of relying as they traditionally have on external vendors. Given these dynamics, we believe that the apps that innovate beyond the LLM + UI formula and significantly rethink the underlying workflows of enterprises or help enterprises better use their own proprietary data stand to perform especially well in this market.
Thats because 2 primary concerns about genAI still loom large in the enterprise: 1) potential issues with hallucination and safety, and 2) public relations issues with deploying genAI, particularly into sensitive consumer sectors (e.g., healthcare and financial services). The most popular use cases of the past year were either focused on internal productivity or routed through a human before getting to a customerlike coding copilots, customer support, and marketing. As we can see in the chart below, these use cases are still dominating in the enterprise in 2024, with enterprises pushing totally internal use cases like text summarization and knowledge management (e.g., internal chatbot) to production at far higher rates than sensitive human-in-the-loop use cases like contract review, or customer-facing use cases like external chatbots or recommendation algorithms. Companies are keen to avoid the fallout from generative AI mishaps like the Air Canada customer service debacle. Because these concerns still loom large for most enterprises, startups who build tooling that can help control for these issues could see significant adoption.
By our calculations, we estimate that the model API (including fine-tuning) market ended 2023 around $1.52B run-rate revenue, including spend on OpenAI models via Azure. Given the anticipated growth in the overall market and concrete indications from enterprises, spend on this area alone will grow to at least $5B run-rate by year end, with significant upside potential. As weve discussed, enterprises have prioritized genAI deployment, increased budgets and reallocated them to standard software lines, optimized use cases across different models, and plan to push even more workloads to production in 2024, which means theyll likely drive a significant chunk of this growth.
Over the past 6 months, enterprises have issued a top-down mandate to find and deploy genAI solutions. Deals that used to take over a year to close are being pushed through in 2 or 3 months, and those deals are much bigger than theyve been in the past. While this post focuses on the foundation model layer, we also believe this opportunity in the enterprise extends to other parts of the stackfrom tooling that helps with fine-tuning, to model serving, to application building, and to purpose-built AI native applications. Were at an inflection point in genAI in the enterprise, and were excited to partner with the next generation of companies serving this dynamic and growing market.
Link:
16 Changes to the Way Enterprises Are Building and Buying Generative AI - Andreessen Horowitz
- Congressman Don Beyer went back to college to learn AI - The Associated Press - April 15th, 2024 [April 15th, 2024]
- Georgia Tech Unveils New AI Makerspace in Collaboration with NVIDIA - Georgia Tech College of Engineering - April 15th, 2024 [April 15th, 2024]
- Energy-Guzzling AI Is Also the Future of Energy Savings - The Wall Street Journal - April 15th, 2024 [April 15th, 2024]
- South Korea to host second AI Safety Summit on May 21-22 - Reuters - April 15th, 2024 [April 15th, 2024]
- What is artificial intelligence (AI)? - Livescience.com - April 15th, 2024 [April 15th, 2024]
- 'Jailbreaking' AI services like ChatGPT and Claude 3 Opus is much easier than you think - Livescience.com - April 15th, 2024 [April 15th, 2024]
- Google AI podcast: 6 conversations with global leaders - The Keyword | Google Product and Technology News - April 15th, 2024 [April 15th, 2024]
- Galaxy AI features are coming to last-gen Samsung phones including the S21 series - The Verge - April 15th, 2024 [April 15th, 2024]
- AI's Most Promising Startups Are Getting Younger And Leaner - Forbes - April 15th, 2024 [April 15th, 2024]
- How to Stop Your Data From Being Used to Train AI - WIRED - April 15th, 2024 [April 15th, 2024]
- 7 of the best Sora AI videos featuring animals - Tom's Guide - April 15th, 2024 [April 15th, 2024]
- How Microsoft discovers and mitigates evolving attacks against AI guardrails - Microsoft - April 15th, 2024 [April 15th, 2024]
- Apple's First AI Features in iOS 18 Reportedly Won't Use Cloud Servers - MacRumors - April 15th, 2024 [April 15th, 2024]
- Samsung officially bringing One UI 6.1 and AI features to Galaxy S22, Fold 4, Flip 4 in May - 9to5Google - April 15th, 2024 [April 15th, 2024]
- Eat the future, pay with your face: my dystopian trip to an AI burger joint - The Guardian - April 15th, 2024 [April 15th, 2024]
- Humans Forget. AI Assistants Will Remember Everything - WIRED - April 15th, 2024 [April 15th, 2024]
- From boom to burst, the AI bubble is only heading in one direction - The Guardian - April 15th, 2024 [April 15th, 2024]
- Meta and Google announce new in-house AI chips, creating a trillion-dollar question for Nvidia - Fortune - April 15th, 2024 [April 15th, 2024]
- Google goes all in on generative AI at Google Cloud Next - TechCrunch - April 15th, 2024 [April 15th, 2024]
- Humane AI Pin review: the post-smartphone future isnt here yet - The Verge - April 15th, 2024 [April 15th, 2024]
- Ukraines attacks on Russian oil refineries show the growing threat AI drones pose to energy markets - CNBC - April 15th, 2024 [April 15th, 2024]
- Texas is replacing thousands of human exam graders with AI - The Verge - April 15th, 2024 [April 15th, 2024]
- Humane AI Pin review and OpenAIs YouTube project - The Verge - April 15th, 2024 [April 15th, 2024]
- AI makes retinal imaging 100 times faster, compared to manual method - National Institutes of Health (NIH) (.gov) - April 15th, 2024 [April 15th, 2024]
- AI Is Poised to Replace the Entry-Level Grunt Work of a Wall Street Career - The New York Times - April 15th, 2024 [April 15th, 2024]
- $10 Billion Productivity Startup Notion Wants To Build Your AI Everything App - Forbes - April 15th, 2024 [April 15th, 2024]
- Wall Street is bullish on copper, thanks to AI. Analysts love these stocks, giving one 234% upside - CNBC - April 15th, 2024 [April 15th, 2024]
- AI editing tools are coming to all Google Photos users - The Keyword | Google Product and Technology News - April 15th, 2024 [April 15th, 2024]
- AI model has potential to detect risk of childbirth-related post-traumatic stress disorder - National Institutes of Health (NIH) (.gov) - April 15th, 2024 [April 15th, 2024]
- How soon will machines outsmart humans? The biggest brains in AI disagree - Financial Times - April 15th, 2024 [April 15th, 2024]
- 3 Hot Artificial Intelligence (AI) Stocks to Buy With $1,000 and Hold Forever - Yahoo Finance - March 24th, 2024 [March 24th, 2024]
- 11 Stocks That Will Profit From AI Evolution - Yahoo Finance - March 24th, 2024 [March 24th, 2024]
- USF plans to launch college focused on artificial intelligence, cybersecurity and computing - University of South Florida - March 24th, 2024 [March 24th, 2024]
- Stability AI CEO resigns to pursue decentralized AI - The Verge - March 24th, 2024 [March 24th, 2024]
- Scientists create AI models that can talk to each other and pass on skills with limited human input - Livescience.com - March 24th, 2024 [March 24th, 2024]
- AI-generated blues misses a human touch and a metronome - The Verge - March 24th, 2024 [March 24th, 2024]
- Financial Times tests an AI chatbot trained on decades of its own articles - The Verge - March 24th, 2024 [March 24th, 2024]
- Microsoft's First AI Surface PC: What Does It Offer? - Investopedia - March 24th, 2024 [March 24th, 2024]
- S&P 500 Stocks: AI Stock Super Micro Is Lapping The Field In 2024, Even Nvidia - Investor's Business Daily - March 24th, 2024 [March 24th, 2024]
- Using AI to expand global access to reliable flood forecasts - Google Research - March 24th, 2024 [March 24th, 2024]
- Generative AI for designing and validating easily synthesizable and structurally novel antibiotics - Nature.com - March 24th, 2024 [March 24th, 2024]
- Top 8 Free AI Tools in 2024 - eWeek - March 24th, 2024 [March 24th, 2024]
- Researchers gave AI an 'inner monologue' and it massively improved its performance - Livescience.com - March 24th, 2024 [March 24th, 2024]
- The iPhone 16 could come with extra RAM and storage just for AI - TechRadar - March 24th, 2024 [March 24th, 2024]
- World's first global AI resolution unanimously adopted by United Nations - Ars Technica - March 24th, 2024 [March 24th, 2024]
- How Google uses AI to improve global flood forecasting - The Keyword | Google Product and Technology News - March 24th, 2024 [March 24th, 2024]
- These are JPMorgan's top AI stock picks outside of the chip space - CNBC - March 24th, 2024 [March 24th, 2024]
- The first batch of Rabbit R1 AI devices will be shipping next week - TechRadar - March 24th, 2024 [March 24th, 2024]
- 7 great Google Gemini AI prompts to try this weekend - Tom's Guide - March 24th, 2024 [March 24th, 2024]
- Stability AI CEO resigns because youre not going to beat centralized AI with more centralized AI - TechCrunch - March 24th, 2024 [March 24th, 2024]
- Nvidia inks tie-ups with Abridge, GE HealthCare and Microsoft as it expands its footprint in healthcare AI - Fierce healthcare - March 24th, 2024 [March 24th, 2024]
- A Stealth AI 'Agent' To Speed Global Hiring Launches With $27 Million Fundraise - Forbes - March 24th, 2024 [March 24th, 2024]
- Microsoft and NVIDIA announce major integrations to accelerate generative AI for enterprises everywhere - Stories - Microsoft - March 24th, 2024 [March 24th, 2024]
- SMCI Stock: Why Chasing the 'Obvious' AI Play Could Leave You Burned - InvestorPlace - March 24th, 2024 [March 24th, 2024]
- Late Night With the Devil Directors Explain Using AI Art in the Film, Say They Experimented With Three Images Only (EXCLUSIVE) - Variety - March 24th, 2024 [March 24th, 2024]
- Broadcom shows a gargantuan AI chip XPU could be the world's largest chip built for a consumer AI company - Tom's Hardware - March 24th, 2024 [March 24th, 2024]
- SOUN Stock: The AI Voice Revolution Starts Here, and Nvidia Knows It - InvestorPlace - March 24th, 2024 [March 24th, 2024]
- Securing generative AI: Applying relevant security controls - AWS Blog - March 24th, 2024 [March 24th, 2024]
- Mustafa Suleyman, DeepMind and Inflection Co-founder, joins Microsoft to lead Copilot - The Official Microsoft Blog - Microsoft - March 24th, 2024 [March 24th, 2024]
- OpenAI Unveils A.I. That Instantly Generates Eye-Popping Videos - The New York Times - February 19th, 2024 [February 19th, 2024]
- Reddit signs content licensing deal with AI company ahead of IPO, Bloomberg reports - Reuters - February 19th, 2024 [February 19th, 2024]
- Install open-source AI in a commercial robot and it'll clean your room - Big Think - February 19th, 2024 [February 19th, 2024]
- There's AI, and Then There's AGI: What You Need to Know to Tell the Difference - CNET - February 19th, 2024 [February 19th, 2024]
- Mysterious Entity Paying Reddit $60 Million to Train AI With Users' Posts - Futurism - February 19th, 2024 [February 19th, 2024]
- PayPal Ventures first AI investment, a credit-based dating app and Robinhoods good week - TechCrunch - February 19th, 2024 [February 19th, 2024]
- I'll Just ChatGPT It: Questioning The Effects of AI Use in Classrooms - The Connecticut College Voice - February 19th, 2024 [February 19th, 2024]
- Reddit has a new AI training deal to sell user content - The Verge - February 19th, 2024 [February 19th, 2024]
- Super Micro Computer: Riding The AI Revolution (NASDAQ:SMCI) - Seeking Alpha - February 19th, 2024 [February 19th, 2024]
- Reddit reportedly signed a multi-million dollar licensing deal to train AI models - Mashable - February 19th, 2024 [February 19th, 2024]
- 6 Compelling Uses of Generative AI | Inc.com - Inc. - February 19th, 2024 [February 19th, 2024]
- Billionaire Investor Chase Coleman Has 46% of His Portfolio Invested in 5 Brilliant Artificial Intelligence (AI) Growth ... - The Motley Fool - February 19th, 2024 [February 19th, 2024]
- We Tested an AI Tutor for Kids. It Struggled With Basic Math. - The Wall Street Journal - February 19th, 2024 [February 19th, 2024]
- OpenAI, Meta and other tech giants sign effort to fight AI election interference - Reuters - February 19th, 2024 [February 19th, 2024]
- OpenAI's Sam Altman has huge chip ambitions. They might not work - Quartz - February 19th, 2024 [February 19th, 2024]
- AI Avatars Will Soon Attend Your Work Meetings, Claims Tech CEO - NDTV - February 19th, 2024 [February 19th, 2024]
- AI In Focus Ahead Of Nvidia's Earnings, Assessing AIO's Outlook (NYSE:AIO) - Seeking Alpha - February 19th, 2024 [February 19th, 2024]
- Scoop: N.Y. governor wants to criminalize deceptive AI - Axios - February 19th, 2024 [February 19th, 2024]
- What Are AI Text Generators? 8 Best Tools To Improve Writing - Forbes - February 19th, 2024 [February 19th, 2024]
- Staying ahead of threat actors in the age of AI - Microsoft - February 19th, 2024 [February 19th, 2024]
- Election security threats in 2024 range from AI to anthrax? - The Register - February 19th, 2024 [February 19th, 2024]
Tags: