Orange isn’t building its own AI foundation model here’s why – Light Reading
There has been a flurry of interest in generative AI (GenAI) from telcos, each of which has taken its own nuanced approach to the idea of building its own large language models (LLMs). While Vodafone seems todismiss the ideaand Verizon appears content to build on existing foundation models, Deutsche Telekom and SK Telecomannounced last yearthey will develop telco-specific LLMs. Orange, meanwhile, doesn't currently see the need to build a foundation model, its chief AI officer Steve Jarrett has recently told Light Reading.
Jarrett said the company is currently content with using existing models and adapting them to its needs using two main approaches. The first one is retrieval-augmented generation (RAG), where a detailed source of information is passed to the model together with the prompt to augment its response.
He said this allows the company to experiment with different prompts easily, adding that existing methodologies can be used to assess the results. "That is a very, very easy way to dynamically test different models, different styles of structuring the RAG and the prompts. And [] that solves the majority of our needs today," he elaborated.
At the same time, Jarrett admitted that the downside of RAG is that it may require a lot of data to be passed along with the prompt, making more complex tasks slow and expensive. In such cases, he argued, fine-tuning is a more appropriate approach.
Distilling models
In this case, he explained, "you take the information that you would have used in the RAG for [] a huge problem area. And you make a new version of the underlying model that embeds all that information." Another related option is to distill the model.
This involves not just structuring the output of the model, but downsizing it, "like you're distilling fruit into alcohol," Jarrett said, adding "there are techniques to actually chop the model down into a much smaller model that runs much faster."
This approach is, however, highly challenging. "Even my most expert people frequently make mistakes," he admitted, saying: "It's not simple, and the state of the art of the tools to fine tune are changing every single day." At the same time, he noted that these tools are improving constantly and, as a result, he expects fine-tuning to get easier over time.
He pointed out that building a foundation model from scratch would be an even more complex task, which the company currently doesn't see a reason to embark on. Nevertheless, he stressed that it's impossible to predict how things will evolve in the future.
Complexity budget
One possibility is that big foundational models will eventually absorb so much information that the need for RAG and other tools will diminish. In this scenario, Orange may never have to create its own foundation model, Jarrett said, "as long as we have the ability to distill and fine tune models, where we need to, to make the model small enough to run faster and cheaper and so on."
He added: "I think it's a very open question in the industry. In the end, will we have a handful of massive models, and everyone's doing 99% RAG and prompt engineering, or are there going to be millions of distilled and fine-tuned models?"
One factor that may determine where things will go in the future is what Jarrett calls the complexity budget. This is a concept that conveys how much computing was needed from start to finish to produce an answer.
While a very large model may be more intensive to train in the beginning, there may be less computing required for RAG and fine-tuning. "The other approach is you have a large language model that also obviously took a lot of training, but then you do a ton more compute to fine tune and distill the model so that your model is much smaller," he added.
Apart from cost, there is also an environmental concern. While hyperscalers tend to perform relatively well in terms of using clean energy, and Jarrett claimed that Orange is "fairly green as a company," he added that the carbon intensity of the energy used for on-premises GPU clusters tends to vary in the industry.
Right tool for the job
The uncertainty surrounding GenAI's future evolution is one of the reasons why Orange is taking a measured approach to the technology, with Jarrett stressing it is not a tool that's suited to every job. "You don't want to use the large language model sledge hammer to hit every nail," he said.
"I think, fairly uniquely compared to most other telco operators, we actually have the ability, the skill inside of Orange to help make these decisions about what tool to use when. So we prefer to use a statistical method or basic machine learning to solve problems because those results are more [] explainable. They're usually cheaper, and they're usually less impactful on the environment," he added.
In fact, Jarrett says one of the things Orange is investigating at the moment is how to use multiple AI models together to solve problems. The notion, he added, is called agents, and refers to a high-level abstraction of a problem, such as asking how the network in France is working on a given day. This, he said, will enable the company to solve complex problems more dynamically.
In the meantime, the company is making a range of GenAI models available to its employees, including ChatGPT, Dolly and Mistral. To do so, it has built a solution that Jarrett says provides a "secure, European-resident version of leading AI models that we make available to the entire company."
Improving customer service
Jarrett says this is a more controlled and safer way for employees to use models than if they were accessed directly. The solution also notifies the employee of the cost of running a specific model to answer a question. Available for several months, it has so far been used by 12% of employees.
Orange has already deployed GenAI in many countries within its customer service solutions to predict what the most appealing offer may be to an individual customer, Jarrett said, adding "what we're trialling right now is can generative AI help us to customize and personalize the text of that offer? Does that make the offer incrementally more appealing?"
Another potential use case is in transcribing a conversation with a customer care agent in real time, using generative AI to create prompts. The tool is still in development but could help new recruits to improve faster, raising employee and customer satisfaction, said Jarrett.
While Orange doesn't currently use GenAI for any use cases in the network, some are under development, although few details are being shared at this stage. One use case involves predicting when batteries at cell sites may need replacing.
Jarrett admits, however, that GenAI is still facing a number of challenges, such as hallucinations. "In a scenario where the outputs have to be correct 100% of the time, we're not going to use generative AI for that today, because [it's] not correct 100% of the time," he said.
Dealing with hallucinations
Yet it can be applied in areas that are less sensitive. "For example, if for internal use you want to have a summary of an enormous transcript of a long meeting that you missed, it's okay if the model hallucinates a little bit," he added.
Hallucinations cannot be stopped entirely and will likely continue to be a problem for some time, said Jarrett. But he believes RAG and fine-tuning could mitigate the issue to some extent.
"The majority of the time, if we're good at prompt engineering and we're good at passing the appropriate information with the response, the model generates very, very useful, relevant answers," Jarrett said about the results achieved with RAG.
The availability and quality of data is another issue that is often discussed, and also one that Orange is trying to address. Using data historically kept in separate silos has been difficult, said Jarrett. "[The] availability of the data from the marketing team to be able to run a campaign on where was our network relatively strong, for example those use cases were either impossible, or took many, many, many months of manual meetings and collaboration."
As a result, the company is trying to create a marketplace where data is made widely available inside each country and appropriately labeled. Orange calls this approach data democracy.
Continued here:
Orange isn't building its own AI foundation model here's why - Light Reading
- Optimization of wear parameters for ECAP-processed ZK30 alloy using response surface and machine learning ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Machine learning approach predicts heart failure outcome risk - HealthITAnalytics.com - April 22nd, 2024 [April 22nd, 2024]
- Practical approaches in evaluating validation and biases of machine learning applied to mobile health studies ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Application of power-law committee machine to combine five machine learning algorithms for enhanced oil recovery ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Free tool uses machine learning to pick better molecules for testing new reactions - Chemical & Engineering News - April 22nd, 2024 [April 22nd, 2024]
- Automated Analysis of Nuclear Parameters in Oral Exfoliative Cytology Using Machine Learning - Cureus - April 22nd, 2024 [April 22nd, 2024]
- An AI Ethics Researcher's Take On The Future Of Machine Learning In The Art World - SlashGear - April 22nd, 2024 [April 22nd, 2024]
- Enhancing Emotion Recognition in Users with Cochlear Implant Through Machine Learning and EEG Analysis - Physician's Weekly - April 22nd, 2024 [April 22nd, 2024]
- Imageomics Applies AI and Vision Advancements to Biological Questions - Photonics.com - April 22nd, 2024 [April 22nd, 2024]
- Machine learning reveals the control mechanics of an insect wing hinge - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- The Future of ML Development Services: Trends and Predictions - FinSMEs - April 22nd, 2024 [April 22nd, 2024]
- CSRWire - Island Conservation Harnesses Machine Learning Solutions From Lenovo and NVIDIA To Restore Island ... - CSRwire.com - April 22nd, 2024 [April 22nd, 2024]
- Investigation of the effectiveness of a classification method based on improved DAE feature extraction for hepatitis C ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Machine Learning Uncovers New Ways to Kill Bacteria With Non-Antibiotic Drugs - ScienceAlert - April 22nd, 2024 [April 22nd, 2024]
- Formal Interaction Model (FIM): A Mathematics-based Machine Learning Model that Formalizes How AI and Users Shape One Another - MarkTechPost - April 22nd, 2024 [April 22nd, 2024]
- A secure approach to generative AI with AWS | Amazon Web Services - AWS Blog - April 22nd, 2024 [April 22nd, 2024]
- Imbalanced Learn: the Python library for rebuilding ML datasets - DataScientest - April 22nd, 2024 [April 22nd, 2024]
- AI has a lot of terms. We've got a glossary for what you need to know - Quartz - April 22nd, 2024 [April 22nd, 2024]
- Texxa AI, Where ideas take flight: Revolutionizing AI Solutions for Businesses and Individuals - GlobeNewswire - April 22nd, 2024 [April 22nd, 2024]
- Using machine learning to identify patients with cancer that would benefit from immunotherapy - Medical Xpress - April 22nd, 2024 [April 22nd, 2024]
- Researchers at Stanford University Explore Direct Preference Optimization (DPO): A New Frontier in Machine Learning and Human Feedback - MarkTechPost - April 22nd, 2024 [April 22nd, 2024]
- Machine Learning Helps Scientists Locate the Neurological Origin of Psychosis - ExtremeTech - April 22nd, 2024 [April 22nd, 2024]
- Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services - AWS Blog - April 22nd, 2024 [April 22nd, 2024]
- Accurate and rapid antibiotic susceptibility testing using a machine learning-assisted nanomotion technology platform - Nature.com - March 20th, 2024 [March 20th, 2024]
- AI reveals the complexity of a simple birdsong - The Washington Post - March 20th, 2024 [March 20th, 2024]
- Researchers from MIT and Harvard Developed UNITS: A Unified Machine Learning Model for Time Series Analysis that Supports a Universal Task... - March 20th, 2024 [March 20th, 2024]
- Undergraduate Researchers Help Unlock Lessons of Machine Learning and AI - College of Natural Sciences - March 20th, 2024 [March 20th, 2024]
- Machine Learning Accelerates the Simulation of Dynamical Fields - Eos - March 20th, 2024 [March 20th, 2024]
- Inter hospital external validation of interpretable machine learning based triage score for the emergency department ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- HEAL: A framework for health equity assessment of machine learning performance - Google Research - March 20th, 2024 [March 20th, 2024]
- Expert on how machine learning could lead to improved outcomes in urology - Urology Times - March 20th, 2024 [March 20th, 2024]
- Unlock the potential of generative AI in industrial operations | Amazon Web Services - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Optimize price-performance of LLM inference on NVIDIA GPUs using the Amazon SageMaker integration with NVIDIA ... - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Wall Street's Favorite Machine Learning Stocks? 3 Names That Could Make You Filthy Rich - InvestorPlace - March 20th, 2024 [March 20th, 2024]
- Edge Impulse machine learning platform adds support for NVIDIA TAO Toolkit and Omniverse - CNX Software - March 20th, 2024 [March 20th, 2024]
- MIT Researchers Developed an Image Dataset that Allows Them to Simulate Peripheral Vision in Machine Learning Models - MarkTechPost - March 20th, 2024 [March 20th, 2024]
- 18 Cutting-Edge Artificial Intelligence Applications in 2024 - Simplilearn - March 20th, 2024 [March 20th, 2024]
- Machine-learning-based global optimization of microwave passives with variable-fidelity EM models and response ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- Benchmarking machine learning and parametric methods for genomic prediction of feed efficiency-related traits in ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- PyCaret: Everything you need to know about this Python library - DataScientest - March 20th, 2024 [March 20th, 2024]
- Crypto Entities That Neglect AI and Machine Learning Investment Will Lag Behind, Warns Binance CTO Bitcoin News - Bitcoin.com News - March 20th, 2024 [March 20th, 2024]
- VictoriaMetrics Machine Learning takes monitoring to the next level - The Bakersfield Californian - March 20th, 2024 [March 20th, 2024]
- How Marketers Can Elevate Creative Performance with AI-Driven Format Optimisation - ExchangeWire - March 20th, 2024 [March 20th, 2024]
- Revolutionizing carbon neutrality: Machine learning paves the way for advanced CO reduction catalysts - EurekAlert - March 20th, 2024 [March 20th, 2024]
- BurstAttention: A Groundbreaking Machine Learning Framework that Transforms Efficiency in Large Language Models with Advanced Distributed Attention... - March 20th, 2024 [March 20th, 2024]
- Construction of environmental vibration prediction model for subway transportation based on machine learning ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- Introducing 'Get started with generative AI on AWS: A guide for public sector organizations' | Amazon Web Services - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Generative deep learning for the development of a type 1 diabetes simulator | Communications Medicine - Nature.com - March 20th, 2024 [March 20th, 2024]
- Integrating core physics and machine learning for improved parameter prediction in boiling water reactor operations ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Top AI Certification Courses to Enroll in 2024 - Analytics Insight - March 11th, 2024 [March 11th, 2024]
- Machine learning techniques applied to construction: A hybrid bibliometric analysis of advances and future directions - ScienceDirect.com - March 11th, 2024 [March 11th, 2024]
- Artificial Intelligence Market towards a USD 2,745 bn by 2032 - Market.us Scoop - Market News - March 11th, 2024 [March 11th, 2024]
- Data Maturation Represents the Essential Reason for Deploying Machine Learning Today | By Adam Mogelonsky - Hospitality Net - March 11th, 2024 [March 11th, 2024]
- The Top 3 Machine Learning Stocks to Buy in March 2024 - InvestorPlace - March 11th, 2024 [March 11th, 2024]
- How to Learn the Math Needed for Data Science - Towards Data Science - March 11th, 2024 [March 11th, 2024]
- This AI Paper from Huawei Introduces DenseSSM: A Novel Machine Learning Approach to Enhance the Flow of Hidden Information between Layers in State... - March 11th, 2024 [March 11th, 2024]
- Machine learning and the prediction of suicide in psychiatric populations: a systematic review | Translational Psychiatry - Nature.com - March 11th, 2024 [March 11th, 2024]
- Machine learning algorithms show applications in OAB, antibiotic resistance - Urology Times - March 11th, 2024 [March 11th, 2024]
- Scientists develop new machine learning method for modeling chemical reactions - Phys.org - March 11th, 2024 [March 11th, 2024]
- Machine learning developed a CD8+ exhausted T cells signature for predicting prognosis, immune infiltration and drug ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Single Transit Detection In Kepler With Machine Learning And Onboard Spacecraft Diagnostics - Astrobiology - Astrobiology News - March 11th, 2024 [March 11th, 2024]
- Meta AI Proposes Wukong: A New Machine Learning Architecture that Exhibits Effective Dense Scaling Properties Towards a Scaling Law for Large-Scale... - March 11th, 2024 [March 11th, 2024]
- Putting the AI in NIA: New opportunities in artificial intelligence - National Institute on Aging - March 11th, 2024 [March 11th, 2024]
- Revolutionizing LLM Training with GaLore: A New Machine Learning Approach to Enhance Memory Efficiency without Compromising Performance - MarkTechPost - March 11th, 2024 [March 11th, 2024]
- Uncertainty-aware deep learning for trustworthy prediction of long-term outcome after endovascular thrombectomy ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- AI Engineer Salary: The Lucrative World of AI Engineering - Simplilearn - March 11th, 2024 [March 11th, 2024]
- Multimodal artificial intelligence-based pathogenomics improves survival prediction in oral squamous cell carcinoma ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Northrop Grumman Partners to Advance Deep Sensing for the US Army | Northrop Grumman - Northrop Grumman Newsroom - March 11th, 2024 [March 11th, 2024]
- Global cellular IoT connections to grow 90% to 6.5 bn by 2028: Juniper Research - ETTelecom - March 11th, 2024 [March 11th, 2024]
- Enhancing statistical reliability of weather forecasts with machine learning - Phys.org - March 11th, 2024 [March 11th, 2024]
- Inside AI: Talking to the Data - Inside Unmanned Systems - March 11th, 2024 [March 11th, 2024]
- Anemond's Factoid 2 is an experimental sampler plugin that uses machine learning to "decompose", remix and ... - MusicRadar - March 11th, 2024 [March 11th, 2024]
- Advancing Chemistry with AI: New Model for Simulating Diverse Organic Reactions - Lab Manager Magazine - March 11th, 2024 [March 11th, 2024]
- Generative AI: Understand the challenges to realize the opportunities | Amazon Web Services - AWS Blog - March 11th, 2024 [March 11th, 2024]
- How To Specialize in Artificial Intelligence - Troy Today - Troy University - March 11th, 2024 [March 11th, 2024]
- Google DeepMind Introduces Two Unique Machine Learning Models, Hawk And Griffin, Combining Gated Linear Recurrences With Local Attention For Efficient... - March 11th, 2024 [March 11th, 2024]
- Unlocking Innovation: AWS and Anthropic push the boundaries of generative AI together | Amazon Web Services - AWS Blog - March 11th, 2024 [March 11th, 2024]
- Introducing Microsoft's AI Red Team And PyRIT - AiThority - March 11th, 2024 [March 11th, 2024]
- Unveiling the World of Artificial Intelligence: A Beginner's Guide - Medium - January 3rd, 2024 [January 3rd, 2024]
- How machine learning might unlock earthquake prediction - MIT Technology Review - January 3rd, 2024 [January 3rd, 2024]
Tags: