Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services – AWS Blog
This post is co-authored by Jackie Rocca, VP of Product, AI at Slack
Slack is where work happens. Its the AI-powered platform for work that connects people, conversations, apps, and systems together in one place. With the newly launched Slack AIa trusted, native, generative artificial intelligence (AI) experience available directly in Slackusers can surface and prioritize information so they can find their focus and do their most productive work.
We are excited to announce that Slack, a Salesforce company, has collaborated with Amazon SageMaker JumpStart to power Slack AIs initial search and summarization features and provide safeguards for Slack to use large language models (LLMs) more securely. Slack worked with SageMaker JumpStart to host industry-leading third-party LLMs so that data is not shared with the infrastructure owned by third party model providers.
This keeps customer data in Slack at all times and upholds the same security practices and compliance standards that customers expect from Slack itself. Slack is also using Amazon SageMaker inference capabilities for advanced routing strategies to scale the solution to customers with optimal performance, latency, and throughput.
With Amazon SageMaker JumpStart, Slack can access state-of-the-art foundation models to power Slack AI, while prioritizing security and privacy. Slack customers can now search smarter, summarize conversations instantly, and be at their most productive.
Jackie Rocca, VP Product, AI at Slack
SageMaker JumpStart is a machine learning (ML) hub that can help accelerate your ML journey. With SageMaker JumpStart, you can evaluate, compare, and select foundation models (FMs) quickly based on predefined quality and responsibility metrics to perform tasks like article summarization and image generation. Pretrained models are fully customizable for your use case with your data, and you can effortlessly deploy them into production with the user interface or SDK. In addition, you can access prebuilt solutions to solve common use cases and share ML artifacts, including ML models and notebooks, within your organization to accelerate ML model building and deployment. None of your data is used to train the underlying models. All the data is encrypted and is never shared with third-party vendors so you can trust that your data remains private and confidential.
Check out the SageMaker JumpStart model page for available models.
Slack launched Slack AI to provide native generative AI capabilities so that customers can easily find and consume large volumes of information quickly, enabling them to get even more value out of their shared knowledge in Slack. For example, users can ask a question in plain language and instantly get clear and concise answers with enhanced search. They can catch up on channels and threads in one click with conversation summaries. And they can access personalized, daily digests of whats happening in select channels with the newly launched recaps.
Because trust is Slacks most important value, Slack AI runs on an enterprise-grade infrastructure they built on AWS, upholding the same security practices and compliance standards that customers expect. Slack AI is built for security-conscious customers and is designed to be secure by designcustomer data remains in-house, data is not used for LLM training purposes, and data remains siloed.
SageMaker JumpStart provides access to many LLMs, and Slack selects the right FMs that fit their use cases. Because these models are hosted on Slacks owned AWS infrastructure, data sent to models during invocation doesnt leave Slacks AWS infrastructure. In addition, to provide a secure solution, data sent for invoking SageMaker models is encrypted in transit. The data sent to SageMaker JumpStart endpoints for invoking models is not used to train base models. SageMaker JumpStart allows Slack to support high standards for security and data privacy, while also using state-of-the-art models that help Slack AI perform optimally for Slack customers.
SageMaker JumpStart endpoints serving Slack business applications are powered by AWS instances. SageMaker supports a wide range of instance types for model deployment, which allows Slack to pick the instance that is best suited to support latency and scalability requirements of Slack AI use cases. Slack AI has access to multi-GPU based instances to host their SageMaker JumpStart models. Multiple GPU instances allow each instance backing Slack AIs endpoint to host multiple copies of a model. This helps improve resource utilization and reduce model deployment cost. For more information, refer to Amazon SageMaker adds new inference capabilities to help reduce foundation model deployment costs and latency.
The following diagram illustrates the solution architecture.
To use the instances most effectively and support the concurrency and latency requirements, Slack used SageMaker-offered routing strategies with their SageMaker endpoints. By default, a SageMaker endpoint uniformly distributes incoming requests to ML instances using a round-robin algorithm routing strategy called RANDOM. However, with generative AI workloads, requests and responses can be extremely variable, and its desirable to load balance by considering the capacity and utilization of the instance rather than random load balancing. To effectively distribute requests across instances backing the endpoints, Slack uses the LEAST_OUTSTANDING_REQUESTS (LAR) routing strategy. This strategy routes requests to the specific instances that have more capacity to process requests instead of randomly picking any available instance. The LAR strategy provides more uniform load balancing and resource utilization. As a result, Slack AI noticed over a 39% latency decrease in their p95 latency numbers when enabling LEAST_OUTSTANDING_REQUESTS compared to RANDOM.
For more details on SageMaker routing strategies, see Minimize real-time inference latency by using Amazon SageMaker routing strategies.
Slack is delivering native generative AI capabilities that will help their customers be more productive and easily tap into the collective knowledge thats embedded in their Slack conversations. With fast access to a large selection of FMs and advanced load balancing capabilities that are hosted in dedicated instances through SageMaker JumpStart, Slack AI is able to provide rich generative AI features in a more robust and quicker manner, while upholding Slacks trust and security standards.
Learn more about SageMaker JumpStart, Slack AI and how the Slack team built Slack AI to be secure and private. Leave your thoughts and questions in the comments section.
Jackie Rocca is VP of Product at Slack, where she oversees the vision and execution of Slack AI, which brings generative AI natively and securely into Slacks user experience. Now shes on a mission to help customers accelerate their productivity and get even more value out of their conversations, data, and collective knowledge with generative AI. Prior to her time at Slack, Jackie was a Product Manager at Google for more than six years, where she helped launch and grow Youtube TV. Jackie is based in the San Francisco Bay Area.
Rachna Chadha is a Principal Solutions Architect AI/ML in Strategic Accounts at AWS. Rachna is an optimist who believes that the ethical and responsible use of AI can improve society in the future and bring economic and social prosperity. In her spare time, Rachna likes spending time with her family, hiking, and listening to music.
Marc Karp is an ML Architect with the Amazon SageMaker Service team. He focuses on helping customers design, deploy, and manage ML workloads at scale. In his spare time, he enjoys traveling and exploring new places.
Maninder (Mani) Kaur is the AI/ML Specialist lead for Strategic ISVs at AWS. With her customer-first approach, Mani helps strategic customers shape their AI/ML strategy, fuel innovation, and accelerate their AI/ML journey. Mani is a firm believer of ethical and responsible AI, and strives to ensure that her customers AI solutions align with these principles.
Gene Ting is a Principal Solutions Architect at AWS. He is focused on helping enterprise customers build and operate workloads securely on AWS. In his free time, Gene enjoys teaching kids technology and sports, as well as following the latest on cybersecurity.
Alan Tan is a Senior Product Manager with SageMaker, leading efforts on large model inference. Hes passionate about applying machine learning to the area of analytics. Outside of work, he enjoys the outdoors.
Here is the original post:
Slack delivers native and secure generative AI powered by Amazon SageMaker JumpStart | Amazon Web Services - AWS Blog
- Optimization of wear parameters for ECAP-processed ZK30 alloy using response surface and machine learning ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Machine learning approach predicts heart failure outcome risk - HealthITAnalytics.com - April 22nd, 2024 [April 22nd, 2024]
- Practical approaches in evaluating validation and biases of machine learning applied to mobile health studies ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Application of power-law committee machine to combine five machine learning algorithms for enhanced oil recovery ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Free tool uses machine learning to pick better molecules for testing new reactions - Chemical & Engineering News - April 22nd, 2024 [April 22nd, 2024]
- Automated Analysis of Nuclear Parameters in Oral Exfoliative Cytology Using Machine Learning - Cureus - April 22nd, 2024 [April 22nd, 2024]
- An AI Ethics Researcher's Take On The Future Of Machine Learning In The Art World - SlashGear - April 22nd, 2024 [April 22nd, 2024]
- Enhancing Emotion Recognition in Users with Cochlear Implant Through Machine Learning and EEG Analysis - Physician's Weekly - April 22nd, 2024 [April 22nd, 2024]
- Imageomics Applies AI and Vision Advancements to Biological Questions - Photonics.com - April 22nd, 2024 [April 22nd, 2024]
- Machine learning reveals the control mechanics of an insect wing hinge - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- The Future of ML Development Services: Trends and Predictions - FinSMEs - April 22nd, 2024 [April 22nd, 2024]
- CSRWire - Island Conservation Harnesses Machine Learning Solutions From Lenovo and NVIDIA To Restore Island ... - CSRwire.com - April 22nd, 2024 [April 22nd, 2024]
- Investigation of the effectiveness of a classification method based on improved DAE feature extraction for hepatitis C ... - Nature.com - April 22nd, 2024 [April 22nd, 2024]
- Machine Learning Uncovers New Ways to Kill Bacteria With Non-Antibiotic Drugs - ScienceAlert - April 22nd, 2024 [April 22nd, 2024]
- Formal Interaction Model (FIM): A Mathematics-based Machine Learning Model that Formalizes How AI and Users Shape One Another - MarkTechPost - April 22nd, 2024 [April 22nd, 2024]
- A secure approach to generative AI with AWS | Amazon Web Services - AWS Blog - April 22nd, 2024 [April 22nd, 2024]
- Imbalanced Learn: the Python library for rebuilding ML datasets - DataScientest - April 22nd, 2024 [April 22nd, 2024]
- AI has a lot of terms. We've got a glossary for what you need to know - Quartz - April 22nd, 2024 [April 22nd, 2024]
- Texxa AI, Where ideas take flight: Revolutionizing AI Solutions for Businesses and Individuals - GlobeNewswire - April 22nd, 2024 [April 22nd, 2024]
- Using machine learning to identify patients with cancer that would benefit from immunotherapy - Medical Xpress - April 22nd, 2024 [April 22nd, 2024]
- Researchers at Stanford University Explore Direct Preference Optimization (DPO): A New Frontier in Machine Learning and Human Feedback - MarkTechPost - April 22nd, 2024 [April 22nd, 2024]
- Machine Learning Helps Scientists Locate the Neurological Origin of Psychosis - ExtremeTech - April 22nd, 2024 [April 22nd, 2024]
- Accurate and rapid antibiotic susceptibility testing using a machine learning-assisted nanomotion technology platform - Nature.com - March 20th, 2024 [March 20th, 2024]
- AI reveals the complexity of a simple birdsong - The Washington Post - March 20th, 2024 [March 20th, 2024]
- Researchers from MIT and Harvard Developed UNITS: A Unified Machine Learning Model for Time Series Analysis that Supports a Universal Task... - March 20th, 2024 [March 20th, 2024]
- Undergraduate Researchers Help Unlock Lessons of Machine Learning and AI - College of Natural Sciences - March 20th, 2024 [March 20th, 2024]
- Machine Learning Accelerates the Simulation of Dynamical Fields - Eos - March 20th, 2024 [March 20th, 2024]
- Inter hospital external validation of interpretable machine learning based triage score for the emergency department ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- HEAL: A framework for health equity assessment of machine learning performance - Google Research - March 20th, 2024 [March 20th, 2024]
- Expert on how machine learning could lead to improved outcomes in urology - Urology Times - March 20th, 2024 [March 20th, 2024]
- Unlock the potential of generative AI in industrial operations | Amazon Web Services - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Optimize price-performance of LLM inference on NVIDIA GPUs using the Amazon SageMaker integration with NVIDIA ... - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Orange isn't building its own AI foundation model here's why - Light Reading - March 20th, 2024 [March 20th, 2024]
- Wall Street's Favorite Machine Learning Stocks? 3 Names That Could Make You Filthy Rich - InvestorPlace - March 20th, 2024 [March 20th, 2024]
- Edge Impulse machine learning platform adds support for NVIDIA TAO Toolkit and Omniverse - CNX Software - March 20th, 2024 [March 20th, 2024]
- MIT Researchers Developed an Image Dataset that Allows Them to Simulate Peripheral Vision in Machine Learning Models - MarkTechPost - March 20th, 2024 [March 20th, 2024]
- 18 Cutting-Edge Artificial Intelligence Applications in 2024 - Simplilearn - March 20th, 2024 [March 20th, 2024]
- Machine-learning-based global optimization of microwave passives with variable-fidelity EM models and response ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- Benchmarking machine learning and parametric methods for genomic prediction of feed efficiency-related traits in ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- PyCaret: Everything you need to know about this Python library - DataScientest - March 20th, 2024 [March 20th, 2024]
- Crypto Entities That Neglect AI and Machine Learning Investment Will Lag Behind, Warns Binance CTO Bitcoin News - Bitcoin.com News - March 20th, 2024 [March 20th, 2024]
- VictoriaMetrics Machine Learning takes monitoring to the next level - The Bakersfield Californian - March 20th, 2024 [March 20th, 2024]
- How Marketers Can Elevate Creative Performance with AI-Driven Format Optimisation - ExchangeWire - March 20th, 2024 [March 20th, 2024]
- Revolutionizing carbon neutrality: Machine learning paves the way for advanced CO reduction catalysts - EurekAlert - March 20th, 2024 [March 20th, 2024]
- BurstAttention: A Groundbreaking Machine Learning Framework that Transforms Efficiency in Large Language Models with Advanced Distributed Attention... - March 20th, 2024 [March 20th, 2024]
- Construction of environmental vibration prediction model for subway transportation based on machine learning ... - Nature.com - March 20th, 2024 [March 20th, 2024]
- Introducing 'Get started with generative AI on AWS: A guide for public sector organizations' | Amazon Web Services - AWS Blog - March 20th, 2024 [March 20th, 2024]
- Generative deep learning for the development of a type 1 diabetes simulator | Communications Medicine - Nature.com - March 20th, 2024 [March 20th, 2024]
- Integrating core physics and machine learning for improved parameter prediction in boiling water reactor operations ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Top AI Certification Courses to Enroll in 2024 - Analytics Insight - March 11th, 2024 [March 11th, 2024]
- Machine learning techniques applied to construction: A hybrid bibliometric analysis of advances and future directions - ScienceDirect.com - March 11th, 2024 [March 11th, 2024]
- Artificial Intelligence Market towards a USD 2,745 bn by 2032 - Market.us Scoop - Market News - March 11th, 2024 [March 11th, 2024]
- Data Maturation Represents the Essential Reason for Deploying Machine Learning Today | By Adam Mogelonsky - Hospitality Net - March 11th, 2024 [March 11th, 2024]
- The Top 3 Machine Learning Stocks to Buy in March 2024 - InvestorPlace - March 11th, 2024 [March 11th, 2024]
- How to Learn the Math Needed for Data Science - Towards Data Science - March 11th, 2024 [March 11th, 2024]
- This AI Paper from Huawei Introduces DenseSSM: A Novel Machine Learning Approach to Enhance the Flow of Hidden Information between Layers in State... - March 11th, 2024 [March 11th, 2024]
- Machine learning and the prediction of suicide in psychiatric populations: a systematic review | Translational Psychiatry - Nature.com - March 11th, 2024 [March 11th, 2024]
- Machine learning algorithms show applications in OAB, antibiotic resistance - Urology Times - March 11th, 2024 [March 11th, 2024]
- Scientists develop new machine learning method for modeling chemical reactions - Phys.org - March 11th, 2024 [March 11th, 2024]
- Machine learning developed a CD8+ exhausted T cells signature for predicting prognosis, immune infiltration and drug ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Single Transit Detection In Kepler With Machine Learning And Onboard Spacecraft Diagnostics - Astrobiology - Astrobiology News - March 11th, 2024 [March 11th, 2024]
- Meta AI Proposes Wukong: A New Machine Learning Architecture that Exhibits Effective Dense Scaling Properties Towards a Scaling Law for Large-Scale... - March 11th, 2024 [March 11th, 2024]
- Putting the AI in NIA: New opportunities in artificial intelligence - National Institute on Aging - March 11th, 2024 [March 11th, 2024]
- Revolutionizing LLM Training with GaLore: A New Machine Learning Approach to Enhance Memory Efficiency without Compromising Performance - MarkTechPost - March 11th, 2024 [March 11th, 2024]
- Uncertainty-aware deep learning for trustworthy prediction of long-term outcome after endovascular thrombectomy ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- AI Engineer Salary: The Lucrative World of AI Engineering - Simplilearn - March 11th, 2024 [March 11th, 2024]
- Multimodal artificial intelligence-based pathogenomics improves survival prediction in oral squamous cell carcinoma ... - Nature.com - March 11th, 2024 [March 11th, 2024]
- Northrop Grumman Partners to Advance Deep Sensing for the US Army | Northrop Grumman - Northrop Grumman Newsroom - March 11th, 2024 [March 11th, 2024]
- Global cellular IoT connections to grow 90% to 6.5 bn by 2028: Juniper Research - ETTelecom - March 11th, 2024 [March 11th, 2024]
- Enhancing statistical reliability of weather forecasts with machine learning - Phys.org - March 11th, 2024 [March 11th, 2024]
- Inside AI: Talking to the Data - Inside Unmanned Systems - March 11th, 2024 [March 11th, 2024]
- Anemond's Factoid 2 is an experimental sampler plugin that uses machine learning to "decompose", remix and ... - MusicRadar - March 11th, 2024 [March 11th, 2024]
- Advancing Chemistry with AI: New Model for Simulating Diverse Organic Reactions - Lab Manager Magazine - March 11th, 2024 [March 11th, 2024]
- Generative AI: Understand the challenges to realize the opportunities | Amazon Web Services - AWS Blog - March 11th, 2024 [March 11th, 2024]
- How To Specialize in Artificial Intelligence - Troy Today - Troy University - March 11th, 2024 [March 11th, 2024]
- Google DeepMind Introduces Two Unique Machine Learning Models, Hawk And Griffin, Combining Gated Linear Recurrences With Local Attention For Efficient... - March 11th, 2024 [March 11th, 2024]
- Unlocking Innovation: AWS and Anthropic push the boundaries of generative AI together | Amazon Web Services - AWS Blog - March 11th, 2024 [March 11th, 2024]
- Introducing Microsoft's AI Red Team And PyRIT - AiThority - March 11th, 2024 [March 11th, 2024]
- Unveiling the World of Artificial Intelligence: A Beginner's Guide - Medium - January 3rd, 2024 [January 3rd, 2024]
- How machine learning might unlock earthquake prediction - MIT Technology Review - January 3rd, 2024 [January 3rd, 2024]
Tags: