Big Data News Hubb
Advertisement
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
Big Data News Hubb
No Result
View All Result
Home Big Data

Running Ray in Cloudera Machine Learning to Power Compute-Hungry LLMs

admin by admin
April 28, 2023
in Big Data


Posted in Technical |
April 27, 2023 5 min read

Lost in the talk about OpenAI is the tremendous amount of compute needed to train and fine-tune LLMs, like GPT, and Generative AI, like ChatGPT. Each iteration requires more compute and the limitation imposed by Moore’s Law quickly moves that task from single compute instances to distributed compute.  To accomplish this, OpenAI has employed Ray to power the distributed compute platform to train each release of the GPT models. Ray has emerged as a popular framework because of its superior erformance over Apache Spark for distributed AI compute workloads.  In the blog we will cover how Ray can be used in Cloudera Machine Learning’s open-by-design architecture to bring fast distributed AI compute to CDP.  This is enabled through a Ray Module in cmlextensions python package published by our team.

Ray’s ability to provide simple and efficient distributed computing capabilities, along with its native support for Python, has made it a favorite among data scientists and engineers alike. Its innovative architecture enables seamless integration with ML and deep learning libraries like TensorFlow and PyTorch. Furthermore, Ray’s unique approach to parallelism, which focuses on fine-grained task scheduling, enables it to handle a wider range of workloads compared to Spark. This enhanced flexibility and ease of use have positioned Ray as the go-to choice for organizations looking to harness the power of distributed computing.

Built on Kubernetes, Cloudera Machine Learning (CML) gives data science teams a platform that works across each stage of Machine Learning Lifecycle, supporting exploratory data analysis, the model development and moving those models and applications to production (aka MLOps). CML is built to be open by design, and that is why it includes a Worker API that can quickly spin up multiple compute pods on demand. Cloudera customers are able to bring together CML’s ability to spin up large compute clusters and integrate that with Ray to enable an easy to use, Python native, distributed compute platform. While Ray brings some of its own libraries for reinforcement learning, hyper parameter tuning, and model training and serving, users can also bring their favorite packages like XGBoost, Pytorch, LightGBM, Dask, and Pandas (using Modin). This fits right in with CML’s open by design, allowing data scientists to be able to take advantage of the latest innovations coming from the open-source community.

To make it easier for CML users to leverage Ray, Cloudera has published a Python package called CMLextensions. CMLextensions has a Ray module that manages provisioning compute workers in CML and then returning a Ray cluster to the user.  

To get started with Ray on CML, first you need to install the CMLextensions library.

With that in place, we can now spin up a Ray cluster.

This returns a provisioned Ray cluster.

Now we have a Ray cluster provisioned and we are ready to get to work. We can test out our Ray cluster with the following code:

Finally, when we are done with the Ray cluster, we can terminate it with:

Ray lowers the barriers to build fast and distributed Python applications.  Now we can spin up a Ray cluster in Cloudera Machine Learning.  Let’s check out how we can parallelize and distribute Python code with Ray.  To best understand this, we need to look at Ray Tasks and Actors, and how the Ray APIs allow you to implement distributed compute.

First, we will look at the concept of taking an existing function and making it into a Ray Task.  Lets look at a simple function to find the square of a number.

To make this into a remote function, all we need to do is use the @ray.remote decorator.

This makes it a remote function and calling the function immediately returns a future with the object reference.

In order to get the result from our function call, we can use the ray.get API call with the function which would result in execution being blocked until the result of the call is returned.

Building off of Ray Tasks, we next have the concept of Ray Actors to explore. Think of an Actor as a remote class that runs on one of our worker nodes. Lets start with a simple class that tracks test scores. We will use that same @ray.remote decorator which this time turns this class into a Ray Actor.

Next, we will create an instance of this Actor.

With this Actor deployed, we can now see the instance in the Ray Dashboard.

 

Just like with Ray Tasks, we will use the “.remote” extension to make function calls within our Ray Actor.

Similar to the Ray Task, calls to a Ray Actor will only result in an object reference being returned. We can use that same ray.get api call to block execution until data is returned.

 

The calls into our Actor also become trackable in the Ray Dashboard. Below you will see our actor, you can trace all of the calls to that actor, and you have access to logs for that worker.

An Actor’s lifetime can be detached from the current job and allowing it to persist afterwards. Through the ray.remote decorator, you can specify the resource requirements for Actors.

This is just a quick look at the Task and Actor concepts in Ray. We are just scratching the surface here but this should give a good foundation as we dive deeper into Ray. In the next installment, we will look at how Ray becomes the foundation to distribute and speed up dataframe workloads.

Enterprises of every size and industry are experimenting and capitalizing on the innovation with LLMs that can power a variety of domain specific applications.  Cloudera customers are well prepared to leverage next generation distributed compute frameworks like Ray right on top of their data.  This is the power of being open by design.

To learn more about Cloudera Machine Learning please visit the website and to get started with Ray in CML check out CMLextensions in our Github.



Source link

Previous Post

If Big Data Is the Immovable Object, Enterprise Search Is the Unstoppable Force

Next Post

3 Ways to Leverage ChatGPT for Your Ecommerce Website

Next Post

3 Ways to Leverage ChatGPT for Your Ecommerce Website

Recommended

6 Career Possibilities With a Degree in Artificial Intelligence and Machine Learning

October 6, 2023

Four Must-Have Elements of a SaaS Content Strategy

November 17, 2022

8 Tips on Using Data for Ecommerce

June 4, 2023

Don't miss it

News

Top 9 Mind Map Makers Online & Offline for Brainstorming

December 1, 2023
Big Data

Sam Altman returns as CEO, OpenAI has a new initial board

December 1, 2023
Big Data

Announcing General Availability of Model Registry

November 30, 2023
Big Data

Sophos Anticipates AI-Based Attack Techniques and Prepares Detections

November 30, 2023
Big Data

Automating Governance of PHI Data in Healthcare

November 30, 2023
News

How Eightfold AI implemented metadata security in a multi-tenant data analytics environment with Amazon Redshift

November 30, 2023
big-data-footer-white

© Big Data News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Big Data
  • News
  • Contact us

Newsletter Sign Up

No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us

© 2022 Big Data News Hubb All rights reserved.