Covalent lets developers send Python functions to GPU or CPU infrastructure with a decorator, without managing Docker or Kubernetes.
2
A workflow can fine-tune multiple models, evaluate their accuracy, select the best one, and deploy it from Python.
3
Covalent runs functions on different machines and charges for the compute time used, such as an L4 evaluation costing 87 cents for six minutes.
Summary
Santosh Radha demonstrates how Covalent lets a developer move Python functions from a local notebook to remote compute. A decorator specifies the machine, GPU type, memory, and time limit. The same approach can deploy an inference service with an API endpoint, scale it to zero, and apply custom autoscaling and authentication settings. His example defines a fine-tuning function, evaluates each model, sorts the results on a CPU machine, and deploys the best model. A workflow loops over the models and sends the whole process to Covalent with a single dispatch call. The application then shows each task running on its assigned hardware. One evaluation runs on an L4 for six minutes and costs 87 cents, while another runs on a V100 for six minutes and costs 11 cents. The talk is a short product demonstration rather than a discussion of model quality or training results.
Covalent sends local Python functions to selected compute
Santosh describes Covalent as an open source product for writing Python locally and sending the code to a compute backend. A decorator can request a specific machine, such as an H100 with 36 GB of memory and a two-day time limit. The function is then shipped to a GPU backend when the user presses Shift-Enter in a Jupyter notebook. With the open source version, the user can attach a cluster or other infrastructure. With the cloud version, the function runs on Covalent's GPU cluster and the user pays for the GPU time consumed.
The platform handles deployment without Docker or Kubernetes code
Jobs are defined as single functions with parameters describing the compute they need. Covalent also supports inference services through an initializer function and an endpoint function such as generate. A single deploy command from a notebook ships the service and returns an API endpoint. The endpoint can scale to zero, scale when requests arrive, and use custom rules. Santosh gives examples including scaling to 10 GPUs at a fixed time, scaling when GPU utilization reaches 80 percent, or scaling when the request count reaches 1,000.
The fine-tuning workflow separates training, evaluation, and selection
Santosh's example starts with ordinary Python functions named fine tune and evaluate. The fine-tuning task accepts a model and data and returns a fine-tuned model. He assigns training to a 24-core CPU machine with one H100 GPU, 48 GB of memory, and an 18-hour limit. After training, another task evaluates accuracy. Sorting and model selection run on a CPU machine so GPU time is not used for that work. The final step deploys the model that performed best.
A Python workflow can run the same process across many models
The workflow loops over a collection of models, calls the fine-tuning function, evaluates each result, and records the accuracy. It then sorts the models, selects the best one, and deploys it. Santosh dispatches this workflow with a single line. Covalent creates a job in its application, and each function runs on the device specified in the code. The example uses 20 models, while he says the number is arbitrary and the same approach can handle hundreds.
The application exposes task machines and compute costs
After dispatch, the application shows the individual evaluation steps and the machines they used. One evaluation runs on an L4 for six minutes and costs 87 cents. Another runs on a V100 for six minutes and costs 11 cents. Santosh presents this as a complete Python workflow that fine-tunes, evaluates, selects, and deploys models without the user writing Docker or Kubernetes configuration.
"You can fine tune pretty much 20 is an arbitrary number here, but hundreds of models that you can do right from Python without needing anything like Kubernetes or Docker on your side."00:03
Who should watch
You have Python training or inference code but do not want to build and maintain Kubernetes and Docker infrastructure for every experiment.
Your workflow needs to run model training on GPUs, move evaluation to CPUs, and deploy the selected model from one notebook.
You are comparing managed GPU execution with attaching your own cloud or on-premises compute and want a concrete example of the workflow.
