LLM

Large Language Models in OCI Data Science

OCI Data Science can be used to fine-tune, deploy, and manage Large Langugage Models (LLMs) effectively, efficiently, and easily. This page curates the links to some common use cases for LLMs.

Fine tune Llama 2 with distributed multi-node, multi-GPU job

Quantize Llama 2 70B to 4 bits and deploy on 2xA10s

Deploy Llama 2 on fully service managed deployment using TGI or vLLM

Deploy Mistral 7B

Deploy GPT-2 using NVidia Triton Inference Server

AI Quick Actions - Fine-tune, deploy, and evaluate LLMs without writing code

AI Quick Actions make working with LLMs super simple and requires no coding. From the AI Quick Actions extension in a Notebook session, you can explore foundation models, kickoff a fine-tuning process, deploy as a web endpoint and test it with a simple chat interface, and run evaluation jobs. Learn more in this blog post: Introducing AI Quick Actions in OCI Data Science