Snowflake Cortex Finetuning LLMs

Large Language Models (LLMs) have demonstrated impressive capabilities in understanding and generating text. However, in specialized domains like healthcare, life sciences and medical devices, where proprietary data and nuanced terminology are common, the effectiveness of general LLMs can be limited.

Consider the scenario of MediSnow, a medical device company facing an influx of customer complaints about various products. Analyzing these complaints requires an understanding of the company's specific device names (like Xerocore 2000, MediLink Pro, SynthoPulse 3000), also specific device models, and potential issues related to those devices—information that isn't readily available in publicly trained models.

What is Fine-Tuning

Fine-tuning is a crucial technique for tailoring a large language model (LLM) to your specific domain or task. It involves training the model on a smaller, more focused dataset that aligns with your specific needs. This not only improves the model's performance on your targeted tasks but also potentially reduces the computational cost of inference compared to using a massive, general-purpose LLM.

Why Fine-Tuning Matters for MediSnow

Proprietary Knowledge: MediSnow's devices and associated terminology are unique to their product line. Fine-tuning allows them to inject this domain-specific knowledge into the LLM, making it more accurate in understanding and responding to customer feedback.
Cost Efficiency: Large, general-purpose LLMs can be computationally expensive to run. Fine-tuning a smaller model on MediSnow's data allows them to achieve comparable performance for their specific use case at a fraction of the cost.
Reduced Latency: Smaller, fine-tuned models generally provide faster responses, crucial for real-time applications like customer support interactions.
RAG Limitations: While Retrieval Augmented Generation (RAG) can provide additional context, it still relies on the base LLM's ability to understand the domain. If the base model is unfamiliar with medical device terminology, RAG's effectiveness will be limited.

Fine Tuning With Snowflake Cortex

Snowflake Cortex allows you to harness the power of Large Language Models (LLMs) and tailor them to your specific domain. In this guide, we'll demonstrate how to fine-tune a pre-trained model using customer complaints and device information from a medical device database (MediSnowDB). This fine-tuned model can then categorize complaints, extract sentiments, or perform other domain-specific tasks.

Note: In addition to scripting, you can also create and fine-tune LLM models using Snowsight's intuitive UI, which streamlines the process by guiding you through each step and generating the corresponding SQL script.

A Practical Example: Medical Devices Complaint Data

The following sections will guide you through a step-by-step process of how MediSnow can leverage Snowflake Cortex to fine-tune an LLM for their specific needs, maximizing the benefits of AI while maintaining cost-effectiveness.

First, run and setup the MediSnowDB sample data model and proceed with the rest of the exercises.

⛁ Sample Data Model: medisnowdb-data-model

The provided database, "MediSnowDB," with its two tables ("devices" and "customer_complaints"), offers a foundation for tracking and analyzing essential aspects of MediSnow's business:

>_ SQL

-- Assign the database schema
use schema medisnowdb.public

Let's delve into how fine-tuning LLMs in Snowflake Cortex can supercharge your data workflow:

1. Setting Up Your Training (and Validation) Data

Create training and validation tables that will guide the fine-tuning process:

SQL

-- Create training table with prompts (complaint details) and completions (business unit)
CREATE OR REPLACE TABLE model_train AS
SELECT c.complaintdetails AS prompt, d.businessunit AS completion 
FROM customer_complaints c 
JOIN devices d ON c.DEVICENAME = d.DEVICENAME;

-- Create validation table (optional) to monitor fine-tuning progress
CREATE OR REPLACE TABLE model_validation AS
SELECT c.complaintdetails AS prompt, d.businessunit AS completion 
FROM customer_complaints c 
JOIN devices d ON c.DEVICENAME = d.DEVICENAME;

-- Validate the prompt format for fine-tuning
SELECT concat('What business unit manufactured the device in this customer complaint text? ', prompt) AS prompt, completion 
FROM

3. Fine-Tuning Your LLM Model with Snowflake Cortex

Fine-tuning large language models (LLMs) like Llama3–8B within Snowflake Cortex is a powerful way to customize their performance for your specific needs. This involves training the model on a focused dataset tailored to your task, resulting in improved accuracy and efficiency for your application.

Snowflake Cortex simplifies the process by handling resource management and optimizing the workflow, allowing you to concentrate on your data and objectives.

After fine-tuning, you can seamlessly deploy and integrate your customized model for tasks like text summarization, question answering, or content generation within your domain, enhancing its capabilities and delivering superior results.

The example below shows how to initiate the fine-tuning process using Snowflake Cortex. This example fine-tunes a Llama3–8B model, but you can choose from a variety of available models (see the Snowflake documentation):

SQL

-- Initialize the Model Finetuning process in Snowflake 
-- We're using the 'CREATE' action to start a new finetuning job

SELECT SNOWFLAKE.CORTEX.FINETUNE(
    'CREATE',                             -- Action: Create a new finetuning job
    'device_llama3_8b',                   -- Model Name (unique identifier for this job)
    'llama3-8b',                          -- Base Model (the pretrained model to finetune)
    'SELECT CONCAT(''What business unit manufactured the device in this customer complaint text? '', prompt) AS prompt, completion FROM model_train', -- Training Data SQL (query to get prompts and completions)
    'SELECT CONCAT(''What business unit manufactured the device in this customer complaint text? '', prompt) AS prompt, completion FROM model_validation'  -- Validation Data SQL (optional, for evaluating model during training)
);


/*
Monitor Finetuning Progress

The following commands are used to check the status of your finetuning jobs:

-- SHOW: Lists all active finetuning jobs
SELECT SNOWFLAKE.CORTEX.FINETUNE('SHOW');

-- DESCRIBE: Provides details about a specific finetuning job (replace with actual ID)
SELECT SNOWFLAKE.CORTEX.FINETUNE(
    'DESCRIBE',
    'CortexFineTuningWorkflow_xxxxxxxx-6f8a-4417-87ff-7c71d680a0d6' 
);
*/ 


/*
Important Notes:

* Naming Conventions: Model names should use only underscores (_), not hyphens (-).
* Validation: The validation data table is optional. You can train without it.
* Finetuning Time: This process can take a while depending on your model, data size, and Snowflake resources. Be prepared for potentially significant wait times.
* Monitoring: Use the 'SHOW' and 'DESCRIBE' commands to track progress and investigate issues.
* Further Actions: Once finetuning is complete, you'll typically use additional commands to deploy and utilize your fine-tuned model.

*/

After the job is complete, the fine-tuned model is available in your environment and also visible in the SnowSight UI. With the right privileges, it can be securely governed and democratized across the organization for usage.

4. Testing and Evaluating Your Fine-Tuned Model

After the fine-tuning process is complete, rigorously test the model for performance and accuracy, compared to the base model:

SQL

-- Initial Testing without Fine-tuning (Baseline Performance)
-- These queries demonstrate the performance of the base llama3 models before fine-tuning
SELECT DeviceName, SNOWFLAKE.CORTEX.COMPLETE(
    'llama3-8b',                                   -- Base Model (8 billion parameters)
    CONCAT('What business unit manufactured this device. Only answer the question with business unit name, not other explanations needed. If there is no known answer, return N/A: ', complaintdetails)  -- Prompt
) as BU_Name 
FROM customer_complaints;

-- Repeating test with a larger base model (70 billion parameters) for comparison
SELECT DeviceName, SNOWFLAKE.CORTEX.COMPLETE(
    'llama3-70b',                                  -- Larger Base Model 
    CONCAT('What business unit manufactured this device. Only answer the question with business unit name, not other explanations needed. If there is no known answer, return N/A: ', complaintdetails)  -- Prompt
) as BU_Name 
FROM customer_complaints;


-- Testing with the Fine-tuned Model
-- This demonstrates the performance of your custom model on the specific task
SELECT DeviceName, SNOWFLAKE.CORTEX.COMPLETE(
    'device_llama3_8b',                            -- Your Fine-tuned Model
    CONCAT('What business unit manufactured this device. Only answer the question with business unit name, not other explanations needed. If there is no known answer, return N/A: ', complaintdetails)  -- Prompt
) as BU_Name 
FROM customer_complaints;


-- Additional Testing on a Different Task with the Fine-tuned Model
-- Testing the model's ability to analyze customer sentiment
SELECT DeviceName, COMPLAINTDETAILS, SNOWFLAKE.CORTEX.COMPLETE(
    'device_llama3_8b',                            -- Your Fine-tuned Model
    CONCAT('Was this customer sentiment, postive or negative: ', complaintdetails)   -- Prompt (slightly modified for this task)
) as Sentiment 
FROM customer_complaints;



/*
Key Points:

* Baseline Comparison: The first two queries establish a baseline performance using the general purpose llama3 models. This helps gauge the improvement gained through fine-tuning.
* Task-Specific Prompting: Notice how the prompt is designed to guide the model towards the specific information you want (business unit name or sentiment).
* Fine-tuning Benefits: Compare the results of the fine-tuned model to the base models. You should ideally see improved accuracy and relevance in identifying business units and sentiment analysis.
* Further Analysis: Consider a more detailed analysis of the results, including metrics like accuracy, false positives/negatives, etc.
* Model Limitations:  Even with fine-tuning, the model might still make errors, especially with ambiguous or complex complaint details.  

*/

Improved Results with Fine-Tuned Models, answered correctly based on knowledge of proprietary data in the database:

Compare this with the less accurate and potentially more costly responses from the Llama-70b base model. Even with larger size and cost, the results are inaccurate for this "specific" task.

By comparing results between the base and fine-tuned models, you can assess the improvements gained from fine-tuning. Beyond the complaint data, you can then use the fine-tuned model to perform specific tasks on your medical device data from the data model.

Fine-tuning as a Strategic Advantage

By fine-tuning a smaller, more affordable LLM, MediSnow can create a highly specialized model that excels at understanding customer complaints, categorizing issues, and potentially even generating automated responses. This not only improves the efficiency of their customer support processes but also allows them to gain deeper insights into product performance and customer sentiment.

Conclusion

Snowflake Cortex enables you to fine-tune powerful Large Language Models for your domain-specific needs. This empowers you to extract valuable insights from your data, automate tasks, and enhance decision-making in the healthcare domain.