Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion example/llm/lemonade/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Ryzen AI LLM Lemonade Examples

The following table contains a curated list of LLMs that have been validated with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) on Ryzen AI hybrid execution mode, along with CPU implementations of those same checkpoints.
The following table contains a curated list of LLMs that have been validated with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) on Ryzen AI hybrid execution mode, along with CPU implementations of those same checkpoints.

The hybrid examples are built on top of OnnxRuntime GenAI (OGA), while the CPU baseline is built on top of Hugging Face (HF) ``transformers``. Validation is defined as running all commands in the example page successfully.

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/CodeLlama_7b_Instruct_hf.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/CodeLlama_7b_Instruct_hf.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`meta-llama/CodeLlama-7b-Instruct-hf`](https://huggingface.co/meta-llama/CodeLlama-7b-Instruct-hf) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i meta-llama/CodeLlama-7b-Instruct-hf huggingface-load --device cpu --

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i meta-llama/CodeLlama-7b-Instruct-hf huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/DeepSeek_R1_Distill_Llama_8B.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/DeepSeek_R1_Distill_Llama_8B.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`deepseek-ai/DeepSeek-R1-Distill-Llama-8B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i deepseek-ai/DeepSeek-R1-Distill-Llama-8B huggingface-load --device c

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i deepseek-ai/DeepSeek-R1-Distill-Llama-8B huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/DeepSeek_R1_Distill_Qwen_1_5B.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/DeepSeek_R1_Distill_Qwen_1_5B.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B huggingface-load --device

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/DeepSeek_R1_Distill_Qwen_7B.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/DeepSeek_R1_Distill_Qwen_7B.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i deepseek-ai/DeepSeek-R1-Distill-Qwen-7B huggingface-load --device cp

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i deepseek-ai/DeepSeek-R1-Distill-Qwen-7B huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/Llama_2_7b_chat_hf.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/Llama_2_7b_chat_hf.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`meta-llama/Llama-2-7b-chat-hf`](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i meta-llama/Llama-2-7b-chat-hf huggingface-load --device cpu --dtype

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i meta-llama/Llama-2-7b-chat-hf huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
12 changes: 6 additions & 6 deletions example/llm/lemonade/cpu/Llama_2_7b_hf.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ This guide contains all of the instructions necessary to get started with the mo

The CPU implementation in this guide is designed to run on most PCs. However, for optimal performance on Ryzen AI 300-series PCs, try the [hybrid execution mode](../hybrid/Llama_2_7b_hf.md).

The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.
The commands and scripts in this guide leverage the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade), which provides everything you need to get up and running with LLMs on the OnnxRuntime GenAI (OGA) framework, as well as the support for Hugging Face `transformers` baselines leveraged in this guide.

# Checkpoint

The Hugging Face CPU implementation of [`meta-llama/Llama-2-7b-hf`](https://huggingface.co/meta-llama/Llama-2-7b-hf) uses the original safetensors weights provided in the Hugging Face checkpoint.

# Setup

To get started with the [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) in a Python environment, follow these instructions.
To get started with the [Lemonade SDK](https://github.com/lemonade-sdk/lemonade) in a Python environment, follow these instructions.

### System-level pre-requisites

Expand All @@ -35,9 +35,9 @@ To create and set up an environment, run these commands in your terminal:
conda activate ryzenai-llm
```

3. Install ONNX TurnkeyML to get access to the LLM tools and APIs.
3. Install the Lemonade SDK to get access to the LLM tools and APIs.
```bash
pip install turnkeyml[llm]
pip install lemonade-sdk[llm]
```

# Validation Tools
Expand All @@ -62,7 +62,7 @@ lemonade -i meta-llama/Llama-2-7b-hf huggingface-load --device cpu --dtype bfloa

## Task Performance

To measure the model's accuracy on the [MMLU test](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/mmlu_accuracy.md) `management` subject, run:
To measure the model's accuracy on the [MMLU test](https://github.com/lemonade-sdk/lemonade/blob/main/docs/mmlu_accuracy.md) `management` subject, run:

```bash
lemonade -i meta-llama/Llama-2-7b-hf huggingface-load --device cpu --dtype bfloat16 accuracy-mmlu --tests management
Expand Down Expand Up @@ -138,7 +138,7 @@ This guide provided instructions for testing and deploying an LLM on a target de

- Visit the [Lemonade LLM examples table](../README.md) to learn how to do this for any of the supported combinations of LLM and device.
- Visit the [overall Ryzen AI LLM documentation](https://ryzenai.docs.amd.com/en/latest/llm/overview.html#) to learn about other deployment options, such as native C++ libraries.
- Visit the [Lemonade SDK repository](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) to learn about more tools and features.
- Visit the [Lemonade SDK repository](https://github.com/lemonade-sdk/lemonade) to learn about more tools and features.

# Copyright

Expand Down
Loading