zkLLM: Zero-Knowledge Large Language Model

The proliferation of Large Language Models (LLMs) has created unprecedented opportunities for AI-powered applications, yet fundamental challenges remain around trust, privacy, and verifiability. Current AI services operate as black boxes where users cannot verify computational integrity, model authenticity, or ensure input privacy. This paper introduces zkLLM, a revolutionary hosting service that leverages Zero-Knowledge Virtual Machines (zkVMs) to provide cryptographically verifiable AI inference while preserving user privacy. Our platform enables hosting providers to offer LLM services with mathematical guarantees of correct computation, authentic model usage, and private data processing

Overview

## Quick Start

First, make sure rustup is installed. The rust-toolchain.toml file will be used by cargo to automatically install the correct version.

To build all methods and execute the method within the zkVM, run the following command:

cargo run

This is an empty template, and so there is no expected output (until you modify the code).

Executing the Project Locally in Development Mode

During development, faster iteration upon code changes can be achieved by leveraging dev-mode, we strongly suggest activating it during your early development phase. Furthermore, you might want to get insights into the execution statistics of your project, and this can be achieved by specifying the environment variable RUST_LOG="[executor]=info" before running your project.

Put together, the command to run your project in development mode while getting execution statistics is:

RUST_LOG="[executor]=info" RISC0_DEV_MODE=1 cargo run

Running Proofs Remotely on Bonsai

Note: The Bonsai proving service is still in early Alpha; an API key is required for access. [Click here to request access][bonsai access].

If you have access to the URL and API key to Bonsai you can run your proofs remotely. To prove in Bonsai mode, invoke cargo run with two additional environment variables:

BONSAI_API_KEY="YOUR_API_KEY" BONSAI_API_URL="BONSAI_URL" cargo run

How to Create a Project Based on This Template

Search this template for the string TODO, and make the necessary changes to implement the required feature described by the TODO comment. Some of these changes will be complex, and so we have a number of instructional resources to assist you in learning how to write your own code for the RISC Zero zkVM:

• The RISC Zero Developer Docs is a great place to get started.
• Example projects are available in the examples folder of risc0 repository.
• Reference documentation is available at https://docs.rs, including risc0-zkvm, cargo-risczero, risc0-build, and others.

Directory Structure

It is possible to organize the files for these components in various ways. However, in this starter template we use a standard directory structure for zkVM applications, which we think is a good starting point for your applications.

project_name
├── Cargo.toml
├── host
│   ├── Cargo.toml
│   └── src
│       └── main.rs                    <-- [Host code goes here]
└── methods
    ├── Cargo.toml
    ├── build.rs
    ├── guest
    │   ├── Cargo.toml
    │   └── src
    │       └── method_name.rs         <-- [Guest code goes here]
    └── src
        └── lib.rs