From 9cc968df7dbea8e39b7fa1db88a72a6f4759a23e Mon Sep 17 00:00:00 2001 From: hualxie Date: Thu, 11 Dec 2025 13:25:09 +0800 Subject: [PATCH 01/24] init --- .../images/profiling/by-model-file.png | 3 + .../profiling/by-process-id-or-name.png | 3 + .../profiling/the-next-session-admin.png | 3 + .../images/profiling/the-next-session.png | 3 + docs/intelligentapps/profiling.md | 85 +++++++++++++++++++ 5 files changed, 97 insertions(+) create mode 100644 docs/intelligentapps/images/profiling/by-model-file.png create mode 100644 docs/intelligentapps/images/profiling/by-process-id-or-name.png create mode 100644 docs/intelligentapps/images/profiling/the-next-session-admin.png create mode 100644 docs/intelligentapps/images/profiling/the-next-session.png create mode 100644 docs/intelligentapps/profiling.md diff --git a/docs/intelligentapps/images/profiling/by-model-file.png b/docs/intelligentapps/images/profiling/by-model-file.png new file mode 100644 index 0000000000..ea85bfa577 --- /dev/null +++ b/docs/intelligentapps/images/profiling/by-model-file.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c036f4d90d503e0e3fb61c206937c0af443c700edc0e7592ee1f6e4da96f8583 +size 11051 diff --git a/docs/intelligentapps/images/profiling/by-process-id-or-name.png b/docs/intelligentapps/images/profiling/by-process-id-or-name.png new file mode 100644 index 0000000000..1862239509 --- /dev/null +++ b/docs/intelligentapps/images/profiling/by-process-id-or-name.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e68a9d4289e7454f6c6ade92f95afb19aa7d3a1d27b35752f537f0fd04ee2dc4 +size 9040 diff --git a/docs/intelligentapps/images/profiling/the-next-session-admin.png b/docs/intelligentapps/images/profiling/the-next-session-admin.png new file mode 100644 index 0000000000..2d62b6db5e --- /dev/null +++ b/docs/intelligentapps/images/profiling/the-next-session-admin.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:27681e458415d25f1e72e7db134f356c25db3c63fc44ffcd4ea6864ad40516c1 +size 6845 diff --git a/docs/intelligentapps/images/profiling/the-next-session.png b/docs/intelligentapps/images/profiling/the-next-session.png new file mode 100644 index 0000000000..f3cb254cfd --- /dev/null +++ b/docs/intelligentapps/images/profiling/the-next-session.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ea38ee6951b75ed3e15239113a8ed1f4f0b1abc216af09382e9a5307795e019 +size 77916 diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md new file mode 100644 index 0000000000..1412393200 --- /dev/null +++ b/docs/intelligentapps/profiling.md @@ -0,0 +1,85 @@ +# Profiling App using Windows Machine Learning + +Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of process, ONNX model on different Execution Provider and Windows ML events. + +## Prerequisites + +- VS Code must be installed. Follow these steps to [set up VS Code](https://code.visualstudio.com/docs/setup/setup-overview). +- AI Toolkit extension must be installed. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). + +## Start with "The next session" + +In this mode, the profiling tool will profile the next newly start app that is sending out Windows ML events. +This option is ideal for testing a run-once app. In this case, you could start profiling and run the app and the resource usages will be shown up. + +![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) + +> [!Important] +> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. +> ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) + +> [!NOTE] +> It will profile the next newly start app. For example for python notebook, if the kernel is already running, you need to click restart kernel to make it restart. If not, running new ipynb will not be captured. + +## Start with "By Process ID or Name" + +In this mode, the profiling tool will profile the process that matches one of the following: + +- Process Id: like 12345 +- Process Name: usually name of app without `.exe`. The first match will be profiled +- Process Path: like `c:\Users\xxx\Inference.Service.Agent.exe`. The first match will be profiled + +This option is ideal for profiling an already running app. + +![Screenshot that shows how to start by process id or name](./images/profiling/by-process-id-or-name.png) + +## Start with "By Model File" + +In this mode, the profiling tool will profile an ONNX model file for the duration on target Execution Provider and you could see the resources it used during running. + +This option is ideal for profiling an ONNX model on different EPs. + +![Screenshot that shows how to start by model file]() + +After profiling, a report folder will be created with logs and data. + +![Screenshot that show the report data]() + +## Resource Usages View + +In the main window, the plot on the top will show usage of CPU, GPU, NPU and Memory. The usage will be updated once per second and kept for 10 minutes. You could use the tools on the top right to navigate the timeline by zoom in, zoom out and pan. + +![Screnshot that shows the resrouce usages view](./images/profiling/by-model-file.png) + +> [!NOTE] +> This features use Performance Counters. To achieve higher accuracy, you could try [Windows Performance Recorder](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder). + +## Windows ML Events View + +In the main window, the plot on the bottom will show Windows ML events. Its timeline is synced with the resource usages view, so you could easily know how resource is used during certain events. + +> [!Important] +> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. +> ![Secreenshot that show a notification to restart VS Code in admin mode]() + +Currently we will show events of the following types: + +- Ensure ExecutionProvider Ready: when Windows ML is preparing the EP +- Session Creation: when the session is created +- Inference: when the model is inferences on the session + +![Screnshot that shows the Windows ML events view]() + +## What you learned + +In this article, you learned how to: + +- Start profiling in different ways +- Inspect the Resource Usages View +- Inspect the Windows ML Events View + +## See also + +- [How to measure performance of AI models running locally](https://learn.microsoft.com/en-us/windows/ai/npu-devices/#how-to-measure-performance-of-ai-models-running-locally-on-the-device-npu) +- [Performance Monitor](https://learn.microsoft.com/en-us/troubleshoot/windows-server/support-tools/troubleshoot-issues-performance-monitor) +- [ONNX Runtime Tracing](https://onnxruntime.ai/docs/performance/tune-performance/logging_tracing.html) From 9259280ec1e9b4f2043f183e004ad41ac7e574e8 Mon Sep 17 00:00:00 2001 From: hualxie Date: Thu, 11 Dec 2025 13:35:34 +0800 Subject: [PATCH 02/24] update --- .../images/profiling/by-model-file-result.png | 3 +++ .../images/profiling/events-view-admin.png | 3 +++ .../images/profiling/resource-usage-view.png | 3 +++ docs/intelligentapps/profiling.md | 8 ++++---- 4 files changed, 13 insertions(+), 4 deletions(-) create mode 100644 docs/intelligentapps/images/profiling/by-model-file-result.png create mode 100644 docs/intelligentapps/images/profiling/events-view-admin.png create mode 100644 docs/intelligentapps/images/profiling/resource-usage-view.png diff --git a/docs/intelligentapps/images/profiling/by-model-file-result.png b/docs/intelligentapps/images/profiling/by-model-file-result.png new file mode 100644 index 0000000000..f0d252a1a6 --- /dev/null +++ b/docs/intelligentapps/images/profiling/by-model-file-result.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28aedd99cacddb3dcde848b738f6788be1a532b1b1c4dd375d77a9b1f3744dfd +size 13485 diff --git a/docs/intelligentapps/images/profiling/events-view-admin.png b/docs/intelligentapps/images/profiling/events-view-admin.png new file mode 100644 index 0000000000..4cbe1b9a49 --- /dev/null +++ b/docs/intelligentapps/images/profiling/events-view-admin.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed3264f6316a54c993bea46accb105777d130d149f3c9a320ea94068ebca4515 +size 6704 diff --git a/docs/intelligentapps/images/profiling/resource-usage-view.png b/docs/intelligentapps/images/profiling/resource-usage-view.png new file mode 100644 index 0000000000..f2cbf18269 --- /dev/null +++ b/docs/intelligentapps/images/profiling/resource-usage-view.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7666e8ef70c44f87fd8999ad0a0ac3731628de32ef14d4d891ec6df10c185651 +size 55940 diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 1412393200..1b17821a69 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -39,17 +39,17 @@ In this mode, the profiling tool will profile an ONNX model file for the duratio This option is ideal for profiling an ONNX model on different EPs. -![Screenshot that shows how to start by model file]() +![Screenshot that shows how to start by model file](./images/profiling/by-model-file.png) After profiling, a report folder will be created with logs and data. -![Screenshot that show the report data]() +![Screenshot that show the report data](./images/profiling/by-model-file-result.png) ## Resource Usages View In the main window, the plot on the top will show usage of CPU, GPU, NPU and Memory. The usage will be updated once per second and kept for 10 minutes. You could use the tools on the top right to navigate the timeline by zoom in, zoom out and pan. -![Screnshot that shows the resrouce usages view](./images/profiling/by-model-file.png) +![Screnshot that shows the resrouce usages view](./images/profiling/resource-usage-view.png) > [!NOTE] > This features use Performance Counters. To achieve higher accuracy, you could try [Windows Performance Recorder](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder). @@ -60,7 +60,7 @@ In the main window, the plot on the bottom will show Windows ML events. Its time > [!Important] > To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. -> ![Secreenshot that show a notification to restart VS Code in admin mode]() +> ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/events-view-admin.png) Currently we will show events of the following types: From 0b66b69a07646c725a64e9946a4d840d56fb9cd6 Mon Sep 17 00:00:00 2001 From: hualxie Date: Thu, 11 Dec 2025 13:45:49 +0800 Subject: [PATCH 03/24] add overview --- docs/intelligentapps/images/overview/profiling.png | 3 +++ docs/intelligentapps/images/profiling/events-view.png | 3 +++ docs/intelligentapps/overview.md | 1 + docs/intelligentapps/profiling.md | 2 +- 4 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 docs/intelligentapps/images/overview/profiling.png create mode 100644 docs/intelligentapps/images/profiling/events-view.png diff --git a/docs/intelligentapps/images/overview/profiling.png b/docs/intelligentapps/images/overview/profiling.png new file mode 100644 index 0000000000..66a7902ea1 --- /dev/null +++ b/docs/intelligentapps/images/overview/profiling.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bab8d3b6bb8694e57b39156a5eb706d5211f7c46675ff54d9fa1caaa4023b112 +size 75605 diff --git a/docs/intelligentapps/images/profiling/events-view.png b/docs/intelligentapps/images/profiling/events-view.png new file mode 100644 index 0000000000..036802b112 --- /dev/null +++ b/docs/intelligentapps/images/profiling/events-view.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:195eb95724dad97fd887ccb5bd2ff4e0b932fd10aa035f51a71107ff172def44 +size 11995 diff --git a/docs/intelligentapps/overview.md b/docs/intelligentapps/overview.md index 8348ffb045..55b20a41f0 100644 --- a/docs/intelligentapps/overview.md +++ b/docs/intelligentapps/overview.md @@ -21,6 +21,7 @@ AI Toolkit offers seamless integration with popular AI models from providers lik | [Fine-tuning](/docs/intelligentapps/finetune) | Customize and adapt models for specific domains and requirements. Train models locally with GPU support or leverage Azure Container Apps for cloud-based fine-tuning. | ![Screenshot showing the Fine-tuning interface with model adaptation and training controls](./images/overview/fine-tune.png) | | [Model Conversion](/docs/intelligentapps/modelconversion) | Convert, quantize, and optimize machine learning models for local deployment. Transform models from Hugging Face and other sources to run efficiently on Windows with CPU, GPU, or NPU acceleration. | ![Screenshot showing the Model Conversion interface with tools for optimizing and transforming AI models](./images/overview/conversion.png) | | [Tracing](/docs/intelligentapps/tracing) | Monitor and analyze the performance of your AI applications. Collect and visualize trace data to gain insights into model behavior and performance. | ![Screenshot showing the Tracing interface with tools for monitoring AI applications](./images/overview/tracing.png) | +| [Profiling](/docs/intelligentapps/profiling) | Diagnose the CPU, GPU, NPU resource usages of process, ONNX model on different Execution Provider and Windows ML events. | ![Screenshot showing the Profiling tool](./images/overview/profiling.png) | ## Who is AI Toolkit for? diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 1b17821a69..8879a48bd6 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -68,7 +68,7 @@ Currently we will show events of the following types: - Session Creation: when the session is created - Inference: when the model is inferences on the session -![Screnshot that shows the Windows ML events view]() +![Screnshot that shows the Windows ML events view](./images/profiling/events-view.png) ## What you learned From f0fe91b165468e1a307a4910c4de243bfd5950ab Mon Sep 17 00:00:00 2001 From: hualxie Date: Thu, 11 Dec 2025 13:57:50 +0800 Subject: [PATCH 04/24] add playground --- .../playground/parameters-show-resource-usage.png | 3 +++ .../images/playground/response-resource-usage.png | 3 +++ docs/intelligentapps/playground.md | 11 +++++++++++ 3 files changed, 17 insertions(+) create mode 100644 docs/intelligentapps/images/playground/parameters-show-resource-usage.png create mode 100644 docs/intelligentapps/images/playground/response-resource-usage.png diff --git a/docs/intelligentapps/images/playground/parameters-show-resource-usage.png b/docs/intelligentapps/images/playground/parameters-show-resource-usage.png new file mode 100644 index 0000000000..6d9afffd7a --- /dev/null +++ b/docs/intelligentapps/images/playground/parameters-show-resource-usage.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e498ad1e01100e490184b7bf103d0331222ee5766385a8284ebb33903c04a898 +size 1952 diff --git a/docs/intelligentapps/images/playground/response-resource-usage.png b/docs/intelligentapps/images/playground/response-resource-usage.png new file mode 100644 index 0000000000..d10d6fdae3 --- /dev/null +++ b/docs/intelligentapps/images/playground/response-resource-usage.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b6b244f6a9f6f81f5f4f939fcff71f6c22e8d0b522e57cd7f5f1d5dc8464fcee +size 4574 diff --git a/docs/intelligentapps/playground.md b/docs/intelligentapps/playground.md index 29c5526968..a0e9f87fda 100644 --- a/docs/intelligentapps/playground.md +++ b/docs/intelligentapps/playground.md @@ -61,6 +61,12 @@ The available parameters depend on the model type and publisher. Common paramete - Frequency Penalty: Reduces the likelihood of the model repeating the same tokens in the response. Higher values discourage repetition. - Presence Penalty: Encourages the model to introduce new topics or concepts in the response. Higher values promote diversity in the generated text. +### Show resource usage +For local model, there will an additional option 'Show resource usage'. When it is enabled, resource usage will be shown with each response. + +![Screenshot showing 'Show resource usage' in Model Preferences panel](./images/playground/parameters-show-resource-usage.png) + + ## Chat prompt ![Chat prompt input box in AI Toolkit playground showing an attachment icon for adding files.](./images/playground/chat-prompt.png) @@ -96,6 +102,11 @@ Available features in the model response area: - **Rendering**: Playground supports rendering of code snippets, images, markdown, LaTex and other content in the model's response. - **Token Count**: Displays the number of tokens used in the model's response. This helps you understand the cost and performance of the model. +### Resource usage +When 'Show resource usage' is enabled for local model, a summary of usage will be shown with each response. You could also switch to the [Profiling](/docs/intelligentapps/profiling) by clicking 'Usage details'. + +![Screenshot showing resource usage for one response](./images/playground/response-resource-usage.png) + ## Tool bar The tool bar in the playground provides quick access to various actions and features: From 6d9c1ac723415b42fc1fd715c859f7b63ae91aac Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 08:58:59 +0800 Subject: [PATCH 05/24] Update docs/intelligentapps/overview.md Co-authored-by: Nick Trogh --- docs/intelligentapps/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/overview.md b/docs/intelligentapps/overview.md index 55b20a41f0..e0322d4ee6 100644 --- a/docs/intelligentapps/overview.md +++ b/docs/intelligentapps/overview.md @@ -21,7 +21,7 @@ AI Toolkit offers seamless integration with popular AI models from providers lik | [Fine-tuning](/docs/intelligentapps/finetune) | Customize and adapt models for specific domains and requirements. Train models locally with GPU support or leverage Azure Container Apps for cloud-based fine-tuning. | ![Screenshot showing the Fine-tuning interface with model adaptation and training controls](./images/overview/fine-tune.png) | | [Model Conversion](/docs/intelligentapps/modelconversion) | Convert, quantize, and optimize machine learning models for local deployment. Transform models from Hugging Face and other sources to run efficiently on Windows with CPU, GPU, or NPU acceleration. | ![Screenshot showing the Model Conversion interface with tools for optimizing and transforming AI models](./images/overview/conversion.png) | | [Tracing](/docs/intelligentapps/tracing) | Monitor and analyze the performance of your AI applications. Collect and visualize trace data to gain insights into model behavior and performance. | ![Screenshot showing the Tracing interface with tools for monitoring AI applications](./images/overview/tracing.png) | -| [Profiling](/docs/intelligentapps/profiling) | Diagnose the CPU, GPU, NPU resource usages of process, ONNX model on different Execution Provider and Windows ML events. | ![Screenshot showing the Profiling tool](./images/overview/profiling.png) | +| [Profiling](/docs/intelligentapps/profiling) | Diagnose the CPU, GPU, NPU resource usages of the process, ONNX model on different execution providers, and Windows ML events. | ![Screenshot showing the Profiling tool](./images/overview/profiling.png) | ## Who is AI Toolkit for? From ff0a8a1d679730fa133bcaaf3061c88e83043129 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 08:59:17 +0800 Subject: [PATCH 06/24] Update docs/intelligentapps/playground.md Co-authored-by: Nick Trogh --- docs/intelligentapps/playground.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/intelligentapps/playground.md b/docs/intelligentapps/playground.md index a0e9f87fda..55fe2f598e 100644 --- a/docs/intelligentapps/playground.md +++ b/docs/intelligentapps/playground.md @@ -62,7 +62,8 @@ The available parameters depend on the model type and publisher. Common paramete - Presence Penalty: Encourages the model to introduce new topics or concepts in the response. Higher values promote diversity in the generated text. ### Show resource usage -For local model, there will an additional option 'Show resource usage'. When it is enabled, resource usage will be shown with each response. + +For local models, there's an additional option **Show resource usage**. When enabled, resource usage is shown with each response. ![Screenshot showing 'Show resource usage' in Model Preferences panel](./images/playground/parameters-show-resource-usage.png) From 526ac705a9bf4b47e63dff0e1e1048f0f56e5616 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 08:59:45 +0800 Subject: [PATCH 07/24] Update docs/intelligentapps/playground.md Co-authored-by: Nick Trogh --- docs/intelligentapps/playground.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/intelligentapps/playground.md b/docs/intelligentapps/playground.md index 55fe2f598e..b6728bc889 100644 --- a/docs/intelligentapps/playground.md +++ b/docs/intelligentapps/playground.md @@ -104,7 +104,8 @@ Available features in the model response area: - **Token Count**: Displays the number of tokens used in the model's response. This helps you understand the cost and performance of the model. ### Resource usage -When 'Show resource usage' is enabled for local model, a summary of usage will be shown with each response. You could also switch to the [Profiling](/docs/intelligentapps/profiling) by clicking 'Usage details'. + +When **Show resource usage** is enabled for a local model, a summary of usage is shown with each response. You could also switch to the [Profiling](/docs/intelligentapps/profiling) details by selecting **Usage details**. ![Screenshot showing resource usage for one response](./images/playground/response-resource-usage.png) From cd13998211d34a56d88d1511eb536ce550722d2a Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 08:59:58 +0800 Subject: [PATCH 08/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 8879a48bd6..35f261b16c 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -1,4 +1,4 @@ -# Profiling App using Windows Machine Learning +# Profiling an app using Windows Machine Learning Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of process, ONNX model on different Execution Provider and Windows ML events. From 3edc3f21e1ccb51b620edcb226fe9f336e6941e6 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:00:11 +0800 Subject: [PATCH 09/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 35f261b16c..7e7482e242 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -62,7 +62,7 @@ In the main window, the plot on the bottom will show Windows ML events. Its time > To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. > ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/events-view-admin.png) -Currently we will show events of the following types: +Currently, we only show events of the following types: - Ensure ExecutionProvider Ready: when Windows ML is preparing the EP - Session Creation: when the session is created From 3d1edcd22f84528c147bf6d521ad1812069224bc Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:00:48 +0800 Subject: [PATCH 10/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 7e7482e242..fc09ef47bc 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -66,7 +66,7 @@ Currently, we only show events of the following types: - Ensure ExecutionProvider Ready: when Windows ML is preparing the EP - Session Creation: when the session is created -- Inference: when the model is inferences on the session +- Inference: when the model inferences on the session ![Screnshot that shows the Windows ML events view](./images/profiling/events-view.png) From f6349ac0f8d0eb6d081ff41294adde567cedb745 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:00:58 +0800 Subject: [PATCH 11/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index fc09ef47bc..f2020df6e8 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -68,7 +68,7 @@ Currently, we only show events of the following types: - Session Creation: when the session is created - Inference: when the model inferences on the session -![Screnshot that shows the Windows ML events view](./images/profiling/events-view.png) +![Screenshot that shows the Windows ML events view](./images/profiling/events-view.png) ## What you learned From 294c9ca66d70eb2c7e729f78dca1d8e99531b80a Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:01:10 +0800 Subject: [PATCH 12/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index f2020df6e8..745d86c935 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -75,7 +75,7 @@ Currently, we only show events of the following types: In this article, you learned how to: - Start profiling in different ways -- Inspect the Resource Usages View +- Inspect the Resource Usages view - Inspect the Windows ML Events View ## See also From 4bbdab6c9b6e9af12b1d273db443636fb3c7e34a Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:01:23 +0800 Subject: [PATCH 13/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 745d86c935..25bd0ed35e 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -76,7 +76,7 @@ In this article, you learned how to: - Start profiling in different ways - Inspect the Resource Usages view -- Inspect the Windows ML Events View +- Inspect the Windows ML Events view ## See also From 6c68efff3815db4a421d116f0339a97bda8b13b1 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:02:47 +0800 Subject: [PATCH 14/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 25bd0ed35e..f6dbccab25 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -1,6 +1,6 @@ # Profiling an app using Windows Machine Learning -Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of process, ONNX model on different Execution Provider and Windows ML events. +Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of processes, ONNX model on different execution providers, and Windows ML events. ## Prerequisites From 08b401b7f1fdf49283e1c3f1aa7edbf3e8134dc3 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:03:31 +0800 Subject: [PATCH 15/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index f6dbccab25..166480ac1c 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -4,7 +4,7 @@ Profiling is a tool designed to help developers and AI engineers to diagnose the ## Prerequisites -- VS Code must be installed. Follow these steps to [set up VS Code](https://code.visualstudio.com/docs/setup/setup-overview). +- Install the latest version of [Visual Studio Code](/download). - AI Toolkit extension must be installed. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). ## Start with "The next session" From f19cee52222c9757af5abd944c6d82a17ad75af2 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:04:11 +0800 Subject: [PATCH 16/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 166480ac1c..53ad5315bf 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -5,7 +5,7 @@ Profiling is a tool designed to help developers and AI engineers to diagnose the ## Prerequisites - Install the latest version of [Visual Studio Code](/download). -- AI Toolkit extension must be installed. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). +- Install the AI Toolkit VS Code extension. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). ## Start with "The next session" From efc4a894a4e40dce7ce7245919727f656fe55a38 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:05:09 +0800 Subject: [PATCH 17/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 53ad5315bf..33c27d868b 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -9,7 +9,7 @@ Profiling is a tool designed to help developers and AI engineers to diagnose the ## Start with "The next session" -In this mode, the profiling tool will profile the next newly start app that is sending out Windows ML events. +In this mode, the profiling tool profiles the next app that is started and that is sending out Windows ML events. This option is ideal for testing a run-once app. In this case, you could start profiling and run the app and the resource usages will be shown up. ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) From bbaaaa483747d900e6b8993396a0f488da201d4a Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:06:02 +0800 Subject: [PATCH 18/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 33c27d868b..81a80ecc41 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -10,7 +10,7 @@ Profiling is a tool designed to help developers and AI engineers to diagnose the ## Start with "The next session" In this mode, the profiling tool profiles the next app that is started and that is sending out Windows ML events. -This option is ideal for testing a run-once app. In this case, you could start profiling and run the app and the resource usages will be shown up. +This option is ideal for testing a run-once app. In this case, you start profiling, then run the app, and the resource usages will begin showing up. ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) From 23700bb50f2835dcaad3e59e915062fe949b20be Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:06:34 +0800 Subject: [PATCH 19/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 81a80ecc41..270fb70577 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -14,7 +14,7 @@ This option is ideal for testing a run-once app. In this case, you start profili ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) -> [!Important] +> [!IMPORTANT] > To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. > ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) From b52cec9b84ed1b9ceb4f83660d75b3f1af249307 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:08:16 +0800 Subject: [PATCH 20/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 270fb70577..c9e07d9067 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -15,7 +15,7 @@ This option is ideal for testing a run-once app. In this case, you start profili ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) > [!IMPORTANT] -> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. +> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification shows up and guides you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. > ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) > [!NOTE] From c5246fd67dbef8cff183c00abb2c8f7436519d2c Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:08:44 +0800 Subject: [PATCH 21/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index c9e07d9067..3c8c07a1ca 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -21,7 +21,7 @@ This option is ideal for testing a run-once app. In this case, you start profili > [!NOTE] > It will profile the next newly start app. For example for python notebook, if the kernel is already running, you need to click restart kernel to make it restart. If not, running new ipynb will not be captured. -## Start with "By Process ID or Name" +## Profile a running app In this mode, the profiling tool will profile the process that matches one of the following: From 636f4877495fc504b5dfc7c748aff9df81b7338e Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:11:49 +0800 Subject: [PATCH 22/24] Apply suggestions from code review Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 38 +++++++++++++++---------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 3c8c07a1ca..1a392cece1 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -7,60 +7,60 @@ Profiling is a tool designed to help developers and AI engineers to diagnose the - Install the latest version of [Visual Studio Code](/download). - Install the AI Toolkit VS Code extension. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). -## Start with "The next session" +## Profile on app startup In this mode, the profiling tool profiles the next app that is started and that is sending out Windows ML events. This option is ideal for testing a run-once app. In this case, you start profiling, then run the app, and the resource usages will begin showing up. ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) +The tool starts profiling a newly started app. This means that for profiling a Python notebook, if the kernel is already running, you need to restart the kernel to begin profiling for it. Just starting a new notebook does not automatically start profiling. > [!IMPORTANT] > To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification shows up and guides you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. -> ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) +> +> ![Screenshot that shows a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) -> [!NOTE] -> It will profile the next newly start app. For example for python notebook, if the kernel is already running, you need to click restart kernel to make it restart. If not, running new ipynb will not be captured. ## Profile a running app -In this mode, the profiling tool will profile the process that matches one of the following: +In this mode, the profiling tool starts profiling an already running app. You can select a process based on these criteria: -- Process Id: like 12345 -- Process Name: usually name of app without `.exe`. The first match will be profiled -- Process Path: like `c:\Users\xxx\Inference.Service.Agent.exe`. The first match will be profiled +- Process ID: like 12345 +- Process name: usually the name of app without `.exe`. The first match will be profiled. +- Process path: like `c:\Users\xxx\Inference.Service.Agent.exe`. The first match will be profiled. This option is ideal for profiling an already running app. ![Screenshot that shows how to start by process id or name](./images/profiling/by-process-id-or-name.png) -## Start with "By Model File" +## Profile an ONNX model -In this mode, the profiling tool will profile an ONNX model file for the duration on target Execution Provider and you could see the resources it used during running. +In this mode, the profiling tool starts profiling an ONNX model file on a target execution provider (EP) for a given duration. You can see the resource usage while it's running. This option is ideal for profiling an ONNX model on different EPs. ![Screenshot that shows how to start by model file](./images/profiling/by-model-file.png) -After profiling, a report folder will be created with logs and data. +After profiling, a report folder is created with logs and data. ![Screenshot that show the report data](./images/profiling/by-model-file-result.png) -## Resource Usages View +## Resource Usages view -In the main window, the plot on the top will show usage of CPU, GPU, NPU and Memory. The usage will be updated once per second and kept for 10 minutes. You could use the tools on the top right to navigate the timeline by zoom in, zoom out and pan. +In the main window, the plot on the top shows usage of CPU, GPU, NPU, and memory. The usage is updated every second, and kept for 10 minutes. You can use the tools on the top right to navigate the timeline by zooming in, zooming out, and panning. -![Screnshot that shows the resrouce usages view](./images/profiling/resource-usage-view.png) +![Screenshot that shows the resource usages view](./images/profiling/resource-usage-view.png) > [!NOTE] -> This features use Performance Counters. To achieve higher accuracy, you could try [Windows Performance Recorder](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder). +> This feature uses performance counters. To achieve higher accuracy, you could also try [Windows Performance Recorder](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder). -## Windows ML Events View +## Windows ML Events view -In the main window, the plot on the bottom will show Windows ML events. Its timeline is synced with the resource usages view, so you could easily know how resource is used during certain events. +In the main window, the plot on the bottom shows Windows ML events. Its timeline is synced with the Resource Usages view, so you can easily determine how resources are used when certain events occur. > [!Important] -> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification will show up and guide you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. -> ![Secreenshot that show a notification to restart VS Code in admin mode](./images/profiling/events-view-admin.png) +> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification shows up and guides you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. +> ![Screenshot that shows a notification to restart VS Code in admin mode](./images/profiling/events-view-admin.png) Currently, we only show events of the following types: From 2462affcee99baa6859948d4f558106f87db2c24 Mon Sep 17 00:00:00 2001 From: xieofxie Date: Fri, 12 Dec 2025 09:12:17 +0800 Subject: [PATCH 23/24] Update docs/intelligentapps/profiling.md Co-authored-by: Nick Trogh --- docs/intelligentapps/profiling.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 1a392cece1..896678596d 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -29,7 +29,7 @@ In this mode, the profiling tool starts profiling an already running app. You ca - Process name: usually the name of app without `.exe`. The first match will be profiled. - Process path: like `c:\Users\xxx\Inference.Service.Agent.exe`. The first match will be profiled. -This option is ideal for profiling an already running app. +This option is ideal for profiling an ap that is already running and you're unable to restart it for profiling purposes. ![Screenshot that shows how to start by process id or name](./images/profiling/by-process-id-or-name.png) From eaf4f1b91aa15b1d9c34b62f257540e9ae2b2dbe Mon Sep 17 00:00:00 2001 From: hualxie Date: Fri, 12 Dec 2025 09:27:12 +0800 Subject: [PATCH 24/24] more fix --- docs/intelligentapps/modelconversion.md | 4 ++-- docs/intelligentapps/profiling.md | 11 +++++++++-- docs/toc.json | 1 + 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/intelligentapps/modelconversion.md b/docs/intelligentapps/modelconversion.md index 95d88a8d2b..cf4304ff19 100644 --- a/docs/intelligentapps/modelconversion.md +++ b/docs/intelligentapps/modelconversion.md @@ -9,8 +9,8 @@ Model conversion is an integrated development environment designed to help devel ## Prerequisites -- VS Code must be installed. Follow these steps to [set up VS Code](https://code.visualstudio.com/docs/setup/setup-overview). -- AI Toolkit extension must be installed. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). +- Install the latest version of [Visual Studio Code](/download). +- Install the AI Toolkit VS Code extension. For more information, see [install AI Toolkit](/docs/intelligentapps/overview.md#install-and-setup). ## Create project diff --git a/docs/intelligentapps/profiling.md b/docs/intelligentapps/profiling.md index 896678596d..575cd763d8 100644 --- a/docs/intelligentapps/profiling.md +++ b/docs/intelligentapps/profiling.md @@ -1,7 +1,14 @@ +--- +ContentId: 8105f83b-8291-467e-abac-2344b4f368cd +DateApproved: 12/14/2025 +MetaDescription: Profiling Quickstart in AI Toolkit. +--- # Profiling an app using Windows Machine Learning Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of processes, ONNX model on different execution providers, and Windows ML events. +In this article, you could learn how to start profiling and how to inspect the resource usages view and the events view. + ## Prerequisites - Install the latest version of [Visual Studio Code](/download). @@ -15,9 +22,9 @@ This option is ideal for testing a run-once app. In this case, you start profili ![Screenshot that shows how to start by the next session](./images/profiling/the-next-session.png) The tool starts profiling a newly started app. This means that for profiling a Python notebook, if the kernel is already running, you need to restart the kernel to begin profiling for it. Just starting a new notebook does not automatically start profiling. + > [!IMPORTANT] > To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification shows up and guides you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work. -> > ![Screenshot that shows a notification to restart VS Code in admin mode](./images/profiling/the-next-session-admin.png) @@ -29,7 +36,7 @@ In this mode, the profiling tool starts profiling an already running app. You ca - Process name: usually the name of app without `.exe`. The first match will be profiled. - Process path: like `c:\Users\xxx\Inference.Service.Agent.exe`. The first match will be profiled. -This option is ideal for profiling an ap that is already running and you're unable to restart it for profiling purposes. +This option is ideal for profiling an app that is already running and you're unable to restart it for profiling purposes. ![Screenshot that shows how to start by process id or name](./images/profiling/by-process-id-or-name.png) diff --git a/docs/toc.json b/docs/toc.json index 7b86a4502f..ac32ded33f 100644 --- a/docs/toc.json +++ b/docs/toc.json @@ -395,6 +395,7 @@ ["Fine-tuning (Project Template)", "/docs/intelligentapps/finetune-legacy"], ["Model Conversion", "/docs/intelligentapps/modelconversion"], ["Tracing", "/docs/intelligentapps/tracing"], + ["Profiling", "/docs/intelligentapps/profiling"], ["FAQ", "/docs/intelligentapps/faq"], ["", "", { "name": "Reference",