Skip to content

Update benchmarking.md #42

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Desoky231 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update benchmarking.md #42

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/concepts/benchmarking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ Collections of tasks can be published as _benchmarking suites_. Seamlessly integ
- standardized train-test splits are provided to ensure that results can be objectively compared - results can be shared in a reproducible way through the APIs
- results from other users can be easily downloaded and reused

You can search for <a href="https://www.openml.org/search?type=benchmark&sort=tasks_included&study_type=task" target="_blank">all existing benchmarking suites</a> or create your own. For all further details, see the [benchmarking guide](../benchmark/benchmark.md).
You can search for <a href="https://www.openml.org/search?type=benchmark&sort=tasks_included&study_type=task" target="_blank">all existing benchmarking suites</a> or create your own. For all further details, see the [benchmarking guide](../benchmark/index.md).

<img src="../../img/studies.png" style="width:100%; max-width:800px;"/>
<img src="../img/studies.png" style="width:100%; max-width:800px;"/>

## Benchmark studies
Collections of runs can be published as _benchmarking studies_. They contain the results of all runs (possibly millions) executed on a specific benchmarking suite. OpenML allows you to easily download all such results at once via the APIs, but also visualized them online in the Analysis tab (next to the complete list of included tasks and runs). Below is an example of <a href="https://www.openml.org/search?type=benchmark&study_type=run&id=226" target="_blamnk">a benchmark study for AutoML algorithms</a>.

<img src="../../img/run_study.png" style="width:100%; max-width:1000px;"/>
<img src="../img/run_study.png" style="width:100%; max-width:1000px;"/>