From dd6df09bd783643d3f8b8b5a3ebd42b999c634c6 Mon Sep 17 00:00:00 2001 From: Priyanshu Chhazed Date: Sun, 15 Aug 2021 17:04:57 +0530 Subject: [PATCH 01/24] Added first three sections --- .../getting-started/index.md | 11 +++ .../getting-started/python/index.md | 15 +++ .../getting-started/python/installation.md | 95 +++++++++++++++++++ docs/test acceleration/index.md | 29 ++++++ docs/test acceleration/system-requirements.md | 35 +++++++ 5 files changed, 185 insertions(+) create mode 100644 docs/test acceleration/getting-started/index.md create mode 100644 docs/test acceleration/getting-started/python/index.md create mode 100644 docs/test acceleration/getting-started/python/installation.md create mode 100644 docs/test acceleration/index.md create mode 100644 docs/test acceleration/system-requirements.md diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md new file mode 100644 index 0000000..c453aab --- /dev/null +++ b/docs/test acceleration/getting-started/index.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Getting started +nav_order: 3 +has_children: true +permalink: /test-acceleration/getting-started +--- + +# Getting started +This document will guide you to install and learn to use YourBase Test Acceleration library in the language and the testing framework of your choice. + diff --git a/docs/test acceleration/getting-started/python/index.md b/docs/test acceleration/getting-started/python/index.md new file mode 100644 index 0000000..6e8dfb2 --- /dev/null +++ b/docs/test acceleration/getting-started/python/index.md @@ -0,0 +1,15 @@ +--- +layout: default +title: Python +nav_order: 1 +parent: Getting started +has_children: true +permalink: /test-acceleration/getting-started/python +--- + +# YourBase Test Acceleration in Python + +This page guides you to install the YourBase Test Acceleration library in your Python environment. + +Make sure that: +YourBase Test Acceleration library supports your technical stack and infrastructure [link to the System Requirements section]. diff --git a/docs/test acceleration/getting-started/python/installation.md b/docs/test acceleration/getting-started/python/installation.md new file mode 100644 index 0000000..f8f64bb --- /dev/null +++ b/docs/test acceleration/getting-started/python/installation.md @@ -0,0 +1,95 @@ +--- +layout: default +title: Installation +nav_order: 2 +parent: Getting started +has_children: true +permalink: /test-acceleration/getting-started/python/install +--- + +# Python installation +{: .no_toc } + +## Table of contents +{: .no_toc } + +1. TOC +{ :toc } + +## Prerequisites +- YourBase Test Acceleration library supports your technical stack and infrastructure [link to the System Requirements section]. + +## Installation + +YourBase Test Acceleration library can be installed with either pip or poetry package managers. + +### Using pip +If you use pip, you can install YourBase Test Acceleration with: + +```pip install yourbase``` + +To check if YourBase Test Acceleration is installed, run the following command in a shell prompt: + +```pip show yourbase``` + +If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. + +``` +Name: yourbase +Version: 5.2.4 +Summary: Skip tests based on tracing data +Home-page: https://yourbase.io +Author: YourBase Test Acceleration +Author-email: python@yourbase.io +License: UNKNOWN +Location: /Path/Python/3.8/lib/python/site-packages +Requires: six, boto3, python-dateutil, coverage, pastel, requests +Required-by: +``` + + +### Using poetry +If you use poetry, you can install YourBase Test Acceleration with: + +```poetry add yourbase``` + +To check if YourBase Test Acceleration is installed, run the following command in a shell prompt: + +```poetry show yourbase``` + +If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. + +``` +Name: yourbase +Version: 5.2.4 +Summary: Skip tests based on tracing data +Home-page: https://yourbase.io +Author: YourBase Test Acceleration +Author-email: python@yourbase.io +License: UNKNOWN +Location: /Path/Python/3.8/lib/python/site-packages +Requires: six, boto3, python-dateutil, coverage, pastel, requests +Required-by: +``` + +## Installation recommendations + +### Use virtual environment + +{: .no_toc } + +We recommend that you install YourBase Test Acceleration in a clean virtual environment. See here to learn how to set up a Python virtual environment. + +## Uninstall +If for any reason, you want to completely remove YourBase Test Acceleration, simply uninstall the package: + +Using pip: + +```pip uninstall yourbase``` + +Or, using poetry: + +```poetry remove yourbase``` + + + diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md new file mode 100644 index 0000000..dcb1768 --- /dev/null +++ b/docs/test acceleration/index.md @@ -0,0 +1,29 @@ +--- +layout: default +title: Introduction +parent: Home +nav_order: 1 +has_children: true +permalink: /test-acceleration +--- + +# Introduction +YourBase Test Acceleration is a library for shortening your test run-times by up to 90%. It hooks into your testing framework to intelligently select which tests should be run for a given code change. + +### How does YourBase Test Acceleration reduce test times? +At the core of YourBase Test Acceleration is the YourBase Dependency Graph that maps which files and functions each of your tests depends on. Every time you run your tests, YourBase Test Acceleration library will load the optimal dependency graph to select and run only the tests that pertain to your code changes, and safely avoid running any unrelated tests. As a result, your builds execute only the optimal fraction of your total tests and finish much faster. + +### Is YourBase Test Acceleration library compatible with your tech-stack and infrastructure? +The library currently supports testing frameworks for Python and Ruby. See here for a complete list of system requirements [link to the System Requirements section]. + +### Which type of tests does YourBase Test Acceleration library accelerate? +The library supports unit tests and integration tests that call code from within the application. Since the library traces dependencies from within the test-runner process, distributed dependencies like network or database calls are not fully traced and accelerated. + +### Will YourBase Test Acceleration library provide benefits at your scale? +The library has the most benefits for use-cases where test runs are taking over 10 minutes. Our customers have reduced their test run times by up to 90%, for example, where one customer was able to skip over 99% of the 11,000+ tests in the suite using our library. + +### Is YourBase Test Acceleration library secure? +Yes. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. See here for more details [link to the Security section]. + +### Is YourBase Test Acceleration library stable? +The library is currently in Beta. While we are confident in the reliability of our offering, we are making improvements all the time. If you identify any issues, please let us know at support@yourbase.io and we will look into them quickly. diff --git a/docs/test acceleration/system-requirements.md b/docs/test acceleration/system-requirements.md new file mode 100644 index 0000000..628a9db --- /dev/null +++ b/docs/test acceleration/system-requirements.md @@ -0,0 +1,35 @@ +--- +layout: default +title: Getting Started +nav_order: 2 +parent: Introduction +has_children: false +permalink: /test-acceleration/system-requirements +--- + +# System requirements + +YourBase Test Acceleration library works seamlessly with the CI system of your choice and is agnostic to where you host your application—be it cloud, on-premise, or offline. + +### Supported version control tools +The library can currently accelerate tests for codebases that are version-controlled using Git. + +### Supported platforms +- Linux +- MacOS: All except those with the M1 chip + +### Supported languages & testing frameworks +- Python 2.7+ and Python 3.5+ that use the following testing frameworks: + - pytest + - unittest + + Note: Any web frameworks, such as Django, that are built atop these, are also supported. + +### Compatible code coverage tools +The library supports integration with the following code coverage reporting tools without changing your workflow, for example: +- Coverage [Link to Advanced usage section that guides on how to configure it for proper coverage reports] +- pytest-cov +- SonarQube +- Codecov + +Reach out to support@yourbase.io if you have questions about support for your favorite coverage tool. \ No newline at end of file From f50cb41c6fcd05f8fec6e3e89c6db79cec85b4c6 Mon Sep 17 00:00:00 2001 From: Priyanshu Chhazed Date: Sun, 15 Aug 2021 19:03:46 +0530 Subject: [PATCH 02/24] Added all test acceleration files --- .../accelerate-parallelized-tests.md | 35 ++++ .../accelerate-tests-across-developers.md | 11 + .../advanced-usage/accelerate-tests-in-ci.md | 67 +++++++ .../force-run-specific-tests.md | 42 ++++ .../test acceleration/advanced-usage/index.md | 11 + .../integrate-code-coverage-tools.md | 32 +++ .../advanced-usage/verify-results.md | 22 ++ docs/test acceleration/deploying.md | 81 ++++++++ .../getting-started/index.md | 1 + .../getting-started/python/index.md | 3 +- .../getting-started/python/installation.md | 1 + .../python/testing frameworks/index.md | 11 + .../python/testing frameworks/pytest.md | 94 +++++++++ .../python/testing frameworks/unittest.md | 57 ++++++ docs/test acceleration/how-it-works.md | 16 ++ .../reference/common-issues.md | 78 ++++++++ .../reference/configuration-options.md | 188 ++++++++++++++++++ docs/test acceleration/reference/index.md | 11 + docs/test acceleration/releases.md | 12 ++ docs/test acceleration/security.md | 17 ++ 20 files changed, 789 insertions(+), 1 deletion(-) create mode 100644 docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md create mode 100644 docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md create mode 100644 docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md create mode 100644 docs/test acceleration/advanced-usage/force-run-specific-tests.md create mode 100644 docs/test acceleration/advanced-usage/index.md create mode 100644 docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md create mode 100644 docs/test acceleration/advanced-usage/verify-results.md create mode 100644 docs/test acceleration/deploying.md create mode 100644 docs/test acceleration/getting-started/python/testing frameworks/index.md create mode 100644 docs/test acceleration/getting-started/python/testing frameworks/pytest.md create mode 100644 docs/test acceleration/getting-started/python/testing frameworks/unittest.md create mode 100644 docs/test acceleration/how-it-works.md create mode 100644 docs/test acceleration/reference/common-issues.md create mode 100644 docs/test acceleration/reference/configuration-options.md create mode 100644 docs/test acceleration/reference/index.md create mode 100644 docs/test acceleration/releases.md create mode 100644 docs/test acceleration/security.md diff --git a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md new file mode 100644 index 0000000..572b980 --- /dev/null +++ b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md @@ -0,0 +1,35 @@ +--- +layout: default +title: Accelerate parallelized tests +nav_order: 7 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/accelerate-parallelized-tests +--- + +# Accelerate parallelized tests + +YourBase Test Acceleration integrates smoothly with your existing parallelization setup in your CI. On top of your parallelization setup, YourBase Test Acceleration will: +1. Split your tests across shards +2. Accelerate tests running on each shard + +## Prerequisites: +- You’re already successfully running tests across multiple shards. +- You've removed any test splitting solution that you may already have in-place. This is required because YourBase Test Acceleration is effective only when the test-splitting solution is sticky—tests don’t get reshuffled across shards when other tests are added or removed. However, since most test-splitting solutions aren’t sticky, YourBase Test Acceleration provides its own built-in sticky test-splitting solution, which you will learn to use in the following section. + +## Steps to accelerate already-parallelized tests: +1. Remove your existing test splitting tools, if any. It should appear as if each shard will run the entire test suite. +2. Set YOURBASE_COHORT_COUNT [link to References → Configuration → YOURBASE_COHORT_COUNT] to your number of cohorts / shards +3. Set YOURBASE_ACTIVE_COHORT [link to References → Configuration → YOURBASE_ACTIVE_COHORT] to the ID of the current cohort / shard – starting from 1. For example, if you have 5 shards and are running the 4th shard, you’d set: + + ``` + YOURBASE_COHORT_COUNT = 5 + YOURBASE_ACTIVE_COHORT = 4 + ``` + +4. Run your tests as usual. + +YourBase Test Acceleration will now just-in-time select and run only the tests that are in the current cohort. This selection will be consistent between runs—given the same cohort ID and total cohorts, a test will always be selected to be run on the same shard for life. + +## Pro tip +As an added benefit of YouBase’s sticky splitting of tests, you can schedule a build for each of the sharded pools of tests—to run sharded builds. This works smoothly because YourBase Test Acceleration will merge graphs from multiple shards, for the same commit, which can be used in future builds. diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md new file mode 100644 index 0000000..65699e7 --- /dev/null +++ b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Accelerate tests locally across developers +nav_order: 6 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/accelerate-tests-locally-across-developers +--- + +# Accelerate tests locally across developers +To have developers, who are working on the same branch, access a shared dependency graph so that tests run faster for them all, you can set them up to use the remote cache following the same steps as in Advanced usage → Accelerate tests in CI for your development environment. \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md new file mode 100644 index 0000000..2d3806f --- /dev/null +++ b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md @@ -0,0 +1,67 @@ +--- +layout: default +title: Accelerate tests in CI +nav_order: 5 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/accelerate-tests-in-ci +--- + +# Accelerate tests in CI +YourBase Test Acceleration harnesses its dependency graph to accelerate your tests. When you run tests on your local machine, with YourBase Test Acceleration enabled, by default, your dependency graph is stored locally—thus providing only local acceleration. + +But the true power of YourBase Test Acceleration comes in when tests can be accelerated in your CI environment. This requires the dependency graph to be accessible by your CI environment—referred to as the Shared Dependency Graph from now on. + +The following steps guide you to accelerate tests in CI: + +## Step 1: Set storage location for shared dependency graph +YourBase Test Acceleration currently supports storing shared graphs in AWS S3 buckets. The following sections help you set up a shared dependency graph for your project. + +### Using S3 bucket +Within your project, set the following environment variable to your S3 bucket using the `s3://` syntax. + +#### 1. Set S3 Bucket name + +```YOURBASE_REMOTE_CACHE=s3://[/key/prefix]``` + +where is an S3 bucket that your machine(s) has Get/Put/List access to. + +#### 2. Set AWS credentials +You can configure YourBase Test Acceleration to use either the system AWS credentials, or you can also specify different credentials that are specific to YourBase Test Acceleration. + +##### Option 1: Use the system AWS credentials +To use your systems’ AWS credentials, export the following in your environment: + +``` +export AWS_ACCESS_KEY_ID= +export AWS_SECRET_ACCESS_KEY= +``` + +##### Option 2: Use YourBase Test Acceleration-specific AWS credentials +If you want YourBase Test Acceleration to use different credentials (or if you're setting the system AWS credentials to mock values for your tests or CI), you can set these YourBase Test Acceleration-specific environment variables instead: + +``` +export YOURBASE_AWS_ACCESS_KEY_ID= +export YOURBASE_AWS_SECRET_ACCESS_KEY= +``` + +Note: + +By default, YourBase Test Acceleration uses the system AWS credentials - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. +However, if YourBase Test Acceleration specific credentials are set - `YOURBASE_AWS_ACCESS_KEY_ID` and `YOURBASE_AWS_SECRET_ACCESS_KEY`, YourBase Test Acceleration uses them instead of the system credentials. + +## Step 2: Install YourBase Test Acceleration in your CI environment +Add yourbase to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your CI environment. + +## Step 3: Run tests +Run tests as usual. + +The above steps will set up YourBase Test Acceleration to synchronize dependency graphs against the specified S3 location when your tests run on your CI environment. + +--- + +## Points to note +- Your code will never be uploaded to the configured bucket. Only the dependency graphs are uploaded to the bucket. Neither your code nor your dependency graphs will touch YourBase Test Acceleration servers. +- A dependency graph that is generated from successful builds and using clean working trees will be synchronized to the specified S3 location. This graph is accessed for future test runs if a local cache is not present. +- Dependency graphs run against uncommitted code changes will be stored only locally, i.e., they can’t be shared over S3. +- It is safe to share one bucket between multiple repositories, even if the repositories use different languages. diff --git a/docs/test acceleration/advanced-usage/force-run-specific-tests.md b/docs/test acceleration/advanced-usage/force-run-specific-tests.md new file mode 100644 index 0000000..266280f --- /dev/null +++ b/docs/test acceleration/advanced-usage/force-run-specific-tests.md @@ -0,0 +1,42 @@ +--- +layout: default +title: Force run specific tests +nav_order: 2 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/force-run-specific-tests +--- + +# Force run specific tests + +{: .no_toc } + +This feature is supported in the following testing frameworks: +- pytest +{:toc} +- unittest + +## pytest +You can tell YourBase Test Acceleration to never skip a specific test using decorators. + +``` +import pytest + +@pytest.mark.do_not_accelerate +def test_function(): + # ... +``` + +The decorator @pytest.mark.do_not_accelerate ensures that the test test_function() is never skipped by YourBase Test Acceleration, even where there are no code changes in its dependencies. + + +## unittest +You can tell YourBase Test Acceleration to never skip specific tests using decorators. + +```import yourbase.plugins.unittest as yourbase + +@yourbase.do_not_accelerate +class TestClass(unittest.TestCase): + def test_function(): + # ... +``` \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/index.md b/docs/test acceleration/advanced-usage/index.md new file mode 100644 index 0000000..a15703a --- /dev/null +++ b/docs/test acceleration/advanced-usage/index.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Advanced usage +nav_order: 5 +parent: Introduction +has_children: true +permalink: /test-acceleration/advanced-usage +--- + +# Advanced usage +This document will guide you to make use of advanced featuers of YourBase Test Acceleration. diff --git a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md new file mode 100644 index 0000000..90f9eaa --- /dev/null +++ b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md @@ -0,0 +1,32 @@ +--- +layout: default +title: Verify results +nav_order: 4 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/integrate-code-coverage-tools +--- + +# Integrate code coverage tools + +{: .no_toc } + +YourBase Test Acceleration supports the following code coverage tools. +- Coverage +{:toc} + +Note that since YourBase Test Acceleration is designed to avoid test runs that do not need to be executed, the percentage covered may appear lower than what is actually covered as a result. You can circumvent this by making configuration changes in your code coverage tool, so that YourBase Test Acceleration can fill missing coverage data from previous runs. +However, this feature of YourBase Test Acceleration works only on specific versions of code coverage tools. The following sections provide details on this. + +Note: If your CI is configured to fail a build based on the percentage covered, you may need to reconfigure it. + +## Coverage +YourBase Test Acceleration is compatible with all versions of Coverage out-of-the-box. To ensure that coverage reports account for skipped tests as well: +1. Use Coverage 5.5+ +2. Set the following in your `.coveragerc` file: +``` +[run] +relative_files = true +``` + +Note: Prior to Coverage 5.5, coverage reports will only include the tests run by YourBase Test Acceleration. Tests that were skipped by YourBase Test Acceleration will be omitted from coverage reporting, thereby decreasing your coverage percentage. diff --git a/docs/test acceleration/advanced-usage/verify-results.md b/docs/test acceleration/advanced-usage/verify-results.md new file mode 100644 index 0000000..59fe154 --- /dev/null +++ b/docs/test acceleration/advanced-usage/verify-results.md @@ -0,0 +1,22 @@ +--- +layout: default +title: Verify results +nav_order: 3 +parent: Advanced usage +grand_parent: Introduction +permalink: /test-acceleration/advanced-usage/verify-results +--- + +# Verify results +YourBase Test Acceleration test selection includes an "Observation Mode" which allows you to test-drive its test selection without actually skipping any tests. + +When “Observation Mode” is set, in addition to running all the tests, YourBase Test Acceleration will also monitor if its test selection would have skipped any tests that would have failed. As its output, it will print out the names of any tests that would have been incorrectly skipped. + +To verify results follow the below steps: + +1. Enable Observation Mode +Before running your tests, set `YOURBASE_OBSERVATION_MODE=true` in your environment. + +2. Check the output of running YourBase Test Acceleration in Observation mode + - If YourBase Test Acceleration is accelerating tests correctly, it will log the total amount of time that could have been saved, to stdout or your log file. + - Else if YourBase Test Acceleration is accelerating tests incorrectly—by skipping a test(s) that would have failed, YourBase Test Acceleration will complain about this loudly and output the details of the errors in your log file or your shell-prompt. Note that if this happens, it means that there is a bug in YourBase Test Acceleration’s tracing or acceleration. Please also report these to bugs@yourbase.io. \ No newline at end of file diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying.md new file mode 100644 index 0000000..08fe45d --- /dev/null +++ b/docs/test acceleration/deploying.md @@ -0,0 +1,81 @@ +--- +layout: default +title: Deploying +parent: Introduction +nav_order: 4 +permalink: /test-acceleration/deploying +--- + +# Deploying + +{: .no_toc } + +This section contains guides to deploy the YourBase Test Acceleration library. + +## Table of contents + +{: .no_toc } + +1. TOC +{ :toc } + +## Rollout recommendation +To safely accelerate tests on production, we recommend that you first run YourBase Test Acceleration in its Observation Mode [link to Advanced usage → Observation mode], and manually verify the output logged [link to Advanced usage → Observation mode → Checking output]. + +We propose the rollout phases to look like the following: + +### Phase 1: Test locally: On your development branch + +{: .no_toc } + +This phase will synchronize your code’s dependency graph on your local machine storage. Hence, after completing this phase, you’ll see the tests on your branch running faster only on your local machine. + +1. Install YourBase Test Acceleration locally [link to Installation] +2. Run YourBase Test Acceleration in Observation Mode [link to Advanced usage → Observation mode] +3. Ensure that YourBase Test Acceleration accelerates your tests correctly [link to Advanced usage → Observation mode → Checking output], or resolve any issues that arise +4. Disable Observation Mode [link to Reference → Configuration → Observation mode] +5. Run YourBase Test Acceleration + +### Phase 2: Test in CI + +This phase will synchronize your code’s dependency graph on remote storage for use by CI. Hence, after completing this phase, you’ll see the tests on your branch running faster on your CI as well. + +In your CI environment: + +1. Set up the Shared Dependency Graph for use by your CI [link to Advanced usage → Accelerate Using Shared Dependency Graph] +2. Install YourBase Test Acceleration to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your CI environment +3. Enable Observation Mode [link to Advanced usage → Observation mode] +4. Set up the following configuration variables for your CI environment: + - YOURBASE_LICENSE_KEY [link to configuration options section] + - YOURBASE_ACCEPT_TOS [link to configuration options section] +5. Run YourBase Test Acceleration for a subset of tests +6. Ensure that YourBase Test Acceleration accelerates these tests correctly [link to Advanced usage → Observation mode → Checking output], or resolve any issues that arise +7. Roll out to the remainder of tests in increments, resolving any potential issues that arise +8. Disable Observation Mode [link to Reference → Configuration → Observation mode] +9. Run YourBase Test Acceleration for all your tests in CI + +### Phase 3: Enable local test acceleration across your development team, starting with a group of beta users. + +After this phase, you’ll see tests run faster for developers across the team because of them sharing their dependency graph. + +1. Set up the Shared Dependency Graph for use by your local machine [link to Advanced usage → Accelerate Using Shared Dependency Graph] +2. Install YourBase Test Acceleration to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your local environment +3. Set up the following configuration variables for your local environment: + - YOURBASE_LICENSE_KEY [link to configuration options section] + - YOURBASE_ACCEPT_TOS` [link to configuration options section] + +--- + +## Logs +Our standard logs are all prefixed with [YB]. We print minimal logs unless explicitly requested using debug mode [Link to References → Configuration → YOURBASE_DEBUG]. + +To obtain more detailed debugging information, set the `YOURBASE_DEBUG` environment variable [link to Configuration Options section]. + +--- + +## Disable YourBase Test Acceleration +If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable `YOURBASE_DISABLE` to true: + +```export YOURBASE_DISABLE=true``` + +To uninstall the package, see uninstall instructions [Link to Uninstall] \ No newline at end of file diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md index c453aab..88eeae9 100644 --- a/docs/test acceleration/getting-started/index.md +++ b/docs/test acceleration/getting-started/index.md @@ -2,6 +2,7 @@ layout: default title: Getting started nav_order: 3 +parent: Introduction has_children: true permalink: /test-acceleration/getting-started --- diff --git a/docs/test acceleration/getting-started/python/index.md b/docs/test acceleration/getting-started/python/index.md index 6e8dfb2..e7fdc3e 100644 --- a/docs/test acceleration/getting-started/python/index.md +++ b/docs/test acceleration/getting-started/python/index.md @@ -3,8 +3,9 @@ layout: default title: Python nav_order: 1 parent: Getting started +grand_parent: Introduction has_children: true -permalink: /test-acceleration/getting-started/python +permalink: /test-acceleration/python --- # YourBase Test Acceleration in Python diff --git a/docs/test acceleration/getting-started/python/installation.md b/docs/test acceleration/getting-started/python/installation.md index f8f64bb..bfd05cd 100644 --- a/docs/test acceleration/getting-started/python/installation.md +++ b/docs/test acceleration/getting-started/python/installation.md @@ -3,6 +3,7 @@ layout: default title: Installation nav_order: 2 parent: Getting started +grand_parent: Introduction has_children: true permalink: /test-acceleration/getting-started/python/install --- diff --git a/docs/test acceleration/getting-started/python/testing frameworks/index.md b/docs/test acceleration/getting-started/python/testing frameworks/index.md new file mode 100644 index 0000000..c1c3b69 --- /dev/null +++ b/docs/test acceleration/getting-started/python/testing frameworks/index.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Python testing frameworks +nav_order: 1 +parent: Python +grand_parent: Getting started +permalink: /test-acceleration/python/testing-frameworks +--- + +# Basic usage with Python testing frameworks +This section gives an introduction on how to get started with running YourBase Test Acceleration in supported testing frameworks. For a full walkthrough of the tool, see [Link to Try in pytest] \ No newline at end of file diff --git a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md new file mode 100644 index 0000000..47ab681 --- /dev/null +++ b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md @@ -0,0 +1,94 @@ +--- +layout: default +title: Pytest +nav_order: 2 +parent: Python +grand_parent: Getting started +permalink: /test-acceleration/python/pytest +--- + +# Try in pytest + +This page will walk you through how to run YourBase Test Acceleration with the pytest testing framework. The quickest way to get started with it is to use it in the sample project provided. + +YourBase Test Acceleration’s pytest hooks ensure that, by default: +- It runs automatically on every invocation of pytest +- It runs without any additional configuration +- It runs irrespective of how you invoke the tests to run—be it via Makefile, Docker, or anything else—without any other setup. + +--- + +## Prerequisites +Make sure that, on your machine or on your virtual environment, you have: +- Tests running successfully with pytest before installing YourBase Test Acceleration +- YourBase Test Acceleration installed [link to the installation section of Python] +- You have git installed + +--- + +## Example usage in a sample project + +### Step 1: Set up the specified sample project on your machine as shown below: + +Open a new shell prompt, and checkout this sample project from git + +```git clone https://github.com/Unleash/unleash-client-python``` + +From your shell prompt, navigate to the directory where you checked out the project + +```cd unleash-client-python``` + +Run its `requirements.txt` file to install the required packages. + +```pip install -r requirements.txt``` + +### Step 2: Execute a command to run all the tests of the project + +```pytest tests``` + +Let’s look at the output. + +All messages that are prefixed with “[YB]” are logged by the YourBase Test Acceleration library. Note, that one of these messages says “No function-level dependency graph found; all tests will be run and traced.”—this is a cold run. During a cold run, YourBase Test Acceleration will build a dependency graph based on the tests that are executed. This dependency graph contains relationships between individual tests and the code they call. + +### Step 3: Without making any code change, execute the command to run all the tests again + +```pytest tests``` + +Let’s look at the messages logged by YourBase Test Acceleration. One of the messages says: “[YB] No code changed. Running only new tests, if any.” + +Since you ran the tests without changing any code, YourBase Test Acceleration skipped all the tests to finish the run much more quickly than the last time—compare the time taken by both runs. + +### Step 4: Make a code change and execute the command to run all tests again as shown below: + +Open ```tests/unit_tests/test_features.py``` in your text editor + +```vim tests/unit_tests/test_features.py``` + +Add the following print statement in the beginning of the method ```test_create_feature_true``` + +```print(“Checking YourBase Test Acceleration after a code-change...”)``` + +Run the tests again using: + +```pytest tests``` + +Let’s look at the output again. + +If you modified the same function as above then your output will closely match the output above. + +Look for the logs traced by YourBase Test Acceleration. You’ll see a message telling you how many functions have been altered and how many tests were affected. If you’re following the steps as is, you should see: + +``` +[YB] 1 function differs from the dependency graph +[YB] ~ tests/unit_tests/test_features.py#test_create_feature_true +[YB] Function-level dependency graph found 1 test affected +``` + +You can see form the logs that the existing dependency graph [link to How it works → Dependency graph section] was used to decide that only one test was affected by your code-change and that test was the only one that was run, while the remaining tests were skipped. + +--- + +## Conclusion +You just ran YourBase Test Acceleration on a project that uses the pytest testing framework. And now, you’re ready to use YourBase Test Acceleration in your own project. + +Note that if your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly. \ No newline at end of file diff --git a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md new file mode 100644 index 0000000..21af2a8 --- /dev/null +++ b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md @@ -0,0 +1,57 @@ +--- +layout: default +title: Pytest +nav_order: 3 +parent: Python +grand_parent: Getting started +permalink: /test-acceleration/python/unittest +--- + +# Try in unittest +This section notes how to run YourBase Test Acceleration in projects that use the unittest testing framework. For a deeper walkthrough with a sample open source project that uses pytest, see . + +--- + +## Prerequisites +Make sure that, on your machine or on your virtual environment, you have: +- Tests running successfully with unittest before installing YourBase Test Acceleration +- YourBase Test Acceleration installed [link to the installation section of Python] +- You have git installed + +--- + +## Example usage in your project + +### Step 1: Attach yourbase to unittest. + +For example, in your ```tests/__init__.py file```, copy-paste the following: + +``` +import unittest +import yourbase + +yourbase.attach(unittest) +``` + +### Step 2: Run your project’s tests + +For example, to run all the tests for the project, use: +```python -m unittest discover``` + +Note: If your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly. + + +###Step 3: Re-run your tests + +Without making a code change, run your tests again using the same command as in step #2. Here, YourBase Test acceleration ensures that no tests are run since no code was changed. + +### Step 4: Make a code change in any one of your tests. + +You can simply add a print statement like below: + +```print(“Checking YourBase Test Acceleration after a code-change...”)``` + + +### Step 5: Run your tests again using the same command as in step #2. + +Here, YourBase Test acceleration ensures that only the one test that you modified is run. diff --git a/docs/test acceleration/how-it-works.md b/docs/test acceleration/how-it-works.md new file mode 100644 index 0000000..24a27ff --- /dev/null +++ b/docs/test acceleration/how-it-works.md @@ -0,0 +1,16 @@ +--- +layout: default +title: How it works +parent: Introduction +nav_order: 3 +permalink: /test-acceleration/how-it-works +--- + +# How it works +The following documents clarify concepts, design decisions, and technical constraints in YourBase Test Acceleration. This is a great place to understand fundamental ideas and philosophy. + +## Dependency Graph +Learn more about the YourBase Test Acceleration Dependency Graph at https://yourbase.io/technology/dependency-graph/. + +## Shared Dependency Graph +Learn more about how to enable team-based acceleration gains with the YourBase Test Acceleration Dependency Graph at https://yourbase.io/technology/shared-graph. \ No newline at end of file diff --git a/docs/test acceleration/reference/common-issues.md b/docs/test acceleration/reference/common-issues.md new file mode 100644 index 0000000..04cf093 --- /dev/null +++ b/docs/test acceleration/reference/common-issues.md @@ -0,0 +1,78 @@ +--- +layout: default +title: Common issues +nav_order: 3 +parent: Reference +grand_parent: Introduction +permalink: /test-acceleration/reference/common-issues +--- + +# Common issues +{: .no_toc } + +This is a list of commonly encountered problems, known issues, and their solutions. + +## Table of contents +{: .no_toc .text-delta } + +- TOC +{:toc} + +## Reduced code coverage percentage +YourBase Test Acceleration is designed to avoid test runs that do not need to be executed. The percentage covered may appear lower than what is actually covered as a result. If your CI is configured to fail a build based on the percentage covered, you may need to reconfigure this feature. YourBase Test Acceleration has future plans to fill missing coverage data with data from previous runs. [link to code coverage] + +## Unittest setUp or tearDown overrides lead to incorrect test skipping + +If you are using unittest and define your own setUp/tearDown functions, be sure they call super before performing other actions: + +``` +class MyTestClass: + def setUp(self): + super(self.__class__, self).setUp() + # ... + + def tearDown(self): + super(self.__class__, self).tearDown() + # ... +``` + +If you are not defining your own setUp and tearDown functions, you do not need to do this. + +## __Sqlite3 module not found +If you run into errors about the _sqlite3 module not being found: + +1. First install sqlite3: + + - For example, for Debian-based OS: + + ```sudo apt-get install libsqlite3-dev``` + + - For macOS: + + ```brew install sqlite3``` + +2. Then rebuild and reinstall the Python version you are using. If you use `pyenv`, this will look something like: + + ```pyenv install --force ``` + +If the above steps don't work, try: +``` +PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" +pyenv install --force +``` + +## Incompatibility with Apple machines having the M1 chip + +YourBase Test Acceleration does not currently support Apple Machines running the M1 chip. Support is on our roadmap. If this is causing an issue for you, please reach out to us at hi@yourbase.io. + +## Conflict with proxy objects + +Python objects that opaquely wrap other objects by overriding Python builtins like __name__ and __class__ can cause tracing issues in YourBase Test Acceleration that may manifest as errors from within those proxy objects. If you experience these issues, you can set to use a slower tracing algorithm that should avoid these errors. + +```export YOURBASE_TIMID=true``` + +Tracing overhead is dramatically increased using this flag, so we don't recommend setting this if you are not experiencing issues. + +## Conflict with plugin pytest-xdist + +The YourBase Test Acceleration Test Selection and pytest-xdist [link] plugins have similar goals, reducing the overall test execution time of tests, but take different approaches to solving the problem. As such, there are conflicts when both plugins are enabled. When using the YourBase Test Acceleration Test Selection plugin, please uninstall pytest-xdist or execute pytest-xdist with `NUMCPUS=0` diff --git a/docs/test acceleration/reference/configuration-options.md b/docs/test acceleration/reference/configuration-options.md new file mode 100644 index 0000000..fc82ce9 --- /dev/null +++ b/docs/test acceleration/reference/configuration-options.md @@ -0,0 +1,188 @@ +--- +layout: default +title: Configuration options +nav_order: 2 +parent: Reference +grand_parent: Introduction +permalink: /test-acceleration/reference/configuration-options +--- + +# Configuration options +{: .no_toc } + +This page documents the available configuration options. These operate as environment variables that should be set the same way environment variables are set for your deployment. + +## Table of contents +{: .no_toc .text-delta } + +- TOC +{:toc} + +### YOURBASE_ACCEPT_TOS +`Type`: `bool-ish (0, false, off, 1, true, on)` + +`Default`: `off` + +When set, YourBase Test Acceleration will consider the terms of service permanently accepted for your organization, and will not output terms of service agreement prompts or info messages. This is helpful, for example, when rolling out YourBase Test Acceleration to CI for an organization. + +### YOURBASE_ACTIVE_COHORT +`Type`: `integer` in the range `[1, $YOURBASE_COHORT_COUNT]` + +`Default`: `1` + +When set alongside YOURBASE_COHORT_COUNT [link], tells YourBase Test Acceleration the cohort ID to run. Used for sharded or otherwise parallelized test suites. + +See here to learn to use this in your parallelized test-suite [link Advanced usage → Accelerate already parallelized tests]. + +--- +### YOURBASE_COHORT_COUNT +`Type`: `integer` +`Default`: `1` + +When set alongside YOURBASE_ACTIVE_COHORT [link], tells YourBase Test Acceleration how many cohorts the tests should be split into. + +This pair of settings lets YourBase Test Acceleration work with your existing sharding or parallelization setup. See here to learn to use this setting [link Advanced usage → Accelerate already parallelized tests]. + +--- + +### YOURBASE_LICENSE_KEY +`Type`: `opaque string` +`Default`: `(unset)` + +When set to a valid license key, YourBase Test Acceleration acceleration will be unlocked for use after the end of the free trial. Email hi@yourbase.io to obtain a license key. + +--- + +### YOURBASE_OBSERVATION_MODE +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will not skip tests, and instead, only record the duration and outcome of each test it believes should be skipped. + +When this setting is off, YourBase Test Acceleration will optimize your test run-time as usual. + +It’s useful to turn this setting on when you’re testing YourBase Test Acceleration before taking it live with your codebase. See here to learn more [link to Advanced usage → Observation mode]. + +--- + +### YOURBASE_REMOTE_CACHE +`Type`: `uri` +`Default`: `(unset)` + + +When set, this synchronizes dependency graphs generated only from clean working trees—dependency graphs generated from dirty working trees will not be synchronized as they can poison the cache. Use YOURBASE_SYNC_DIRTY [link] to override this behavior. + +Example usage: +``` +# Without a key prefix +export YOURBASE_REMOTE_CACHE=s3://my-bucket-name +``` + +``` +# With a key prefix +export YOURBASE_REMOTE_CACHE=s3://my-bucket-name/my/key/prefix +``` + +This setting is recommended for use when using YourBase Test Acceleration in CI, as the filesystem will not be a dependable store for dependency graphs. See [link: Advanced usage → Accelerate using shared dependency graph on how to use this setting] + +--- + +### YOURBASE_AWS_ACCESS_KEY_ID +`Type`: `AWS access key ID` +`Default`: `(unset)` + +When set alongside YOURBASE_AWS_SECRET_ACCESS_KEY [link], it forces YourBase Test Acceleration to use these credentials over system credentials when interacting with AWS. + +These environment variables are recommended for use if your AWS system credentials are fudged for the sake of your tests. + +--- + +### YOURBASE_AWS_SECRET_ACCESS_KEY +`Type`: `AWS secret access key` +`Default`: `(unset)` + +When set alongside YOURBASE_AWS_ACCESS_KEY_ID [link], it forces YourBase Test Acceleration to use these credentials over AWS system credentials when interacting with AWS. + +These environment variables are recommended for use if your system credentials are mocked for the sake of your tests. + +--- + +### YOURBASE_DEBUG +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will report significantly more internal information to stdout, stderr, and XDG (see the file returned by this expression): + +``` +echo ${XDG_STATE_HOME-~/.local/state}/yourbase/python.log +``` + +This setting is most beneficial when collaborating with the YourBase Test Acceleration team to debug issues. + +--- + +### YOURBASE_DISABLE +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will not load. + +Enabling this setting and then manually attaching to a test framework using yourbase.attach produces undefined behavior. + +--- + +### YOURBASE_IGNORE_LOCAL_CACHE +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will not look in the local filesystem for a dependency graph. + +If you’ve set YOURBASE_REMOTE_CACHE [link] to a valid location, YourBase Test Acceleration will look for a dependency graph there, and synchronize with the same location. + +Else, if YOURBASE_REMOTE_CACHE [link] is not set, YourBase Test Acceleration will do a cold run of your tests—it will run all the tests since it won’t be able to find any dependency graph. + +This setting can be used if the local cache is expected to be poisoned. For instance, this can happen if cohorting is used against a local cache. + +--- + +### YOURBASE_SYNC_DIRTY +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will synchronize graphs even if the Git working tree is dirty. + +This setting is not recommended for use when you run YourBase Test Acceleration locally, as it will poison the remote cache. + +This setting is useful when you use YourBase Test Acceleration in CI. There it can help you overcome situations where you need your working tree to be dirty while building, and you know the dirtiness will not affect the dependency graph. +If that situation does not apply to you, do not enable this setting. + +--- + +### YOURBASE_TELEMETRY +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `on` + +When on, YourBase Test Acceleration will send anonymized telemetry data to api.yourbase.io over HTTPS for the purposes of improving the product. +Note that, telemetry data never includes your code. + +Turn it off, if you want to opt out of sending usage statistics and error reports to YourBase Test Acceleration. + +--- + +### YOURBASE_TIMID +`Type`: `bool-ish (0, false, off, 1, true, on)` +`Default`: `off` + +When on, YourBase Test Acceleration will use a slower tracing algorithm that is less prone to conflicts with other packages than the default. + +We recommend reaching out to support@yourbase.io if you are encountering this scenario, and only enabling this if you experience issues with the default algorithm. + +--- + +### YOURBASE_WORKDIR +`Type`: `absolute or relative path` +`Default`: `.` + +This is the directory that YourBase Test Acceleration treats as the project directory. Only the code in this directory, or one of its descendants, will be traced. + +You usually do not need to change this, as it’s mainly for debugging purposes. diff --git a/docs/test acceleration/reference/index.md b/docs/test acceleration/reference/index.md new file mode 100644 index 0000000..cb23d39 --- /dev/null +++ b/docs/test acceleration/reference/index.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Reference +nav_order: 6 +parent: Introduction +has_children: true +permalink: /test-acceleration/reference +--- + +# Reference +The following pages come in handy when you want to override default values of environment variables, and debug common issues. \ No newline at end of file diff --git a/docs/test acceleration/releases.md b/docs/test acceleration/releases.md new file mode 100644 index 0000000..08151d3 --- /dev/null +++ b/docs/test acceleration/releases.md @@ -0,0 +1,12 @@ +--- +layout: default +title: Releases +nav_order: 7 +parent: Introduction +has_children: false +permalink: /test-acceleration/releases +--- + +#Releases + +See YourBase Test Acceleration release history on PyPI: https://pypi.org/project/yourbase/#history \ No newline at end of file diff --git a/docs/test acceleration/security.md b/docs/test acceleration/security.md new file mode 100644 index 0000000..e81b497 --- /dev/null +++ b/docs/test acceleration/security.md @@ -0,0 +1,17 @@ +--- +layout: default +title: Security +nav_order: 7 +parent: Introduction +has_children: false +permalink: /test-acceleration/security +--- + +# Security +YourBase Test Acceleration library is secure. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. + +By default, YourBase Test Acceleration collects anonymized data about how the YourBase Test Acceleration library is behaving to help resolve technical issues and develop the product with people's use--cases in mind. The information we collect via telemetry is documented at https://yourbase.io/data-usage. + +If YourBase Test Acceleration changes the type of data that it collects, it will always inform you at runtime. + +You can opt-out of sending any crash data or usage statistics to the YourBase Test Acceleration team, by setting the environment variable YOURBASE_TELEMETRY to false. From 6d3cd52b94c38c6ac2479bdb141df6181dd30f4f Mon Sep 17 00:00:00 2001 From: Priyanshu Chhazed Date: Mon, 16 Aug 2021 00:36:13 +0530 Subject: [PATCH 03/24] Trying to fix main nav --- docs/_config.yml | 34 +++++++++++++++++++++++++++++++++ docs/index.md | 16 +++++----------- docs/test acceleration/index.md | 5 ++--- 3 files changed, 41 insertions(+), 14 deletions(-) diff --git a/docs/_config.yml b/docs/_config.yml index e3b333b..a757db6 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -36,6 +36,40 @@ logo: "/images/logo.png" remote_theme: pmarsceill/just-the-docs color_scheme: yourbase +# Enable or disable the site search +# Supports true (default) or false +search_enabled: true + +search: + # Split pages into sections that can be searched individually + # Supports 1 - 6, default: 2 + heading_level: 2 + # Maximum amount of previews per search result + # Default: 3 + previews: 3 + # Maximum amount of words to display before a matched word in the preview + # Default: 5 + preview_words_before: 5 + # Maximum amount of words to display after a matched word in the preview + # Default: 10 + preview_words_after: 10 + # Set the search token separator + # Default: /[\s\-/]+/ + # Example: enable support for hyphenated search words + tokenizer_separator: /[\s/]+/ + # Display the relative url in search results + # Supports true (default) or false + rel_url: true + # Enable or disable the search button that appears in the bottom right corner of every page + # Supports true or false (default) + button: false + +# Heading anchor links appear on hover over h1-h6 tags in page content +# allowing users to deep link to a particular heading on a page. +# +# Supports true (default) or false +heading_anchors: true + kramdown: input: GFM parse_block_html: true diff --git a/docs/index.md b/docs/index.md index 022f6a7..f855964 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,15 +1,9 @@ --- +layout: default title: Home -nav_exclude: true +nav_order: 1 +permalink: / --- -YourBase - -# Test Acceleration - -- Python Test Acceleration - - [Getting Started](python/getting-started.md) - ***new users start here!*** - - [Configuration Options](python/configuration.md) - - [PyPI package](https://pypi.org/project/yourbase/) -- Ruby Test Acceleration - Coming soon! Contact us at [hi@yourbase.io](mailto:hi@yourbase.io) if you - want to join the private beta. +# YourBase +Some text talking about test acceleration & CI CLI \ No newline at end of file diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md index dcb1768..c96f8a9 100644 --- a/docs/test acceleration/index.md +++ b/docs/test acceleration/index.md @@ -1,8 +1,7 @@ --- layout: default -title: Introduction -parent: Home -nav_order: 1 +title: Test acceleration +nav_order: 2 has_children: true permalink: /test-acceleration --- From 9d3bf5fc138c9dc0f72502e156572f669af24ed8 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 16 Aug 2021 00:49:57 +0530 Subject: [PATCH 04/24] Removed old python pages --- docs/cli/index.md | 3 + docs/python/configuration.md | 250 --------------------------------- docs/python/getting-started.md | 195 ------------------------- 3 files changed, 3 insertions(+), 445 deletions(-) delete mode 100644 docs/python/configuration.md delete mode 100644 docs/python/getting-started.md diff --git a/docs/cli/index.md b/docs/cli/index.md index 557351f..e75dda4 100644 --- a/docs/cli/index.md +++ b/docs/cli/index.md @@ -1,6 +1,9 @@ --- +layout: default +title: CI and CLI nav_order: 3 has_children: true +permalink: /ci-cli --- # CI and CLI Documentation diff --git a/docs/python/configuration.md b/docs/python/configuration.md deleted file mode 100644 index 35ee73a..0000000 --- a/docs/python/configuration.md +++ /dev/null @@ -1,250 +0,0 @@ ---- -nav_order: 2 ---- -# Python Test Acceleration Configuration - -Python Test Acceleration works out of the box, but can be configured to take -advantage of your specific environment using the environment variables listed -below. - -Have configuration needs not addressed here? Give us a holler at -[hi@yourbase.io][hi]. - -- Common Settings - - [`YOURBASE_ACCEPT_TOS`](#yourbase_accept_tos) - - [`YOURBASE_ACTIVE_COHORT`](#yourbase_active_cohort) - - [`YOURBASE_COHORT_COUNT`](#yourbase_cohort_count) - - [`YOURBASE_LICENSE_KEY`](#yourbase_license_key) - - [`YOURBASE_OBSERVATION_MODE`](#yourbase_observation_mode) - - [`YOURBASE_REMOTE_CACHE`](#yourbase_remote_cache) -- Uncommon settings - - [`YOURBASE_AWS_ACCESS_KEY_ID`](#yourbase_aws_access_key_id) - - [`YOURBASE_AWS_SECRET_ACCESS_KEY`](#yourbase_aws_secret_access_key) - - [`YOURBASE_DEBUG`](#yourbase_debug) - - [`YOURBASE_DISABLE`](#yourbase_disable) - - [`YOURBASE_IGNORE_LOCAL_CACHE`](#yourbase_ignore_local_cache) - - [`YOURBASE_SYNC_DIRTY`](#yourbase_sync_dirty) - - [`YOURBASE_TELEMETRY`](#yourbase_telemetry) - - [`YOURBASE_TIMID`](#yourbase_timid) - - [`YOURBASE_WORKDIR`](#yourbase_workdir) - -## Common Settings - -### `YOURBASE_ACCEPT_TOS` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When set, YourBase will consider the terms of service permanently accepted for -your organization, and will not output terms of service agreement prompts or -info messages. - -### `YOURBASE_ACTIVE_COHORT` - -- **Type:** integer in the range `[1, $YOURBASE_COHORT_COUNT]` -- **Default:** 1 - -When set alongside [`YOURBASE_COHORT_COUNT`](#yourbase_cohort_count), tells -YourBase the cohort ID to run. Used for sharded or otherwise parallelized test -suites. - -See [`YOURBASE_COHORT_COUNT`](#yourbase_cohort_count) for more information. - -### `YOURBASE_COHORT_COUNT` - -- **Type:** integer -- **Default:** 1 - -When set alongside [`YOURBASE_ACTIVE_COHORT`](#yourbase_active_cohort), tells -YourBase how many cohorts tests should be split into. Used for sharded or -otherwise parallelized test suites. - -This pair of settings lets YourBase work with your existing sharding or -parallelization setup. You likely already have a test splitting solution in -place whose job is to slice up your pool of tests so that each shard or process -runs their fair share of tests. - -However, most test splitting solutions are not sticky; tests "jump" shards as -the pool changes. The effectiveness of YourBase acceleration scales with test -stickiness, so these solutions are not recommended for use with YourBase. - -As a convenience, YourBase provides a built-in sticky test splitting -implementation. To use it, first remove your existing test splitting tools. It -should appear as if each shard will run the entire test suite. - -Then, set `YOURBASE_ACTIVE_COHORT` to the ID of the current shard or process -(starting from 1), and set `YOURBASE_COHORT_COUNT` to the total number of shards -or processes. Under the hood, YourBase will just-in-time deselect tests that -aren't in the current cohort. This selection is consistent between runs; given -the same cohort ID and count, tests will be selected to the same shard for life. - -### `YOURBASE_LICENSE_KEY` - -- **Type:** opaque string -- **Default:** _(unset)_ - -When set to a valid license key, YourBase acceleration will be unlocked for use -after the end of the free trial. Email [hi@yourbase.io][hi] to obtain a license -key. - -### `YOURBASE_OBSERVATION_MODE` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will not skip tests. Instead, it will record the duration and -outcome of each test it believes can be skipped. - -If any of these tests fail, this is a bug in YourBase's tracing or acceleration. -YourBase will complain loudly and output details; please report these to -[bugs@yourbase.io][]. - -Otherwise, the total amount of time that could have been saved is output to -stdout. - -[bugs@yourbase.io]: mailto:bugs@yourbase.io - -### `YOURBASE_REMOTE_CACHE` - -- **Type:** uri -- **Default:** _(unset)_ - -When set, YourBase will synchronize dependency graphs with the given remote -location. This setting is recommended for use when using YourBase in CI, as the -filesystem will not be a dependable store for dependency graphs. - -Dependency graphs generated from dirty working trees will not be synchronized, -as they can poison the cache. See [`YOURBASE_SYNC_DIRTY`](#yourbase_sync_dirty) -to override this behavior. - -Currently, the only supported protocol is `s3`. - -#### `s3` - -Sets an S3 bucket as a remote cache. `Get`, `Put`, and `List` permissions are -needed. - -[System credentials][aws-sys-creds] for AWS will be used if present. To use -different credentials than the system credentials, see -[`YOURBASE_AWS_ACCESS_KEY_ID`](#yourbase_aws_access_key_id) and -[`YOURBASE_AWS_SECRET_ACCESS_KEY`](#yourbase_aws_secret_access_key). - -[aws-sys-creds]: https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html - -##### Example - -```sh -# Without a key prefix -export YOURBASE_REMOTE_CACHE=s3://my-bucket-name - -# With a key prefix -export YOURBASE_REMOTE_CACHE=s3://my-bucket-name/my/key/prefix -``` - -## Uncommon settings - -### `YOURBASE_AWS_ACCESS_KEY_ID` - -- **Type:** AWS access key ID -- **Default:** _(unset)_ - -When set alongside -[`YOURBASE_AWS_SECRET_ACCESS_KEY`](#yourbase_aws_secret_access_key), forces -YourBase to use these credentials over system credentials when interacting with -AWS. - -These environment variables are recommended for use if your system credentials -are fudged for the sake of your tests. - -### `YOURBASE_AWS_SECRET_ACCESS_KEY` - -- **Type:** AWS secret access key -- **Default:** _(unset)_ - -When set alongside [`YOURBASE_AWS_ACCESS_KEY_ID`](#yourbase_aws_access_key_id), -forces YourBase to use these credentials over system credentials when -interacting with AWS. - -These environment variables are recommended for use if your system credentials -are fudged for the sake of your tests. - -### `YOURBASE_DEBUG` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will report significantly more internal information to stdout, -stderr, and the file returned by this expression: - -```sh -echo ${XDG_STATE_HOME-~/.local/state}/yourbase/python.log -``` - -### `YOURBASE_DISABLE` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will not load. - -Enabling this setting then manually attaching to a test framework using -`yourbase.attach` produces undefined behavior. - -### `YOURBASE_IGNORE_LOCAL_CACHE` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will not read or write dependency graphs on the filesystem. If -[`YOURBASE_REMOTE_CACHE`](#yourbase_remote_cache) is set, it will still be used -as normal. - -This setting can be used if the local cache is expected to be poisoned. This can -happen if cohorting is used against a local cache. - -### `YOURBASE_SYNC_DIRTY` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will [synchronize graphs](#yourbase_remote_cache) even if the -Git working tree is dirty. - -This setting is not recommended for use if you plan to use YourBase on developer -machines, as it will poison the remote cache. - -If you only plan to use YourBase in CI, this setting can help you overcome -situations where you need your working tree to be dirty while building, and you -know the dirtiness will not affect the dependency graph. - -If that situation does not apply to you, do not enable this setting. - -### `YOURBASE_TELEMETRY` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** on - -When on, YourBase will send anonymized telemetry data to `api.yourbase.io` over -HTTPS for the purposes of improving the product. - -Telemetry data **never** includes your code. - -### `YOURBASE_TIMID` - -- **Type:** bool-ish (`0`, `false`, `off`, `1`, `true`, `on`) -- **Default:** off - -When on, YourBase will use a slower tracing algorithm that is less prone to -conflicts with other packages than the default. Only enable this if you -experience issues with the default algorithm. - -### `YOURBASE_WORKDIR` - -- **Type:** absolute or relative path -- **Default:** `.` - -The directory YourBase should treat as the project directory. Only code in this -directory, or one of its descendants, is guaranteed to be traced. You usually do -not need to change this. - -[hi]: mailto:hi@yourbase.io diff --git a/docs/python/getting-started.md b/docs/python/getting-started.md deleted file mode 100644 index 982330e..0000000 --- a/docs/python/getting-started.md +++ /dev/null @@ -1,195 +0,0 @@ ---- -nav_order: 1 ---- - -# Getting Started with Python Test Acceleration - -YourBase Test Acceleration is a plug-in that automatically determines which -tests need to run based on which changes were made to code. - -All it takes to get started is a `pip install yourbase`. - -## Table of Contents - * [Accelerate Tests Locally](#accelerate-tests-locally) - + [pytest](#pytest) - + [unittest](#unittest) - * [Set up Shared Dependency Graph](#set-up-shared-dependency-graph) - + [Introduction](#introduction) - + [Setup](#setup) - + [For use in a CI](#for-use-in-a-ci) - * [Support for Parallelized Tests](#support-for-parallelized-tests) - * [Product Usage Data](#product-usage-data) - -## Accelerate Tests Locally - -YourBase intelligent test selection for Python comes in the form of a library -that can be installed from PyPI using `pip` or `poetry` with a single command. -The first time you run your tests with YourBase, a _Dependency Graph_ will be -automatically built that maps which files and functions your tests depend on. -After making a change and re-running your tests, YourBase will load the optimal -dependency graph and use that information to intelligently select which tests -need to be run based on the changes that have been made to your source code. - -YourBase has support for both `pytest` and `unittest` - you can see examples of -both below, depending on the test framework you use. For further information you -can also refer to the [PyPI documentation page][]. - -[pypi documentation page]: https://pypi.org/project/yourbase/ - -### pytest - -YourBase is bundled with hooks for the pytest framework; to use it you run -`pip install yourbase` or add `yourbase` to your `requirements.txt` before -running `pip install -r requirements.txt` (or equivalent, `poetry` will also -work just fine). This installs the YourBase test selection library, which -includes a pytest plugin that will be automatically discovered by pytest without -any additional configuration. Once it has been installed, the plugin will be -loaded for any invocation of pytest that uses the python installation. - -For those who want to get started with an existing project, you can follow along -with the below example, using the `unleash-client-python` project to demonstrate -how things work. - -1. Checkout the project: - `git clone https://github.com/Unleash/unleash-client-python` - - - -2. (Optional) Create a virtual environment for this demonstration if you want to - use a clean Python environment (macOS / Linux can use `python -m venv .venv` - and activate it with `source .venv/bin/activate` ; Windows users can use - `python -m venv .venv` and then activate it with `source .venv/scripts/activate`) - -3. Install the YourBase Python library using `pip install yourbase` - -4. Run the Unleash client's tests using `pytest tests` - as you will see, this - executes all of the tests and builds an initial dependency graph. - - - -5. Run `pytest tests` again (making no code changes to the project) - this time - you will see that no code has changed and all the tests will be skipped, as - the outcome will be the same as before. - - - -6. Make a minor change to a source file; in our example we have modified the - function `test_create_feature_true` in the `tests/unit_tests/test_features.py` - file. - -7. Run `pytest tests` again, this time you will see a message telling you which - functions have been altered and how many tests were affected. If you modified - the same function as above then your output will closely match the output - below in which only one test gets re-run, while 167 of them are skipped, - bringing our test time down from 2 minutes to 1 second. - - - -8. Feel free to experiment with other functions in the source tree and see what - happens, then go ahead and use `yourbase` in your own project! - -### unittest - -YourBase comes bundled with hooks that will automatically detect when `unittest` -is being used and automatically take care of loading itself without any -additional action on your part. These hooks intercept the existing unittest -setup and teardown handlers in order to provide test acceleration. -Run `pip install yourbase` or add `yourbase` to your `requirements.txt` before -running `pip install -r requirements.txt` in order to install the test selection -library. - -For those wanting to get started with an example project, the following example -will get you up and running with minimal fuss by using the `python-unidiff` -project (which uses `unittest` to run its tests). - -1. Check out the project: `git clone https://github.com/matiasb/python-unidiff.git` - - - -2. (Optional) Create a virtual environment for this demonstration if you want to - use a clean Python environment (macOS / Linux can use `python -m venv .venv` - and activate it with `source .venv/bin/activate` ; Windows users can use - `python -m venv .venv` and then activate it with `source .venv/scripts/activate`) - -3. Install the YourBase Python library using `pip install yourbase` - -4. Update `tests/__init__.py` - - ```python - import unittest - import yourbase - - yourbase.attach(unittest) - ``` - -5. Run `python -m unittest discover` - - - -6. Run `python -m unittest discover` a second time with no changes to the - `python-unidiff` code files; observe that all tests will be skipped because - there have been no code changes. - - - -7. Make a small change to the project - in our case we made a small change to - `test_preserve_dos_line_endings` inside the `TestUnidiffParser` class located - in `tests/test_parser.py` - you can add a comment, a print statement or - something else small (or even break the test if you like). - -8. Run `pytest tests` after having made the change; here you will see that our - modification resulted in one test being affected and as such 37 out of the 38 - tests were skipped completely. - - - -9. Feel free to experiment with other functions in the source tree and see what - happens, then go ahead and use `yourbase` in your own project! - -## Set up Shared Dependency Graph - -### Introduction - -To get acceleration gains across team members, you can store the Dependency -Graph in the cloud and share it with others working on the same codebase. -YourBase will use information from your project's commit history to determine -the optimal Dependency Graph for each build. As a result, when you've submitted -code that is the same as code already tested by your colleagues, YourBase will -be able to skip those tests anywhere that has access to the shared graph -storage. - -### Setup - -YourBase currently supports storing shared graphs in [S3 buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html). - -1. In AWS, either create a new or have an existing S3 bucket for storing your - team's YourBase [dependency graph](https://yourbase.io/technology/dependency-graph). Note that the graphs are separated by unique project. - -2. Name and configure your S3 bucket(s), and have your S3 credentials. Set them to have read, write and list permissions. - -For each codebase, you can use either one bucket per project or use one bucket for all of your projects. - -3. Within your project, export an environment variable called `YOURBASE_REMOTE_CACHE` and set it to `s3://YOUR_BUCKET_NAME` with the name of your project's shared graphs S3 bucket. E.g., `export YOURBASE_REMOTE_CACHE="s3://acmecorp-yourbase-graphs"` - -4. [Generate or retrieve an AWS access key ID](https://aws.amazon.com/premiumsupport/knowledge-center/create-access-key/) and secret access key. [Configure your AWS credentials the standard AWS way](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) or set them to YourBase-specific environment variables `YOURBASE_AWS_ACCESS_KEY_ID` and `YOURBASE_AWS_SECRET_ACCESS_KEY`. - -### For use in a CI - -1. After setting up the S3 bucket for your YourBase dependency graph storage in the steps above, add - `yourbase` to your project (via `requirements.txt` or whatever mechanism you use - to install your dependencies). - -2. Use YourBase to accelerate the builds in your CI system by running your tests like normal. - -## Support for Parallelized Tests - -Python Test Acceleration supports test pipelines that are parallelized by other -tools or CIs. See [configuration][config/cohorting] for details. - -[config/cohorting]: configuration.md#YOURBASE_ACTIVE_COHORT - -## Product Usage Data - -By default, YourBase tracks how many tests are run and how many are skipped with -each build. YourBase also tracks the length of the tests. You can opt out of -data sharing by setting the environment variable `YOURBASE_TELEMETRY` to `false`. From d1833ff64770e72d7f4770e55a83678b9ae6df6f Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 16 Aug 2021 15:35:20 +0530 Subject: [PATCH 05/24] Moved cli md files to its docs/cli folder --- docs/{ => cli}/buildpacks.md | 0 docs/{ => cli}/ci-caching.md | 0 docs/{ => cli}/cli-settings.md | 0 docs/{ => cli}/configuration.md | 0 docs/{ => cli}/design-philosophy.md | 0 docs/{ => cli}/getting-started.md | 0 docs/{ => cli}/installation.md | 0 docs/python-getting-started/index.md | 15 --------------- 8 files changed, 15 deletions(-) rename docs/{ => cli}/buildpacks.md (100%) rename docs/{ => cli}/ci-caching.md (100%) rename docs/{ => cli}/cli-settings.md (100%) rename docs/{ => cli}/configuration.md (100%) rename docs/{ => cli}/design-philosophy.md (100%) rename docs/{ => cli}/getting-started.md (100%) rename docs/{ => cli}/installation.md (100%) delete mode 100644 docs/python-getting-started/index.md diff --git a/docs/buildpacks.md b/docs/cli/buildpacks.md similarity index 100% rename from docs/buildpacks.md rename to docs/cli/buildpacks.md diff --git a/docs/ci-caching.md b/docs/cli/ci-caching.md similarity index 100% rename from docs/ci-caching.md rename to docs/cli/ci-caching.md diff --git a/docs/cli-settings.md b/docs/cli/cli-settings.md similarity index 100% rename from docs/cli-settings.md rename to docs/cli/cli-settings.md diff --git a/docs/configuration.md b/docs/cli/configuration.md similarity index 100% rename from docs/configuration.md rename to docs/cli/configuration.md diff --git a/docs/design-philosophy.md b/docs/cli/design-philosophy.md similarity index 100% rename from docs/design-philosophy.md rename to docs/cli/design-philosophy.md diff --git a/docs/getting-started.md b/docs/cli/getting-started.md similarity index 100% rename from docs/getting-started.md rename to docs/cli/getting-started.md diff --git a/docs/installation.md b/docs/cli/installation.md similarity index 100% rename from docs/installation.md rename to docs/cli/installation.md diff --git a/docs/python-getting-started/index.md b/docs/python-getting-started/index.md deleted file mode 100644 index 4b3db00..0000000 --- a/docs/python-getting-started/index.md +++ /dev/null @@ -1,15 +0,0 @@ - - - - -Redirecting to https://docs.yourbase.io/python/getting-started - - From 259dd1d140efb4ddd884d0f94ea4fd08c94f24ef Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 16 Aug 2021 15:44:13 +0530 Subject: [PATCH 06/24] Changed url to prichhazed.github.io --- docs/_config.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/_config.yml b/docs/_config.yml index a757db6..0428a13 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -25,8 +25,10 @@ email: hi@yourbase.io # line in _config.yml. It will appear in your document head meta (for # Google search results) and in your feed.xml site description. baseurl: "" # the subpath of your site, e.g. /blog -url: https://docs.yourbase.io -domain: docs.yourbase.io +# url: https://docs.yourbase.io +url: https://prichhazed.github.io/yb/ +#domain: docs.yourbase.io +domain: prichhazed.github.io twitter_username: YourBaseIO github_username: yourbase matomo_url: https://web-analytics.yourbase.io/ From c64fad61ae9c0dcd37094168e841dcd0a892a648 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 16 Aug 2021 15:48:23 +0530 Subject: [PATCH 07/24] Changed CNAME to prichhazed.github.io --- docs/CNAME | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/CNAME b/docs/CNAME index b1a56a2..4ee502b 100644 --- a/docs/CNAME +++ b/docs/CNAME @@ -1 +1 @@ -docs.yourbase.io \ No newline at end of file +prichhazed.github.io \ No newline at end of file From ea84144066353db4e7bd413f1515771132f17b2c Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 16 Aug 2021 15:49:24 +0530 Subject: [PATCH 08/24] Removed search related JTD configuration --- docs/_config.yml | 34 ---------------------------------- 1 file changed, 34 deletions(-) diff --git a/docs/_config.yml b/docs/_config.yml index 0428a13..22a59a7 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -38,40 +38,6 @@ logo: "/images/logo.png" remote_theme: pmarsceill/just-the-docs color_scheme: yourbase -# Enable or disable the site search -# Supports true (default) or false -search_enabled: true - -search: - # Split pages into sections that can be searched individually - # Supports 1 - 6, default: 2 - heading_level: 2 - # Maximum amount of previews per search result - # Default: 3 - previews: 3 - # Maximum amount of words to display before a matched word in the preview - # Default: 5 - preview_words_before: 5 - # Maximum amount of words to display after a matched word in the preview - # Default: 10 - preview_words_after: 10 - # Set the search token separator - # Default: /[\s\-/]+/ - # Example: enable support for hyphenated search words - tokenizer_separator: /[\s/]+/ - # Display the relative url in search results - # Supports true (default) or false - rel_url: true - # Enable or disable the search button that appears in the bottom right corner of every page - # Supports true or false (default) - button: false - -# Heading anchor links appear on hover over h1-h6 tags in page content -# allowing users to deep link to a particular heading on a page. -# -# Supports true (default) or false -heading_anchors: true - kramdown: input: GFM parse_block_html: true From 8c756366def0eccc75151b0623e4b0fc6f06e2c4 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Wed, 18 Aug 2021 16:58:10 +0530 Subject: [PATCH 09/24] Fixed GH pages URL --- docs/_config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_config.yml b/docs/_config.yml index 22a59a7..501d909 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -26,7 +26,7 @@ email: hi@yourbase.io # Google search results) and in your feed.xml site description. baseurl: "" # the subpath of your site, e.g. /blog # url: https://docs.yourbase.io -url: https://prichhazed.github.io/yb/ +url: https://prichhazed.github.io/yb #domain: docs.yourbase.io domain: prichhazed.github.io twitter_username: YourBaseIO From 7e60f2f2e961d6f3dc3f96e99b6f8eed8510bf83 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Wed, 18 Aug 2021 23:22:38 +0530 Subject: [PATCH 10/24] Fixed most of the main navigation --- .../advanced-usage/accelerate-parallelized-tests.md | 4 ++-- .../accelerate-tests-across-developers.md | 4 ++-- .../advanced-usage/accelerate-tests-in-ci.md | 4 ++-- .../advanced-usage/force-run-specific-tests.md | 4 ++-- docs/test acceleration/advanced-usage/index.md | 2 +- .../advanced-usage/integrate-code-coverage-tools.md | 4 ++-- .../test acceleration/advanced-usage/verify-results.md | 4 ++-- docs/test acceleration/deploying.md | 2 +- docs/test acceleration/getting-started/index.md | 4 ++-- docs/test acceleration/getting-started/python/index.md | 4 ++-- .../getting-started/python/installation.md | 7 +++---- .../getting-started/python/testing frameworks/index.md | 5 +++-- .../python/testing frameworks/pytest.md | 8 ++++---- .../python/testing frameworks/unittest.md | 10 +++++----- docs/test acceleration/how-it-works.md | 2 +- docs/test acceleration/reference/common-issues.md | 4 ++-- .../reference/configuration-options.md | 4 ++-- docs/test acceleration/reference/index.md | 2 +- docs/test acceleration/releases.md | 4 ++-- docs/test acceleration/security.md | 2 +- docs/test acceleration/system-requirements.md | 6 +++--- 21 files changed, 45 insertions(+), 45 deletions(-) diff --git a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md index 572b980..75d409f 100644 --- a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md +++ b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md @@ -1,9 +1,9 @@ --- layout: default title: Accelerate parallelized tests -nav_order: 7 +nav_order: 6 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/accelerate-parallelized-tests --- diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md index 65699e7..64b00da 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md @@ -1,9 +1,9 @@ --- layout: default title: Accelerate tests locally across developers -nav_order: 6 +nav_order: 5 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/accelerate-tests-locally-across-developers --- diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md index 2d3806f..6316d75 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md @@ -1,9 +1,9 @@ --- layout: default title: Accelerate tests in CI -nav_order: 5 +nav_order: 4 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/accelerate-tests-in-ci --- diff --git a/docs/test acceleration/advanced-usage/force-run-specific-tests.md b/docs/test acceleration/advanced-usage/force-run-specific-tests.md index 266280f..6318ec8 100644 --- a/docs/test acceleration/advanced-usage/force-run-specific-tests.md +++ b/docs/test acceleration/advanced-usage/force-run-specific-tests.md @@ -1,9 +1,9 @@ --- layout: default title: Force run specific tests -nav_order: 2 +nav_order: 1 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/force-run-specific-tests --- diff --git a/docs/test acceleration/advanced-usage/index.md b/docs/test acceleration/advanced-usage/index.md index a15703a..a6759eb 100644 --- a/docs/test acceleration/advanced-usage/index.md +++ b/docs/test acceleration/advanced-usage/index.md @@ -2,7 +2,7 @@ layout: default title: Advanced usage nav_order: 5 -parent: Introduction +parent: Test acceleration has_children: true permalink: /test-acceleration/advanced-usage --- diff --git a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md index 90f9eaa..b913260 100644 --- a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md +++ b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md @@ -1,9 +1,9 @@ --- layout: default title: Verify results -nav_order: 4 +nav_order: 3 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/integrate-code-coverage-tools --- diff --git a/docs/test acceleration/advanced-usage/verify-results.md b/docs/test acceleration/advanced-usage/verify-results.md index 59fe154..76903fd 100644 --- a/docs/test acceleration/advanced-usage/verify-results.md +++ b/docs/test acceleration/advanced-usage/verify-results.md @@ -1,9 +1,9 @@ --- layout: default title: Verify results -nav_order: 3 +nav_order: 2 parent: Advanced usage -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/advanced-usage/verify-results --- diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying.md index 08fe45d..6cbe495 100644 --- a/docs/test acceleration/deploying.md +++ b/docs/test acceleration/deploying.md @@ -1,8 +1,8 @@ --- layout: default title: Deploying -parent: Introduction nav_order: 4 +parent: Test acceleration permalink: /test-acceleration/deploying --- diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md index 88eeae9..a9a5f2d 100644 --- a/docs/test acceleration/getting-started/index.md +++ b/docs/test acceleration/getting-started/index.md @@ -1,8 +1,8 @@ --- layout: default title: Getting started -nav_order: 3 -parent: Introduction +nav_order: 2 +parent: Test acceleration has_children: true permalink: /test-acceleration/getting-started --- diff --git a/docs/test acceleration/getting-started/python/index.md b/docs/test acceleration/getting-started/python/index.md index e7fdc3e..f59e780 100644 --- a/docs/test acceleration/getting-started/python/index.md +++ b/docs/test acceleration/getting-started/python/index.md @@ -3,9 +3,9 @@ layout: default title: Python nav_order: 1 parent: Getting started -grand_parent: Introduction +grand_parent: Test acceleration has_children: true -permalink: /test-acceleration/python +permalink: /test-acceleration/getting-started/python --- # YourBase Test Acceleration in Python diff --git a/docs/test acceleration/getting-started/python/installation.md b/docs/test acceleration/getting-started/python/installation.md index bfd05cd..a7ba1c0 100644 --- a/docs/test acceleration/getting-started/python/installation.md +++ b/docs/test acceleration/getting-started/python/installation.md @@ -1,10 +1,9 @@ --- layout: default title: Installation -nav_order: 2 -parent: Getting started -grand_parent: Introduction -has_children: true +nav_order: 1 +parent: Python +grand_parent: Getting started permalink: /test-acceleration/getting-started/python/install --- diff --git a/docs/test acceleration/getting-started/python/testing frameworks/index.md b/docs/test acceleration/getting-started/python/testing frameworks/index.md index c1c3b69..ac9da48 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/index.md +++ b/docs/test acceleration/getting-started/python/testing frameworks/index.md @@ -1,10 +1,11 @@ --- layout: default title: Python testing frameworks -nav_order: 1 +nav_order: 2 parent: Python grand_parent: Getting started -permalink: /test-acceleration/python/testing-frameworks +has_children: true +permalink: /test-acceleration/getting-started/python/testing-frameworks --- # Basic usage with Python testing frameworks diff --git a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md index 47ab681..b723916 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md +++ b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md @@ -1,10 +1,10 @@ --- layout: default title: Pytest -nav_order: 2 -parent: Python -grand_parent: Getting started -permalink: /test-acceleration/python/pytest +nav_order: 1 +parent: Python testing frameworks +grand_parent: Python testing frameworks +permalink: /test-acceleration/getting-started/python/pytest --- # Try in pytest diff --git a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md index 21af2a8..e583c47 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md +++ b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md @@ -1,10 +1,10 @@ --- layout: default -title: Pytest -nav_order: 3 -parent: Python -grand_parent: Getting started -permalink: /test-acceleration/python/unittest +title: Unittest +nav_order: 2 +parent: Python testing frameworks +grand_parent: Python testing frameworks +permalink: /test-acceleration/getting-started/python/unittest --- # Try in unittest diff --git a/docs/test acceleration/how-it-works.md b/docs/test acceleration/how-it-works.md index 24a27ff..262b705 100644 --- a/docs/test acceleration/how-it-works.md +++ b/docs/test acceleration/how-it-works.md @@ -1,8 +1,8 @@ --- layout: default title: How it works -parent: Introduction nav_order: 3 +parent: Test acceleration permalink: /test-acceleration/how-it-works --- diff --git a/docs/test acceleration/reference/common-issues.md b/docs/test acceleration/reference/common-issues.md index 04cf093..422cfcd 100644 --- a/docs/test acceleration/reference/common-issues.md +++ b/docs/test acceleration/reference/common-issues.md @@ -1,9 +1,9 @@ --- layout: default title: Common issues -nav_order: 3 +nav_order: 2 parent: Reference -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/reference/common-issues --- diff --git a/docs/test acceleration/reference/configuration-options.md b/docs/test acceleration/reference/configuration-options.md index fc82ce9..3cd25ba 100644 --- a/docs/test acceleration/reference/configuration-options.md +++ b/docs/test acceleration/reference/configuration-options.md @@ -1,9 +1,9 @@ --- layout: default title: Configuration options -nav_order: 2 +nav_order: 1 parent: Reference -grand_parent: Introduction +grand_parent: Test acceleration permalink: /test-acceleration/reference/configuration-options --- diff --git a/docs/test acceleration/reference/index.md b/docs/test acceleration/reference/index.md index cb23d39..856cff2 100644 --- a/docs/test acceleration/reference/index.md +++ b/docs/test acceleration/reference/index.md @@ -2,7 +2,7 @@ layout: default title: Reference nav_order: 6 -parent: Introduction +parent: Test acceleration has_children: true permalink: /test-acceleration/reference --- diff --git a/docs/test acceleration/releases.md b/docs/test acceleration/releases.md index 08151d3..76eceaf 100644 --- a/docs/test acceleration/releases.md +++ b/docs/test acceleration/releases.md @@ -1,8 +1,8 @@ --- layout: default title: Releases -nav_order: 7 -parent: Introduction +nav_order: 8 +parent: Test acceleration has_children: false permalink: /test-acceleration/releases --- diff --git a/docs/test acceleration/security.md b/docs/test acceleration/security.md index e81b497..abb0f93 100644 --- a/docs/test acceleration/security.md +++ b/docs/test acceleration/security.md @@ -2,7 +2,7 @@ layout: default title: Security nav_order: 7 -parent: Introduction +parent: Test acceleration has_children: false permalink: /test-acceleration/security --- diff --git a/docs/test acceleration/system-requirements.md b/docs/test acceleration/system-requirements.md index 628a9db..a9adcb0 100644 --- a/docs/test acceleration/system-requirements.md +++ b/docs/test acceleration/system-requirements.md @@ -1,8 +1,8 @@ --- layout: default -title: Getting Started -nav_order: 2 -parent: Introduction +title: System requirements +nav_order: 1 +parent: Test acceleration has_children: false permalink: /test-acceleration/system-requirements --- From 0b96a124c989d57b355905fa99407e03f8552756 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Thu, 19 Aug 2021 16:02:32 +0530 Subject: [PATCH 11/24] Fixed code highlights --- .../accelerate-parallelized-tests.md | 2 +- .../advanced-usage/accelerate-tests-in-ci.md | 4 +- .../force-run-specific-tests.md | 5 ++- .../integrate-code-coverage-tools.md | 4 +- docs/test acceleration/deploying.md | 4 +- .../getting-started/python/installation.md | 12 ++++-- .../python/testing frameworks/pytest.md | 40 +++++++++++++------ .../python/testing frameworks/unittest.md | 12 ++++-- .../reference/common-issues.md | 28 ++++++------- .../reference/configuration-options.md | 6 +-- 10 files changed, 70 insertions(+), 47 deletions(-) diff --git a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md index 75d409f..b285011 100644 --- a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md +++ b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md @@ -22,7 +22,7 @@ YourBase Test Acceleration integrates smoothly with your existing parallelizatio 2. Set YOURBASE_COHORT_COUNT [link to References → Configuration → YOURBASE_COHORT_COUNT] to your number of cohorts / shards 3. Set YOURBASE_ACTIVE_COHORT [link to References → Configuration → YOURBASE_ACTIVE_COHORT] to the ID of the current cohort / shard – starting from 1. For example, if you have 5 shards and are running the 4th shard, you’d set: - ``` + ```bash YOURBASE_COHORT_COUNT = 5 YOURBASE_ACTIVE_COHORT = 4 ``` diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md index 6316d75..46c7802 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md @@ -32,7 +32,7 @@ You can configure YourBase Test Acceleration to use either the system AWS creden ##### Option 1: Use the system AWS credentials To use your systems’ AWS credentials, export the following in your environment: -``` +```bash export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= ``` @@ -40,7 +40,7 @@ export AWS_SECRET_ACCESS_KEY= ##### Option 2: Use YourBase Test Acceleration-specific AWS credentials If you want YourBase Test Acceleration to use different credentials (or if you're setting the system AWS credentials to mock values for your tests or CI), you can set these YourBase Test Acceleration-specific environment variables instead: -``` +```bash export YOURBASE_AWS_ACCESS_KEY_ID= export YOURBASE_AWS_SECRET_ACCESS_KEY= ``` diff --git a/docs/test acceleration/advanced-usage/force-run-specific-tests.md b/docs/test acceleration/advanced-usage/force-run-specific-tests.md index 6318ec8..c1e18a3 100644 --- a/docs/test acceleration/advanced-usage/force-run-specific-tests.md +++ b/docs/test acceleration/advanced-usage/force-run-specific-tests.md @@ -19,7 +19,7 @@ This feature is supported in the following testing frameworks: ## pytest You can tell YourBase Test Acceleration to never skip a specific test using decorators. -``` +```python import pytest @pytest.mark.do_not_accelerate @@ -33,7 +33,8 @@ The decorator @pytest.mark.do_not_accelerate ensures that the test test_function ## unittest You can tell YourBase Test Acceleration to never skip specific tests using decorators. -```import yourbase.plugins.unittest as yourbase +```python +import yourbase.plugins.unittest as yourbase @yourbase.do_not_accelerate class TestClass(unittest.TestCase): diff --git a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md index b913260..5a4faa1 100644 --- a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md +++ b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md @@ -1,6 +1,6 @@ --- layout: default -title: Verify results +title: Integrate code coverage tools nav_order: 3 parent: Advanced usage grand_parent: Test acceleration @@ -24,7 +24,7 @@ Note: If your CI is configured to fail a build based on the percentage covered, YourBase Test Acceleration is compatible with all versions of Coverage out-of-the-box. To ensure that coverage reports account for skipped tests as well: 1. Use Coverage 5.5+ 2. Set the following in your `.coveragerc` file: -``` +```python [run] relative_files = true ``` diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying.md index 6cbe495..01b255c 100644 --- a/docs/test acceleration/deploying.md +++ b/docs/test acceleration/deploying.md @@ -76,6 +76,8 @@ To obtain more detailed debugging information, set the `YOURBASE_DEBUG` environm ## Disable YourBase Test Acceleration If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable `YOURBASE_DISABLE` to true: -```export YOURBASE_DISABLE=true``` +```bash +export YOURBASE_DISABLE=true +``` To uninstall the package, see uninstall instructions [Link to Uninstall] \ No newline at end of file diff --git a/docs/test acceleration/getting-started/python/installation.md b/docs/test acceleration/getting-started/python/installation.md index a7ba1c0..bd9446a 100644 --- a/docs/test acceleration/getting-started/python/installation.md +++ b/docs/test acceleration/getting-started/python/installation.md @@ -34,7 +34,7 @@ To check if YourBase Test Acceleration is installed, run the following command i If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. -``` +```bash Name: yourbase Version: 5.2.4 Summary: Skip tests based on tracing data @@ -51,15 +51,19 @@ Required-by: ### Using poetry If you use poetry, you can install YourBase Test Acceleration with: -```poetry add yourbase``` +```bash +poetry add yourbase +``` To check if YourBase Test Acceleration is installed, run the following command in a shell prompt: -```poetry show yourbase``` +```bash +poetry show yourbase +``` If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. -``` +```bash Name: yourbase Version: 5.2.4 Summary: Skip tests based on tracing data diff --git a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md index b723916..93d3d20 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md +++ b/docs/test acceleration/getting-started/python/testing frameworks/pytest.md @@ -32,19 +32,27 @@ Make sure that, on your machine or on your virtual environment, you have: Open a new shell prompt, and checkout this sample project from git -```git clone https://github.com/Unleash/unleash-client-python``` +```bash +git clone https://github.com/Unleash/unleash-client-python +``` From your shell prompt, navigate to the directory where you checked out the project -```cd unleash-client-python``` +```bash +cd unleash-client-python +``` -Run its `requirements.txt` file to install the required packages. +Install dependencies -```pip install -r requirements.txt``` +```bash +pip install -r requirements.txt +``` ### Step 2: Execute a command to run all the tests of the project -```pytest tests``` +```bash +pytest tests +``` Let’s look at the output. @@ -52,7 +60,9 @@ All messages that are prefixed with “[YB]” are logged by the YourBase Test A ### Step 3: Without making any code change, execute the command to run all the tests again -```pytest tests``` +```bash +pytest tests +``` Let’s look at the messages logged by YourBase Test Acceleration. One of the messages says: “[YB] No code changed. Running only new tests, if any.” @@ -60,17 +70,23 @@ Since you ran the tests without changing any code, YourBase Test Acceleration sk ### Step 4: Make a code change and execute the command to run all tests again as shown below: -Open ```tests/unit_tests/test_features.py``` in your text editor +Open `tests/unit_tests/test_features.py` in your text editor -```vim tests/unit_tests/test_features.py``` +```bash +vim tests/unit_tests/test_features.py +``` -Add the following print statement in the beginning of the method ```test_create_feature_true``` +Add the following print statement in the beginning of the method `test_create_feature_true` -```print(“Checking YourBase Test Acceleration after a code-change...”)``` +```python +print(“Checking YourBase Test Acceleration after a code-change...”) +``` Run the tests again using: -```pytest tests``` +```bash +pytest tests +``` Let’s look at the output again. @@ -78,7 +94,7 @@ If you modified the same function as above then your output will closely match t Look for the logs traced by YourBase Test Acceleration. You’ll see a message telling you how many functions have been altered and how many tests were affected. If you’re following the steps as is, you should see: -``` +```bash [YB] 1 function differs from the dependency graph [YB] ~ tests/unit_tests/test_features.py#test_create_feature_true [YB] Function-level dependency graph found 1 test affected diff --git a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md index e583c47..a1a44fd 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md +++ b/docs/test acceleration/getting-started/python/testing frameworks/unittest.md @@ -24,9 +24,9 @@ Make sure that, on your machine or on your virtual environment, you have: ### Step 1: Attach yourbase to unittest. -For example, in your ```tests/__init__.py file```, copy-paste the following: +For example, in your `tests/__init__.py file`, copy-paste the following: -``` +```python import unittest import yourbase @@ -36,7 +36,9 @@ yourbase.attach(unittest) ### Step 2: Run your project’s tests For example, to run all the tests for the project, use: -```python -m unittest discover``` +```bash +python -m unittest discover +``` Note: If your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly. @@ -49,7 +51,9 @@ Without making a code change, run your tests again using the same command as in You can simply add a print statement like below: -```print(“Checking YourBase Test Acceleration after a code-change...”)``` +```python +print(“Checking YourBase Test Acceleration after a code-change...”) +``` ### Step 5: Run your tests again using the same command as in step #2. diff --git a/docs/test acceleration/reference/common-issues.md b/docs/test acceleration/reference/common-issues.md index 422cfcd..ce8b2ac 100644 --- a/docs/test acceleration/reference/common-issues.md +++ b/docs/test acceleration/reference/common-issues.md @@ -25,7 +25,7 @@ YourBase Test Acceleration is designed to avoid test runs that do not need to be If you are using unittest and define your own setUp/tearDown functions, be sure they call super before performing other actions: -``` +```python class MyTestClass: def setUp(self): super(self.__class__, self).setUp() @@ -39,24 +39,18 @@ class MyTestClass: If you are not defining your own setUp and tearDown functions, you do not need to do this. ## __Sqlite3 module not found -If you run into errors about the _sqlite3 module not being found: - -1. First install sqlite3: - - - For example, for Debian-based OS: - - ```sudo apt-get install libsqlite3-dev``` - - - For macOS: - - ```brew install sqlite3``` +If you run into errors about the _sqlite3 module not being found, follow the below steps: -2. Then rebuild and reinstall the Python version you are using. If you use `pyenv`, this will look something like: +1. Install sqlite3 - ```pyenv install --force ``` +2. Rebuild and reinstall the Python version you are using. If you use `pyenv`, this will look something like: -If the above steps don't work, try: +```bash +pyenv install --force ``` + +If the above step doesn't work, try: +```bash PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" pyenv install --force ``` @@ -69,7 +63,9 @@ YourBase Test Acceleration does not currently support Apple Machines running the Python objects that opaquely wrap other objects by overriding Python builtins like __name__ and __class__ can cause tracing issues in YourBase Test Acceleration that may manifest as errors from within those proxy objects. If you experience these issues, you can set to use a slower tracing algorithm that should avoid these errors. -```export YOURBASE_TIMID=true``` +```bash +export YOURBASE_TIMID=true +``` Tracing overhead is dramatically increased using this flag, so we don't recommend setting this if you are not experiencing issues. diff --git a/docs/test acceleration/reference/configuration-options.md b/docs/test acceleration/reference/configuration-options.md index 3cd25ba..c1114ec 100644 --- a/docs/test acceleration/reference/configuration-options.md +++ b/docs/test acceleration/reference/configuration-options.md @@ -73,12 +73,12 @@ It’s useful to turn this setting on when you’re testing YourBase Test Accele When set, this synchronizes dependency graphs generated only from clean working trees—dependency graphs generated from dirty working trees will not be synchronized as they can poison the cache. Use YOURBASE_SYNC_DIRTY [link] to override this behavior. Example usage: -``` +```bash # Without a key prefix export YOURBASE_REMOTE_CACHE=s3://my-bucket-name ``` -``` +```bash # With a key prefix export YOURBASE_REMOTE_CACHE=s3://my-bucket-name/my/key/prefix ``` @@ -113,7 +113,7 @@ These environment variables are recommended for use if your system credentials a When on, YourBase Test Acceleration will report significantly more internal information to stdout, stderr, and XDG (see the file returned by this expression): -``` +```bash echo ${XDG_STATE_HOME-~/.local/state}/yourbase/python.log ``` From f80788359281bf0cd7643e6dfda49268ce8ec00f Mon Sep 17 00:00:00 2001 From: prichhazed Date: Thu, 19 Aug 2021 16:20:13 +0530 Subject: [PATCH 12/24] Fixed hierarchy issue --- .../test acceleration/getting-started/index.md | 3 +-- .../{python/testing frameworks => }/pytest.md | 4 ++-- .../getting-started/python/index.md | 16 ---------------- .../python/testing frameworks/index.md | 12 ------------ .../testing frameworks => }/unittest.md | 4 ++-- .../python/installation.md => install.md} | 18 +++++++++--------- 6 files changed, 14 insertions(+), 43 deletions(-) rename docs/test acceleration/getting-started/{python/testing frameworks => }/pytest.md (98%) delete mode 100644 docs/test acceleration/getting-started/python/index.md delete mode 100644 docs/test acceleration/getting-started/python/testing frameworks/index.md rename docs/test acceleration/getting-started/{python/testing frameworks => }/unittest.md (96%) rename docs/test acceleration/{getting-started/python/installation.md => install.md} (92%) diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md index a9a5f2d..89edcc4 100644 --- a/docs/test acceleration/getting-started/index.md +++ b/docs/test acceleration/getting-started/index.md @@ -8,5 +8,4 @@ permalink: /test-acceleration/getting-started --- # Getting started -This document will guide you to install and learn to use YourBase Test Acceleration library in the language and the testing framework of your choice. - +This section gives an introduction on how to get started with running YourBase Test Acceleration in supported testing frameworks. For a full walkthrough of the tool, see [Link to Try in pytest] diff --git a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md b/docs/test acceleration/getting-started/pytest.md similarity index 98% rename from docs/test acceleration/getting-started/python/testing frameworks/pytest.md rename to docs/test acceleration/getting-started/pytest.md index 93d3d20..136539b 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/pytest.md +++ b/docs/test acceleration/getting-started/pytest.md @@ -2,8 +2,8 @@ layout: default title: Pytest nav_order: 1 -parent: Python testing frameworks -grand_parent: Python testing frameworks +parent: Getting started +grand_parent: Test acceleration permalink: /test-acceleration/getting-started/python/pytest --- diff --git a/docs/test acceleration/getting-started/python/index.md b/docs/test acceleration/getting-started/python/index.md deleted file mode 100644 index f59e780..0000000 --- a/docs/test acceleration/getting-started/python/index.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -layout: default -title: Python -nav_order: 1 -parent: Getting started -grand_parent: Test acceleration -has_children: true -permalink: /test-acceleration/getting-started/python ---- - -# YourBase Test Acceleration in Python - -This page guides you to install the YourBase Test Acceleration library in your Python environment. - -Make sure that: -YourBase Test Acceleration library supports your technical stack and infrastructure [link to the System Requirements section]. diff --git a/docs/test acceleration/getting-started/python/testing frameworks/index.md b/docs/test acceleration/getting-started/python/testing frameworks/index.md deleted file mode 100644 index ac9da48..0000000 --- a/docs/test acceleration/getting-started/python/testing frameworks/index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -layout: default -title: Python testing frameworks -nav_order: 2 -parent: Python -grand_parent: Getting started -has_children: true -permalink: /test-acceleration/getting-started/python/testing-frameworks ---- - -# Basic usage with Python testing frameworks -This section gives an introduction on how to get started with running YourBase Test Acceleration in supported testing frameworks. For a full walkthrough of the tool, see [Link to Try in pytest] \ No newline at end of file diff --git a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md b/docs/test acceleration/getting-started/unittest.md similarity index 96% rename from docs/test acceleration/getting-started/python/testing frameworks/unittest.md rename to docs/test acceleration/getting-started/unittest.md index a1a44fd..6666e1b 100644 --- a/docs/test acceleration/getting-started/python/testing frameworks/unittest.md +++ b/docs/test acceleration/getting-started/unittest.md @@ -2,8 +2,8 @@ layout: default title: Unittest nav_order: 2 -parent: Python testing frameworks -grand_parent: Python testing frameworks +parent: Getting started +grand_parent: Test acceleration permalink: /test-acceleration/getting-started/python/unittest --- diff --git a/docs/test acceleration/getting-started/python/installation.md b/docs/test acceleration/install.md similarity index 92% rename from docs/test acceleration/getting-started/python/installation.md rename to docs/test acceleration/install.md index bd9446a..932681c 100644 --- a/docs/test acceleration/getting-started/python/installation.md +++ b/docs/test acceleration/install.md @@ -1,20 +1,20 @@ --- layout: default -title: Installation -nav_order: 1 -parent: Python -grand_parent: Getting started -permalink: /test-acceleration/getting-started/python/install +title: Install +nav_order: 2 +parent: Test acceleration +has_children: false +permalink: /test-acceleration/install --- -# Python installation -{: .no_toc } +# Install +{:.no_toc} ## Table of contents -{: .no_toc } +{:.no_toc} 1. TOC -{ :toc } +{:toc} ## Prerequisites - YourBase Test Acceleration library supports your technical stack and infrastructure [link to the System Requirements section]. From 66faa53dd3a54e91ec448e2e929fb960383042b1 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Thu, 19 Aug 2021 18:19:24 +0530 Subject: [PATCH 13/24] Fixed config, added some links --- docs/test acceleration/index.md | 24 ++++++++++++++----- docs/test acceleration/system-requirements.md | 24 +++++++++---------- 2 files changed, 30 insertions(+), 18 deletions(-) diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md index c96f8a9..3cb52ae 100644 --- a/docs/test acceleration/index.md +++ b/docs/test acceleration/index.md @@ -3,26 +3,38 @@ layout: default title: Test acceleration nav_order: 2 has_children: true +has_toc: false permalink: /test-acceleration --- # Introduction +{:.no_toc} YourBase Test Acceleration is a library for shortening your test run-times by up to 90%. It hooks into your testing framework to intelligently select which tests should be run for a given code change. ### How does YourBase Test Acceleration reduce test times? At the core of YourBase Test Acceleration is the YourBase Dependency Graph that maps which files and functions each of your tests depends on. Every time you run your tests, YourBase Test Acceleration library will load the optimal dependency graph to select and run only the tests that pertain to your code changes, and safely avoid running any unrelated tests. As a result, your builds execute only the optimal fraction of your total tests and finish much faster. -### Is YourBase Test Acceleration library compatible with your tech-stack and infrastructure? -The library currently supports testing frameworks for Python and Ruby. See here for a complete list of system requirements [link to the System Requirements section]. +--- + +## Is it compatible with your tech-stack and infrastructure? +The library currently supports [testing frameworks for Python](test-acceleration/system-requirements#supported-languages--testing-frameworks). See here for a complete list of system requirements [link to the System Requirements section]. -### Which type of tests does YourBase Test Acceleration library accelerate? +--- + +## Which type of tests does it accelerate? The library supports unit tests and integration tests that call code from within the application. Since the library traces dependencies from within the test-runner process, distributed dependencies like network or database calls are not fully traced and accelerated. -### Will YourBase Test Acceleration library provide benefits at your scale? +--- + +## Will it provide benefits at your scale? The library has the most benefits for use-cases where test runs are taking over 10 minutes. Our customers have reduced their test run times by up to 90%, for example, where one customer was able to skip over 99% of the 11,000+ tests in the suite using our library. -### Is YourBase Test Acceleration library secure? +--- + +## Is it secure? Yes. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. See here for more details [link to the Security section]. -### Is YourBase Test Acceleration library stable? +--- + +## Is it stable? The library is currently in Beta. While we are confident in the reliability of our offering, we are making improvements all the time. If you identify any issues, please let us know at support@yourbase.io and we will look into them quickly. diff --git a/docs/test acceleration/system-requirements.md b/docs/test acceleration/system-requirements.md index a9adcb0..1b836f7 100644 --- a/docs/test acceleration/system-requirements.md +++ b/docs/test acceleration/system-requirements.md @@ -11,25 +11,25 @@ permalink: /test-acceleration/system-requirements YourBase Test Acceleration library works seamlessly with the CI system of your choice and is agnostic to where you host your application—be it cloud, on-premise, or offline. -### Supported version control tools -The library can currently accelerate tests for codebases that are version-controlled using Git. +## Supported version control tools +The library can accelerate tests only for codebases that are version-controlled using [Git](https://git-scm.com/). -### Supported platforms +## Supported platforms - Linux - MacOS: All except those with the M1 chip -### Supported languages & testing frameworks +## Supported languages & testing frameworks - Python 2.7+ and Python 3.5+ that use the following testing frameworks: - - pytest - - unittest + - [pytest](https://docs.pytest.org/en/6.2.x/) + - [unittest](https://docs.python.org/3/library/unittest.html) - Note: Any web frameworks, such as Django, that are built atop these, are also supported. +_Note: Any web frameworks, such as Django, that are built atop the above testing frameworks, are also supported._ -### Compatible code coverage tools +## Compatible code coverage tools The library supports integration with the following code coverage reporting tools without changing your workflow, for example: -- Coverage [Link to Advanced usage section that guides on how to configure it for proper coverage reports] -- pytest-cov -- SonarQube -- Codecov +- [Coverage](https://coverage.readthedocs.io/en/coverage-5.5/) +- [pytest-cov](https://pypi.org/project/pytest-cov/) +- [SonarQube](https://www.sonarqube.org/) +- [Codecov](https://about.codecov.io/) Reach out to support@yourbase.io if you have questions about support for your favorite coverage tool. \ No newline at end of file From 996fe43a716fadcc4c05083546e02603bbd18c20 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Thu, 19 Aug 2021 18:19:45 +0530 Subject: [PATCH 14/24] Fixed config --- docs/_config.yml | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/_config.yml b/docs/_config.yml index 501d909..15d84cd 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -25,10 +25,8 @@ email: hi@yourbase.io # line in _config.yml. It will appear in your document head meta (for # Google search results) and in your feed.xml site description. baseurl: "" # the subpath of your site, e.g. /blog -# url: https://docs.yourbase.io -url: https://prichhazed.github.io/yb -#domain: docs.yourbase.io -domain: prichhazed.github.io +url: https://docs.yourbase.io +domain: docs.yourbase.io twitter_username: YourBaseIO github_username: yourbase matomo_url: https://web-analytics.yourbase.io/ @@ -38,6 +36,13 @@ logo: "/images/logo.png" remote_theme: pmarsceill/just-the-docs color_scheme: yourbase +# Footer content +# appears at the bottom of every page's main content + +# Back to top link +back_to_top: true +back_to_top_text: "Back to top" + kramdown: input: GFM parse_block_html: true From 978b6bd8805b8871e3af5ac3ac40e1173ff2add4 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Thu, 19 Aug 2021 20:48:52 +0530 Subject: [PATCH 15/24] Added CI CLI docs --- docs/cli/buildpacks.md | 2 +- docs/cli/ci-caching.md | 2 +- docs/cli/cli-settings.md | 2 +- docs/cli/configuration.md | 2 +- docs/cli/design-philosophy.md | 2 +- docs/cli/getting-started.md | 2 +- docs/cli/installation.md | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/cli/buildpacks.md b/docs/cli/buildpacks.md index 9159848..d21bfc8 100644 --- a/docs/cli/buildpacks.md +++ b/docs/cli/buildpacks.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 3 --- diff --git a/docs/cli/ci-caching.md b/docs/cli/ci-caching.md index 2291efa..51ee49d 100644 --- a/docs/cli/ci-caching.md +++ b/docs/cli/ci-caching.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 6 --- diff --git a/docs/cli/cli-settings.md b/docs/cli/cli-settings.md index df930c9..a7d74d3 100644 --- a/docs/cli/cli-settings.md +++ b/docs/cli/cli-settings.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 5 --- diff --git a/docs/cli/configuration.md b/docs/cli/configuration.md index 20418df..1db258d 100644 --- a/docs/cli/configuration.md +++ b/docs/cli/configuration.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 4 --- diff --git a/docs/cli/design-philosophy.md b/docs/cli/design-philosophy.md index 2451132..35a138b 100644 --- a/docs/cli/design-philosophy.md +++ b/docs/cli/design-philosophy.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 7 --- diff --git a/docs/cli/getting-started.md b/docs/cli/getting-started.md index 1b006a1..287dc6d 100644 --- a/docs/cli/getting-started.md +++ b/docs/cli/getting-started.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 2 --- diff --git a/docs/cli/installation.md b/docs/cli/installation.md index 9b05409..1f279c6 100644 --- a/docs/cli/installation.md +++ b/docs/cli/installation.md @@ -1,5 +1,5 @@ --- -parent: CI and CLI Documentation +parent: CI and CLI nav_order: 1 --- From 9762e46b52daf5691cc6c77d2f6b8ee9d2cfd8b2 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 11:55:28 +0530 Subject: [PATCH 16/24] Added internal links part 1 --- .../force-run-specific-tests.md | 43 ------------- .../test acceleration/advanced-usage/index.md | 2 +- .../integrate-code-coverage-tools.md | 25 +++++--- .../advanced-usage/mark-test-unskippable.md | 42 +++++++++++++ .../advanced-usage/verify-results.md | 41 +++++++++--- docs/test acceleration/deploying.md | 32 +++++----- .../getting-started/index.md | 2 +- .../getting-started/pytest.md | 59 ++++++++++++------ .../getting-started/unittest.md | 46 +++++++++----- docs/test acceleration/how-it-works.md | 8 ++- docs/test acceleration/index.md | 12 ++-- docs/test acceleration/install.md | 62 ++++++++++++------- docs/test acceleration/system-requirements.md | 6 +- 13 files changed, 238 insertions(+), 142 deletions(-) delete mode 100644 docs/test acceleration/advanced-usage/force-run-specific-tests.md create mode 100644 docs/test acceleration/advanced-usage/mark-test-unskippable.md diff --git a/docs/test acceleration/advanced-usage/force-run-specific-tests.md b/docs/test acceleration/advanced-usage/force-run-specific-tests.md deleted file mode 100644 index c1e18a3..0000000 --- a/docs/test acceleration/advanced-usage/force-run-specific-tests.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -layout: default -title: Force run specific tests -nav_order: 1 -parent: Advanced usage -grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/force-run-specific-tests ---- - -# Force run specific tests - -{: .no_toc } - -This feature is supported in the following testing frameworks: -- pytest -{:toc} -- unittest - -## pytest -You can tell YourBase Test Acceleration to never skip a specific test using decorators. - -```python -import pytest - -@pytest.mark.do_not_accelerate -def test_function(): - # ... -``` - -The decorator @pytest.mark.do_not_accelerate ensures that the test test_function() is never skipped by YourBase Test Acceleration, even where there are no code changes in its dependencies. - - -## unittest -You can tell YourBase Test Acceleration to never skip specific tests using decorators. - -```python -import yourbase.plugins.unittest as yourbase - -@yourbase.do_not_accelerate -class TestClass(unittest.TestCase): - def test_function(): - # ... -``` \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/index.md b/docs/test acceleration/advanced-usage/index.md index a6759eb..c8529eb 100644 --- a/docs/test acceleration/advanced-usage/index.md +++ b/docs/test acceleration/advanced-usage/index.md @@ -8,4 +8,4 @@ permalink: /test-acceleration/advanced-usage --- # Advanced usage -This document will guide you to make use of advanced featuers of YourBase Test Acceleration. +This document will guide you to make use of advanced features of YourBase Test Acceleration. diff --git a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md index 5a4faa1..9f36b23 100644 --- a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md +++ b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md @@ -8,17 +8,28 @@ permalink: /test-acceleration/advanced-usage/integrate-code-coverage-tools --- # Integrate code coverage tools - {: .no_toc } -YourBase Test Acceleration supports the following code coverage tools. -- Coverage -{:toc} +Since YourBase Test Acceleration is designed to avoid test runs that do not need to be executed, the percentage covered as reported by your code coverage tool will likely be lower than what is actually covered as a result. + +This decrease in the coverage reported by your tool will affect you if your CI is configured to fail a build on the basis of code coverage percentage. + +The following sections lists ways to circumvent this problem. + +## Configure your code coverage tool + +You can circumvent this by making configuration changes in your code coverage tool, so that YourBase Test Acceleration can fill missing coverage data from previous runs. +However, this feature of YourBase Test Acceleration works only on specific versions of the supported code coverage tools. The following sections provide details on this. -Note that since YourBase Test Acceleration is designed to avoid test runs that do not need to be executed, the percentage covered may appear lower than what is actually covered as a result. You can circumvent this by making configuration changes in your code coverage tool, so that YourBase Test Acceleration can fill missing coverage data from previous runs. -However, this feature of YourBase Test Acceleration works only on specific versions of code coverage tools. The following sections provide details on this. -Note: If your CI is configured to fail a build based on the percentage covered, you may need to reconfigure it. + +--- + +## Configure your builds + +YourBase Test Acceleration supports the following code coverage tools. +- TOC +{:toc} ## Coverage YourBase Test Acceleration is compatible with all versions of Coverage out-of-the-box. To ensure that coverage reports account for skipped tests as well: diff --git a/docs/test acceleration/advanced-usage/mark-test-unskippable.md b/docs/test acceleration/advanced-usage/mark-test-unskippable.md new file mode 100644 index 0000000..babace4 --- /dev/null +++ b/docs/test acceleration/advanced-usage/mark-test-unskippable.md @@ -0,0 +1,42 @@ +--- +layout: default +title: Mark a test as unskippable +nav_order: 1 +parent: Advanced usage +grand_parent: Test acceleration +permalink: /test-acceleration/advanced-usage/mark-test-unskippable +--- + +# Mark a test as unskippable +{: .no_toc } +This feature is supported in the following testing frameworks: +- TOC +{:toc} + +## pytest +If you're using [pytest](https://docs.pytest.org/en/6.2.x/), you can tell YourBase Test Acceleration to never skip a specific test using decorators as shown in the below example: + +```python +import pytest + +# decorator to never skip this test +@pytest.mark.do_not_accelerate +def test_function(): + # ... +``` + +The decorator `@pytest.mark.do_not_accelerate` ensures that the `test_function()` is never skipped, even where there are no code changes in its dependencies. + + +## unittest +If you're using [unittest](https://docs.python.org/3/library/unittest.html), you can tell YourBase Test Acceleration to never skip specific tests using decorators as shown in the below example: + +```python +import yourbase.plugins.unittest as yourbase + +# decorator to never skip this test +@yourbase.do_not_accelerate +class TestClass(unittest.TestCase): + def test_function(): + # ... +``` \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/verify-results.md b/docs/test acceleration/advanced-usage/verify-results.md index 76903fd..bcaae47 100644 --- a/docs/test acceleration/advanced-usage/verify-results.md +++ b/docs/test acceleration/advanced-usage/verify-results.md @@ -1,22 +1,45 @@ --- layout: default -title: Verify results +title: Verify test selection nav_order: 2 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/verify-results +permalink: /test-acceleration/advanced-usage/verify --- -# Verify results -YourBase Test Acceleration test selection includes an "Observation Mode" which allows you to test-drive its test selection without actually skipping any tests. +# Verify test selection +{: .no_toc } +To test-drive YourBase Test Acceleration’s test selection without actually skipping any tests, you can run it under the “Observation Mode”. When “Observation Mode” is set, in addition to running all the tests, YourBase Test Acceleration will also monitor if its test selection would have skipped any tests that would have failed. As its output, it will print out the names of any tests that would have been incorrectly skipped. -To verify results follow the below steps: +--- + +Table of contents +{: .text-delta } +1. TOC +{:toc} + +--- -1. Enable Observation Mode -Before running your tests, set `YOURBASE_OBSERVATION_MODE=true` in your environment. +## Verification steps +You can follow the below steps to verify: -2. Check the output of running YourBase Test Acceleration in Observation mode +1. [Install YourBase Test Acceleration](../install.md). +2. Enable [Observation Mode](../reference/configuration-options.md/#yourbase_observation_mode) in your environment. For example, if you use a bash shell to set environment variables, you can set is as follows: + ```sh + export YOURBASE_OBSERVATION_MODE=true + ``` + + Else if you use an environment specific configuration file, set [YOURBASE_OBSERVATION_MODE](../reference/configuration-options.md/#yourbase_observation_mode) in that file. + +3. Run ALL your tests as usual. +4. Check your logs manually: - If YourBase Test Acceleration is accelerating tests correctly, it will log the total amount of time that could have been saved, to stdout or your log file. - - Else if YourBase Test Acceleration is accelerating tests incorrectly—by skipping a test(s) that would have failed, YourBase Test Acceleration will complain about this loudly and output the details of the errors in your log file or your shell-prompt. Note that if this happens, it means that there is a bug in YourBase Test Acceleration’s tracing or acceleration. Please also report these to bugs@yourbase.io. \ No newline at end of file + - Else if, it's accelerating tests incorrectly, i.e. it's skipping one or more tests that would have failed, then it'll complain about this loudly by outputting the details of the errors in your log file or your shell-prompt. _Note that if this happens, it means that there's a bug in YourBase Test Acceleration’s tracing or acceleration._ Please report these to bugs@yourbase.io. +5. Ensure that you disable [YOURBASE_OBSERVATION_MODE](../reference/configuration-options.md/#yourbase_observation_mode) only after YourBase Test Acceleration accelerates tests correctly. + +--- + +## Recommendation +We strongly recommend that you run YourBase Test Acceleration in the "Observation "Mode" every time before rolling out code-changes to your production. \ No newline at end of file diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying.md index 01b255c..b9886aa 100644 --- a/docs/test acceleration/deploying.md +++ b/docs/test acceleration/deploying.md @@ -7,17 +7,18 @@ permalink: /test-acceleration/deploying --- # Deploying - {: .no_toc } -This section contains guides to deploy the YourBase Test Acceleration library. - -## Table of contents - -{: .no_toc } +This section contains guides to deploy the YourBase Test Acceleration library. -1. TOC -{ :toc } +
+ + Table of contents + + {: .text-delta } +- TOC +{:toc} +
## Rollout recommendation To safely accelerate tests on production, we recommend that you first run YourBase Test Acceleration in its Observation Mode [link to Advanced usage → Observation mode], and manually verify the output logged [link to Advanced usage → Observation mode → Checking output]. @@ -25,7 +26,6 @@ To safely accelerate tests on production, we recommend that you first run YourBa We propose the rollout phases to look like the following: ### Phase 1: Test locally: On your development branch - {: .no_toc } This phase will synchronize your code’s dependency graph on your local machine storage. Hence, after completing this phase, you’ll see the tests on your branch running faster only on your local machine. @@ -37,6 +37,7 @@ This phase will synchronize your code’s dependency graph on your local machine 5. Run YourBase Test Acceleration ### Phase 2: Test in CI +{: .no_toc } This phase will synchronize your code’s dependency graph on remote storage for use by CI. Hence, after completing this phase, you’ll see the tests on your branch running faster on your CI as well. @@ -55,6 +56,7 @@ In your CI environment: 9. Run YourBase Test Acceleration for all your tests in CI ### Phase 3: Enable local test acceleration across your development team, starting with a group of beta users. +{: .no_toc } After this phase, you’ll see tests run faster for developers across the team because of them sharing their dependency graph. @@ -66,18 +68,18 @@ After this phase, you’ll see tests run faster for developers across the team b --- -## Logs -Our standard logs are all prefixed with [YB]. We print minimal logs unless explicitly requested using debug mode [Link to References → Configuration → YOURBASE_DEBUG]. +## Logging +We prefix all logs with `[YB]`. -To obtain more detailed debugging information, set the `YOURBASE_DEBUG` environment variable [link to Configuration Options section]. +By default, minimal logs are printed. To obtain more detailed debugging information, [set the YOURBASE_DEBUG environment variable](reference/configuration-options.md#yourbase_debug). --- ## Disable YourBase Test Acceleration -If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable `YOURBASE_DISABLE` to true: +If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable [YOURBASE_DISABLE](reference/configuration-options.md#yourbase_disable) to true: -```bash +```sh export YOURBASE_DISABLE=true ``` -To uninstall the package, see uninstall instructions [Link to Uninstall] \ No newline at end of file +To uninstall the package, [see the uninstall instructions](install.md#uninstall). \ No newline at end of file diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md index 89edcc4..7d53699 100644 --- a/docs/test acceleration/getting-started/index.md +++ b/docs/test acceleration/getting-started/index.md @@ -8,4 +8,4 @@ permalink: /test-acceleration/getting-started --- # Getting started -This section gives an introduction on how to get started with running YourBase Test Acceleration in supported testing frameworks. For a full walkthrough of the tool, see [Link to Try in pytest] +This section gives an introduction on how to get started with running YourBase Test Acceleration in supported testing frameworks. For a full walkthrough of the tool, [see here](pytest.md). diff --git a/docs/test acceleration/getting-started/pytest.md b/docs/test acceleration/getting-started/pytest.md index 136539b..e258b08 100644 --- a/docs/test acceleration/getting-started/pytest.md +++ b/docs/test acceleration/getting-started/pytest.md @@ -4,75 +4,92 @@ title: Pytest nav_order: 1 parent: Getting started grand_parent: Test acceleration -permalink: /test-acceleration/getting-started/python/pytest +permalink: /test-acceleration/getting-started/pytest --- # Try in pytest +{:.no_toc} -This page will walk you through how to run YourBase Test Acceleration with the pytest testing framework. The quickest way to get started with it is to use it in the sample project provided. +This page will walk you through how to run YourBase Test Acceleration with the [pytest testing framework](https://docs.pytest.org/en/6.2.x/). The quickest way to get started with it is to use it in the sample project provided. +
+ + Table of contents + + {: .text-delta } +1. TOC +{:toc} +
+ +--- + +## Introduction YourBase Test Acceleration’s pytest hooks ensure that, by default: -- It runs automatically on every invocation of pytest -- It runs without any additional configuration +- It runs automatically on every invocation of pytest. +- It runs without any additional configuration. - It runs irrespective of how you invoke the tests to run—be it via Makefile, Docker, or anything else—without any other setup. --- ## Prerequisites -Make sure that, on your machine or on your virtual environment, you have: -- Tests running successfully with pytest before installing YourBase Test Acceleration -- YourBase Test Acceleration installed [link to the installation section of Python] -- You have git installed +Make sure that, on your machine or on your virtual environment: +- Tests are running successfully with [pytest](https://docs.pytest.org/en/6.2.x/) before installing YourBase Test Acceleration. +- [YourBase Test Acceleration is installed](../install.md). +- [Git is installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). --- ## Example usage in a sample project ### Step 1: Set up the specified sample project on your machine as shown below: +{:.no_toc} Open a new shell prompt, and checkout this sample project from git -```bash +```sh git clone https://github.com/Unleash/unleash-client-python ``` From your shell prompt, navigate to the directory where you checked out the project -```bash +```sh cd unleash-client-python ``` Install dependencies -```bash +```sh pip install -r requirements.txt ``` ### Step 2: Execute a command to run all the tests of the project +{:.no_toc} -```bash +```sh pytest tests ``` Let’s look at the output. -All messages that are prefixed with “[YB]” are logged by the YourBase Test Acceleration library. Note, that one of these messages says “No function-level dependency graph found; all tests will be run and traced.”—this is a cold run. During a cold run, YourBase Test Acceleration will build a dependency graph based on the tests that are executed. This dependency graph contains relationships between individual tests and the code they call. +All messages that are prefixed with `[YB]` are logged by the YourBase Test Acceleration library. Note, that one of these messages says “No function-level dependency graph found; all tests will be run and traced.”—this is a cold run. During a cold run, YourBase Test Acceleration will build a dependency graph based on the tests that are executed. This dependency graph contains relationships between individual tests and the code they call. ### Step 3: Without making any code change, execute the command to run all the tests again +{:.no_toc} -```bash +```sh pytest tests ``` -Let’s look at the messages logged by YourBase Test Acceleration. One of the messages says: “[YB] No code changed. Running only new tests, if any.” +Let’s look at the messages logged by YourBase Test Acceleration. One of the messages says: `[YB]` No code changed. Running only new tests, if any.” Since you ran the tests without changing any code, YourBase Test Acceleration skipped all the tests to finish the run much more quickly than the last time—compare the time taken by both runs. ### Step 4: Make a code change and execute the command to run all tests again as shown below: +{:.no_toc} Open `tests/unit_tests/test_features.py` in your text editor -```bash +```sh vim tests/unit_tests/test_features.py ``` @@ -84,7 +101,7 @@ print(“Checking YourBase Test Acceleration after a code-change...”) Run the tests again using: -```bash +```sh pytest tests ``` @@ -94,7 +111,7 @@ If you modified the same function as above then your output will closely match t Look for the logs traced by YourBase Test Acceleration. You’ll see a message telling you how many functions have been altered and how many tests were affected. If you’re following the steps as is, you should see: -```bash +```sh [YB] 1 function differs from the dependency graph [YB] ~ tests/unit_tests/test_features.py#test_create_feature_true [YB] Function-level dependency graph found 1 test affected @@ -105,6 +122,8 @@ You can see form the logs that the existing dependency graph [link to How it wor --- ## Conclusion -You just ran YourBase Test Acceleration on a project that uses the pytest testing framework. And now, you’re ready to use YourBase Test Acceleration in your own project. +{:.no_toc} + +You just ran YourBase Test Acceleration on a project that uses the pytest testing framework. Now, you’re ready to use it in your own project. -Note that if your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly. \ No newline at end of file +_Note that if your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly._ \ No newline at end of file diff --git a/docs/test acceleration/getting-started/unittest.md b/docs/test acceleration/getting-started/unittest.md index 6666e1b..0c90265 100644 --- a/docs/test acceleration/getting-started/unittest.md +++ b/docs/test acceleration/getting-started/unittest.md @@ -4,27 +4,39 @@ title: Unittest nav_order: 2 parent: Getting started grand_parent: Test acceleration -permalink: /test-acceleration/getting-started/python/unittest +permalink: /test-acceleration/getting-started/unittest --- # Try in unittest -This section notes how to run YourBase Test Acceleration in projects that use the unittest testing framework. For a deeper walkthrough with a sample open source project that uses pytest, see . +{:.no_toc} + +This section notes how to run YourBase Test Acceleration in your own project that uses the [unittest testing framework](https://docs.python.org/3/library/unittest.html). For a deeper walkthrough of the library using a sample project, [see here](pytest.md). + +
+ + Table of contents + + {: .text-delta } +1. TOC +{:toc} +
--- ## Prerequisites -Make sure that, on your machine or on your virtual environment, you have: -- Tests running successfully with unittest before installing YourBase Test Acceleration -- YourBase Test Acceleration installed [link to the installation section of Python] -- You have git installed - +Make sure that, on your machine or on your virtual environment: +- Tests are running successfully with [unittest](https://docs.python.org/3/library/unittest.html) before installing YourBase Test Acceleration. +- [YourBase Test Acceleration is installed](../install.md). +- [Git is installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). + --- ## Example usage in your project ### Step 1: Attach yourbase to unittest. +{:.no_toc} -For example, in your `tests/__init__.py file`, copy-paste the following: +In your `tests/__init__.py file`, copy-paste the following: ```python import unittest @@ -34,20 +46,23 @@ yourbase.attach(unittest) ``` ### Step 2: Run your project’s tests +{:.no_toc} For example, to run all the tests for the project, use: -```bash +```sh python -m unittest discover ``` -Note: If your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly. +_Note: If your tests are going to take a while to run, you can run just a subset of your tests. Running a subset of tests will create a dependency graph just for those tests, so you can see YourBase Test Acceleration in action more quickly._ -###Step 3: Re-run your tests +### Step 3: Re-run your tests +{:.no_toc} -Without making a code change, run your tests again using the same command as in step #2. Here, YourBase Test acceleration ensures that no tests are run since no code was changed. +Without making a code change, run your tests again using the same command as in [step #2](#step-2-run-your-projects-tests). You'll see that no tests are run. Here, since no code was changed, YourBase Test acceleration ensures that no tests are run. -### Step 4: Make a code change in any one of your tests. +### Step 4: Make a code change in any one of your tests +{:.no_toc} You can simply add a print statement like below: @@ -56,6 +71,7 @@ print(“Checking YourBase Test Acceleration after a code-change...”) ``` -### Step 5: Run your tests again using the same command as in step #2. +### Step 5: Run your tests again +{:.no_toc} -Here, YourBase Test acceleration ensures that only the one test that you modified is run. +Use the same command as in [step #2](#step-2-run-your-projects-tests) to run your tests. Here, YourBase Test acceleration ensures that only the one test that you modified is run. diff --git a/docs/test acceleration/how-it-works.md b/docs/test acceleration/how-it-works.md index 262b705..dd6d261 100644 --- a/docs/test acceleration/how-it-works.md +++ b/docs/test acceleration/how-it-works.md @@ -7,10 +7,12 @@ permalink: /test-acceleration/how-it-works --- # How it works -The following documents clarify concepts, design decisions, and technical constraints in YourBase Test Acceleration. This is a great place to understand fundamental ideas and philosophy. +The following documents clarify concepts, design decisions, and technical constraints in YourBase Test Acceleration. + +--- ## Dependency Graph -Learn more about the YourBase Test Acceleration Dependency Graph at https://yourbase.io/technology/dependency-graph/. +Learn about the YourBase Test Acceleration [Dependency Graph here](https://yourbase.io/technology/dependency-graph/). ## Shared Dependency Graph -Learn more about how to enable team-based acceleration gains with the YourBase Test Acceleration Dependency Graph at https://yourbase.io/technology/shared-graph. \ No newline at end of file +Learn about team-based acceleration gains using a [Shared Dependency Graph here](https://yourbase.io/technology/shared-graph). \ No newline at end of file diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md index 3cb52ae..62859c0 100644 --- a/docs/test acceleration/index.md +++ b/docs/test acceleration/index.md @@ -11,13 +11,15 @@ permalink: /test-acceleration {:.no_toc} YourBase Test Acceleration is a library for shortening your test run-times by up to 90%. It hooks into your testing framework to intelligently select which tests should be run for a given code change. -### How does YourBase Test Acceleration reduce test times? +--- + +## How does YourBase Test Acceleration reduce test times? At the core of YourBase Test Acceleration is the YourBase Dependency Graph that maps which files and functions each of your tests depends on. Every time you run your tests, YourBase Test Acceleration library will load the optimal dependency graph to select and run only the tests that pertain to your code changes, and safely avoid running any unrelated tests. As a result, your builds execute only the optimal fraction of your total tests and finish much faster. --- ## Is it compatible with your tech-stack and infrastructure? -The library currently supports [testing frameworks for Python](test-acceleration/system-requirements#supported-languages--testing-frameworks). See here for a complete list of system requirements [link to the System Requirements section]. +The library currently supports [testing frameworks for Python](system-requirements#supported-languages--testing-frameworks). Check the [complete list of system requirements here](system-requirements). --- @@ -27,14 +29,14 @@ The library supports unit tests and integration tests that call code from within --- ## Will it provide benefits at your scale? -The library has the most benefits for use-cases where test runs are taking over 10 minutes. Our customers have reduced their test run times by up to 90%, for example, where one customer was able to skip over 99% of the 11,000+ tests in the suite using our library. +The library has the most benefits for use-cases where test runs are taking over 10 minutes. [Our customers have reduced their test run times by up to 90%](https://yourbase.io/case-studies/), for example, where one customer was able to skip over 99% of the 11,000+ tests in the suite using our library. --- ## Is it secure? -Yes. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. See here for more details [link to the Security section]. +Yes. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. [Learn more about security here](security.md). --- ## Is it stable? -The library is currently in Beta. While we are confident in the reliability of our offering, we are making improvements all the time. If you identify any issues, please let us know at support@yourbase.io and we will look into them quickly. +The library is currently in Beta. While we're confident in the reliability of our offering, we're making improvements all the time. If you identify any issues, please let us know at support@yourbase.io and we will look into them quickly. diff --git a/docs/test acceleration/install.md b/docs/test acceleration/install.md index 932681c..ae9a756 100644 --- a/docs/test acceleration/install.md +++ b/docs/test acceleration/install.md @@ -1,6 +1,6 @@ --- layout: default -title: Install +title: Installation nav_order: 2 parent: Test acceleration has_children: false @@ -10,31 +10,43 @@ permalink: /test-acceleration/install # Install {:.no_toc} -## Table of contents -{:.no_toc} - -1. TOC +
+ + Table of contents + + {: .text-delta } +1. TOC {:toc} +
+ +--- ## Prerequisites -- YourBase Test Acceleration library supports your technical stack and infrastructure [link to the System Requirements section]. +Make sure that the YourBase Test Acceleration library [supports your technical stack and infrastructure](system-requirements.md). + +--- ## Installation YourBase Test Acceleration library can be installed with either pip or poetry package managers. + ### Using pip -If you use pip, you can install YourBase Test Acceleration with: +If you use [pip](https://pip.pypa.io/en/stable/), you can install YourBase Test Acceleration with: -```pip install yourbase``` +```sh +pip install yourbase +``` To check if YourBase Test Acceleration is installed, run the following command in a shell prompt: -```pip show yourbase``` +```sh +pip show yourbase +``` If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. -```bash +```sh Name: yourbase Version: 5.2.4 Summary: Skip tests based on tracing data @@ -47,23 +59,24 @@ Requires: six, boto3, python-dateutil, coverage, pastel, requests Required-by: ``` +--- ### Using poetry -If you use poetry, you can install YourBase Test Acceleration with: +If you use [poetry](https://python-poetry.org/docs/), you can install YourBase Test Acceleration with: -```bash +```sh poetry add yourbase ``` To check if YourBase Test Acceleration is installed, run the following command in a shell prompt: -```bash +```sh poetry show yourbase ``` If the installation is successful, you should see an output similar to what’s shown below, having a Version key that displays a valid version of YourBase Test Acceleration. -```bash +```sh Name: yourbase Version: 5.2.4 Summary: Skip tests based on tracing data @@ -76,24 +89,31 @@ Requires: six, boto3, python-dateutil, coverage, pastel, requests Required-by: ``` -## Installation recommendations +--- + +## Recommendations ### Use virtual environment - {: .no_toc } -We recommend that you install YourBase Test Acceleration in a clean virtual environment. See here to learn how to set up a Python virtual environment. +We recommend that you install YourBase Test Acceleration in a clean virtual environment. [Learn to set up a Python virtual environment here](https://docs.python.org/3/tutorial/venv.html). + +--- ## Uninstall If for any reason, you want to completely remove YourBase Test Acceleration, simply uninstall the package: -Using pip: +Using [pip](https://pip.pypa.io/en/stable/): -```pip uninstall yourbase``` +```sh +pip uninstall yourbase +``` -Or, using poetry: +Or, using [poetry](https://python-poetry.org/docs/): -```poetry remove yourbase``` +```sh +poetry remove yourbase +``` diff --git a/docs/test acceleration/system-requirements.md b/docs/test acceleration/system-requirements.md index 1b836f7..ebe00a6 100644 --- a/docs/test acceleration/system-requirements.md +++ b/docs/test acceleration/system-requirements.md @@ -9,7 +9,9 @@ permalink: /test-acceleration/system-requirements # System requirements -YourBase Test Acceleration library works seamlessly with the CI system of your choice and is agnostic to where you host your application—be it cloud, on-premise, or offline. +YourBase Test Acceleration works seamlessly with the CI system of your choice and is agnostic to where you host your application—be it cloud, on-premise, or offline. + +--- ## Supported version control tools The library can accelerate tests only for codebases that are version-controlled using [Git](https://git-scm.com/). @@ -23,7 +25,7 @@ The library can accelerate tests only for codebases that are version-controlled - [pytest](https://docs.pytest.org/en/6.2.x/) - [unittest](https://docs.python.org/3/library/unittest.html) -_Note: Any web frameworks, such as Django, that are built atop the above testing frameworks, are also supported._ +_Note: Any web frameworks, such as [Django](https://www.djangoproject.com/), that are built atop the above testing frameworks, are also supported._ ## Compatible code coverage tools The library supports integration with the following code coverage reporting tools without changing your workflow, for example: From 17a6eba542b6374ce214df624c3b9d2878d5a963 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 17:14:33 +0530 Subject: [PATCH 17/24] Added links in Advnced usage --- .../accelerate-parallelized-tests.md | 26 ++++--- .../accelerate-tests-across-developers.md | 6 +- .../advanced-usage/accelerate-tests-in-ci.md | 69 ++++++++----------- .../integrate-code-coverage-tools.md | 45 +++++++----- .../advanced-usage/mark-test-unskippable.md | 2 +- .../advanced-usage/verify-results.md | 2 +- 6 files changed, 78 insertions(+), 72 deletions(-) diff --git a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md index b285011..83ee107 100644 --- a/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md +++ b/docs/test acceleration/advanced-usage/accelerate-parallelized-tests.md @@ -9,27 +9,33 @@ permalink: /test-acceleration/advanced-usage/accelerate-parallelized-tests # Accelerate parallelized tests -YourBase Test Acceleration integrates smoothly with your existing parallelization setup in your CI. On top of your parallelization setup, YourBase Test Acceleration will: -1. Split your tests across shards -2. Accelerate tests running on each shard +You can integrate YourBase Test Acceleration smoothly with your existing parallelization setup in your CI. You can use YourBase Test Acceleration to: +1. Split your tests across shards, and +2. Accelerate tests that run on each shard + +## Assumptions: +You’re already successfully running parallel tests across multiple shards. ## Prerequisites: -- You’re already successfully running tests across multiple shards. -- You've removed any test splitting solution that you may already have in-place. This is required because YourBase Test Acceleration is effective only when the test-splitting solution is sticky—tests don’t get reshuffled across shards when other tests are added or removed. However, since most test-splitting solutions aren’t sticky, YourBase Test Acceleration provides its own built-in sticky test-splitting solution, which you will learn to use in the following section. +You'll need remove any test splitting solution that you may already have in-place. + +This is required because YourBase Test Acceleration is effective only when the test-splitting solution is sticky, i.e. tests don’t get reshuffled across shards when other tests are added or removed. Since most test-splitting solutions aren’t sticky, YourBase Test Acceleration provides its own built-in sticky test-splitting solution. -## Steps to accelerate already-parallelized tests: +## Steps to accelerate parallelized tests: 1. Remove your existing test splitting tools, if any. It should appear as if each shard will run the entire test suite. -2. Set YOURBASE_COHORT_COUNT [link to References → Configuration → YOURBASE_COHORT_COUNT] to your number of cohorts / shards -3. Set YOURBASE_ACTIVE_COHORT [link to References → Configuration → YOURBASE_ACTIVE_COHORT] to the ID of the current cohort / shard – starting from 1. For example, if you have 5 shards and are running the 4th shard, you’d set: +2. Set [YOURBASE_COHORT_COUNT](../reference/configuration-options.md#yourbase_cohort_count) to your number of cohorts / shards +3. Set [YOURBASE_ACTIVE_COHORT](../reference/configuration-options.md#yourbase_active_cohort) to the ID of the current cohort / shard – starting from 1. ```bash + # For example, if you have 5 shards and are running the 4th shard, you’ll set: + YOURBASE_COHORT_COUNT = 5 YOURBASE_ACTIVE_COHORT = 4 ``` 4. Run your tests as usual. -YourBase Test Acceleration will now just-in-time select and run only the tests that are in the current cohort. This selection will be consistent between runs—given the same cohort ID and total cohorts, a test will always be selected to be run on the same shard for life. +YourBase Test Acceleration will now just-in-time select and run only the tests that are in the current cohort. This selection will be consistent between runs, i.e. given the same cohort ID and total cohorts, a test will always be selected to be run on the same shard for life. -## Pro tip +## Pro tip: As an added benefit of YouBase’s sticky splitting of tests, you can schedule a build for each of the sharded pools of tests—to run sharded builds. This works smoothly because YourBase Test Acceleration will merge graphs from multiple shards, for the same commit, which can be used in future builds. diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md index 64b00da..9a23e45 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-across-developers.md @@ -4,8 +4,10 @@ title: Accelerate tests locally across developers nav_order: 5 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/accelerate-tests-locally-across-developers +permalink: /test-acceleration/accelerate-locally-across-developers --- # Accelerate tests locally across developers -To have developers, who are working on the same branch, access a shared dependency graph so that tests run faster for them all, you can set them up to use the remote cache following the same steps as in Advanced usage → Accelerate tests in CI for your development environment. \ No newline at end of file +Developers who are working on the same branch can gain acceleration from the dependency graphs created on their peers' local machines by accessing a [shared dependency graph](../how-it-works.md#shared-dependency-graph). + +You can set it up using the same steps that are used to [accelerate tests in your CI environment](accelerate-tests-in-ci.md). \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md index 46c7802..7876781 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md @@ -4,64 +4,53 @@ title: Accelerate tests in CI nav_order: 4 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/accelerate-tests-in-ci +permalink: /test-acceleration/integrate-ci --- -# Accelerate tests in CI -YourBase Test Acceleration harnesses its dependency graph to accelerate your tests. When you run tests on your local machine, with YourBase Test Acceleration enabled, by default, your dependency graph is stored locally—thus providing only local acceleration. +# Integrate with CI +YourBase Test Acceleration uses a [dependency graph](../how-it-works.md#dependency-graph) to accelerate your tests. When you run tests on your local machine, with YourBase Test Acceleration enabled, by default, your dependency graph is stored locally—thus provides only for local acceleration. -But the true power of YourBase Test Acceleration comes in when tests can be accelerated in your CI environment. This requires the dependency graph to be accessible by your CI environment—referred to as the Shared Dependency Graph from now on. +But the true power of YourBase Test Acceleration comes in when tests can be accelerated in your CI environment. This requires the [dependency graph](../how-it-works.md#dependency-graph) to be accessible by your CI environment—referred to as [Shared Dependency Graph](../how-it-works.md#shared-dependency-graph) from now on. -The following steps guide you to accelerate tests in CI: +The following section guides you to accelerate tests in CI. -## Step 1: Set storage location for shared dependency graph -YourBase Test Acceleration currently supports storing shared graphs in AWS S3 buckets. The following sections help you set up a shared dependency graph for your project. +## Steps to integrate with CI -### Using S3 bucket -Within your project, set the following environment variable to your S3 bucket using the `s3://` syntax. +### Step 1: Set up shared dependency graph +YourBase Test Acceleration currently supports storing [shared dependency graphs](../how-it-works.md#shared-dependency-graph) only in AWS S3 buckets. The following sections help you set up your project to use a shared dependency graph in your CI environment. -#### 1. Set S3 Bucket name +First, set the [YOURBASE_REMOTE_CACHE](../reference/configuration-options.md#yourbase_remote_cache) variable in your environment to a valid S3 bucket location. -```YOURBASE_REMOTE_CACHE=s3://[/key/prefix]``` +```sh +YOURBASE_REMOTE_CACHE=s3://[/key/prefix] -where is an S3 bucket that your machine(s) has Get/Put/List access to. - -#### 2. Set AWS credentials -You can configure YourBase Test Acceleration to use either the system AWS credentials, or you can also specify different credentials that are specific to YourBase Test Acceleration. +# where is an S3 bucket that your machine(s) has Get/Put/List access to. +``` -##### Option 1: Use the system AWS credentials -To use your systems’ AWS credentials, export the following in your environment: +Next, set AWS credentials to be used by YourBase Test Acceleration. -```bash -export AWS_ACCESS_KEY_ID= -export AWS_SECRET_ACCESS_KEY= -``` +By default, YourBase Test Acceleration uses the system AWS credentials as specified in the environment variables: [AWS_ACCESS_KEY_ID](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list) and [AWS_SECRET_ACCESS_KEY](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list). -##### Option 2: Use YourBase Test Acceleration-specific AWS credentials -If you want YourBase Test Acceleration to use different credentials (or if you're setting the system AWS credentials to mock values for your tests or CI), you can set these YourBase Test Acceleration-specific environment variables instead: +Alternately, you can specify different credentials to be used exclusively by YourBase Test Acceleration by setting the following environment variables: -```bash +```sh export YOURBASE_AWS_ACCESS_KEY_ID= export YOURBASE_AWS_SECRET_ACCESS_KEY= ``` + +_Note: If YourBase Test Acceleration specific environment variables - [YOURBASE_AWS_ACCESS_KEY_ID](../reference/configuration-options.md#yourbase_aws_access_key_id) and [YOURBASE_AWS_SECRET_ACCESS_KEY](../reference/configuration-options.md#yourbase_aws_secret_access_key) are set, YourBase Test Acceleration uses them instead of the system credentials._ -Note: - -By default, YourBase Test Acceleration uses the system AWS credentials - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. -However, if YourBase Test Acceleration specific credentials are set - `YOURBASE_AWS_ACCESS_KEY_ID` and `YOURBASE_AWS_SECRET_ACCESS_KEY`, YourBase Test Acceleration uses them instead of the system credentials. +### Step 2: Install YourBase Test Acceleration in your CI environment +Add YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. -## Step 2: Install YourBase Test Acceleration in your CI environment -Add yourbase to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your CI environment. - -## Step 3: Run tests +### Step 3: Run tests Run tests as usual. -The above steps will set up YourBase Test Acceleration to synchronize dependency graphs against the specified S3 location when your tests run on your CI environment. - ---- +## Conclusion +The above steps will set up YourBase Test Acceleration to synchronize dependency graphs against the specified storage location when your tests run on your CI environment. -## Points to note -- Your code will never be uploaded to the configured bucket. Only the dependency graphs are uploaded to the bucket. Neither your code nor your dependency graphs will touch YourBase Test Acceleration servers. -- A dependency graph that is generated from successful builds and using clean working trees will be synchronized to the specified S3 location. This graph is accessed for future test runs if a local cache is not present. -- Dependency graphs run against uncommitted code changes will be stored only locally, i.e., they can’t be shared over S3. -- It is safe to share one bucket between multiple repositories, even if the repositories use different languages. +_Note:_ +_A dependency graph is synchronized with the specified remote storage location only when:_ +- _Only when it's generated from a successful build, and_ +- _Only if it's created from a clean working tree, and_ +- _Only if it's created from committed code changes. A dependency graph that is created from uncommitted code changes is stored only locally, i.e., it can’t be synchronized against a remote location._ \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md index 9f36b23..1eae98e 100644 --- a/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md +++ b/docs/test acceleration/advanced-usage/integrate-code-coverage-tools.md @@ -4,40 +4,49 @@ title: Integrate code coverage tools nav_order: 3 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/integrate-code-coverage-tools +permalink: /test-acceleration/integrate-code-coverage-tools --- # Integrate code coverage tools {: .no_toc } -Since YourBase Test Acceleration is designed to avoid test runs that do not need to be executed, the percentage covered as reported by your code coverage tool will likely be lower than what is actually covered as a result. +YourBase Test Acceleration supports integration with the following code coverage reporting tools without changing your workflow, for example: +- [Coverage](https://coverage.readthedocs.io/en/coverage-5.5/) +- [pytest-cov](https://pypi.org/project/pytest-cov/) +- [SonarQube](https://www.sonarqube.org/) +- [Codecov](https://about.codecov.io/) -This decrease in the coverage reported by your tool will affect you if your CI is configured to fail a build on the basis of code coverage percentage. +However, since YourBase Test Acceleration is designed to avoid test runs that do not need to be executed, the test coverage percentage as reported by your code coverage tool will be lower than what is actually covered. -The following sections lists ways to circumvent this problem. - -## Configure your code coverage tool - -You can circumvent this by making configuration changes in your code coverage tool, so that YourBase Test Acceleration can fill missing coverage data from previous runs. -However, this feature of YourBase Test Acceleration works only on specific versions of the supported code coverage tools. The following sections provide details on this. +This may cause your CI builds to fail if they're configured to pass only when the test coverage is above a specified threshold. The following sections lists ways in which you can circumvent this problem. +--- +## Option 1: Configure code coverage tool to fill missing coverage data ---- +This feature allows you to configure your code coverage tool such that YourBase Test Acceleration can fill missing coverage data from previous runs. -## Configure your builds +The feature is available only on the following code coverage tools: -YourBase Test Acceleration supports the following code coverage tools. -- TOC -{:toc} +### 1. Coverage 5.5+ -## Coverage -YourBase Test Acceleration is compatible with all versions of Coverage out-of-the-box. To ensure that coverage reports account for skipped tests as well: -1. Use Coverage 5.5+ +To ensure that coverage reports account for skipped tests as well: +1. Use Coverage 5.5+ 2. Set the following in your `.coveragerc` file: + ```python [run] relative_files = true ``` -Note: Prior to Coverage 5.5, coverage reports will only include the tests run by YourBase Test Acceleration. Tests that were skipped by YourBase Test Acceleration will be omitted from coverage reporting, thereby decreasing your coverage percentage. +_Note: Prior to Coverage 5.5, coverage reports will only include the tests run by YourBase Test Acceleration. Tests that were skipped by YourBase Test Acceleration will be omitted from coverage reporting, thereby decreasing your coverage percentage._ + +You can configure your code coverage tool such that YourBase Test Acceleration can fill missing coverage data from previous runs. + +--- + +## Option 2: Reconfigure build-failure threshold + +On your CI, you can reduce the build-pass threshold for the percentage of tests covered, so that you can use YourBase Test Acceleration seamlessly. + +### Note: We highly recommend that you make use of [Option 1](#option-1-configure-code-coverage-tool-to-fill-missing-coverage-data) instead of [Option 2](#option-2-reconfigure-build-failure-threshold). \ No newline at end of file diff --git a/docs/test acceleration/advanced-usage/mark-test-unskippable.md b/docs/test acceleration/advanced-usage/mark-test-unskippable.md index babace4..9bedba4 100644 --- a/docs/test acceleration/advanced-usage/mark-test-unskippable.md +++ b/docs/test acceleration/advanced-usage/mark-test-unskippable.md @@ -4,7 +4,7 @@ title: Mark a test as unskippable nav_order: 1 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/mark-test-unskippable +permalink: /test-acceleration/mark-test-unskippable --- # Mark a test as unskippable diff --git a/docs/test acceleration/advanced-usage/verify-results.md b/docs/test acceleration/advanced-usage/verify-results.md index bcaae47..267610d 100644 --- a/docs/test acceleration/advanced-usage/verify-results.md +++ b/docs/test acceleration/advanced-usage/verify-results.md @@ -4,7 +4,7 @@ title: Verify test selection nav_order: 2 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/advanced-usage/verify +permalink: /test-acceleration/verify --- # Verify test selection From 8cb4375d5c4e1c638f9b322ee70b1505ca769b21 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 17:54:38 +0530 Subject: [PATCH 18/24] Added all links --- .../advanced-usage/accelerate-tests-in-ci.md | 39 +++++---- .../reference/common-issues.md | 2 +- .../reference/configuration-options.md | 87 ++++++++++--------- docs/test acceleration/releases.md | 4 +- docs/test acceleration/security.md | 12 ++- 5 files changed, 78 insertions(+), 66 deletions(-) diff --git a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md index 7876781..b551d83 100644 --- a/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md +++ b/docs/test acceleration/advanced-usage/accelerate-tests-in-ci.md @@ -4,7 +4,7 @@ title: Accelerate tests in CI nav_order: 4 parent: Advanced usage grand_parent: Test acceleration -permalink: /test-acceleration/integrate-ci +permalink: /test-acceleration/advanced-usage/accelerate-tests-in-ci --- # Integrate with CI @@ -19,26 +19,27 @@ The following section guides you to accelerate tests in CI. ### Step 1: Set up shared dependency graph YourBase Test Acceleration currently supports storing [shared dependency graphs](../how-it-works.md#shared-dependency-graph) only in AWS S3 buckets. The following sections help you set up your project to use a shared dependency graph in your CI environment. -First, set the [YOURBASE_REMOTE_CACHE](../reference/configuration-options.md#yourbase_remote_cache) variable in your environment to a valid S3 bucket location. +1. Set YOURBASE_REMOTE_CACHE +Set [YOURBASE_REMOTE_CACHE](../reference/configuration-options.md#yourbase_remote_cache) in your environment to a valid S3 bucket location. -```sh -YOURBASE_REMOTE_CACHE=s3://[/key/prefix] - -# where is an S3 bucket that your machine(s) has Get/Put/List access to. -``` - -Next, set AWS credentials to be used by YourBase Test Acceleration. - -By default, YourBase Test Acceleration uses the system AWS credentials as specified in the environment variables: [AWS_ACCESS_KEY_ID](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list) and [AWS_SECRET_ACCESS_KEY](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list). - -Alternately, you can specify different credentials to be used exclusively by YourBase Test Acceleration by setting the following environment variables: - -```sh -export YOURBASE_AWS_ACCESS_KEY_ID= -export YOURBASE_AWS_SECRET_ACCESS_KEY= -``` + ```sh + YOURBASE_REMOTE_CACHE=s3://[/key/prefix] -_Note: If YourBase Test Acceleration specific environment variables - [YOURBASE_AWS_ACCESS_KEY_ID](../reference/configuration-options.md#yourbase_aws_access_key_id) and [YOURBASE_AWS_SECRET_ACCESS_KEY](../reference/configuration-options.md#yourbase_aws_secret_access_key) are set, YourBase Test Acceleration uses them instead of the system credentials._ + # where is an S3 bucket that your machine(s) has Get/Put/List access to. + ``` + +2. Set the AWS credentials to be used by YourBase Test Acceleration. + + By default, YourBase Test Acceleration uses the system AWS credentials as specified in the environment variables: [AWS_ACCESS_KEY_ID](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list) and [AWS_SECRET_ACCESS_KEY](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list). + + Alternately, you can specify different credentials to be used exclusively by YourBase Test Acceleration by setting the following environment variables: + + ```sh + export YOURBASE_AWS_ACCESS_KEY_ID= + export YOURBASE_AWS_SECRET_ACCESS_KEY= + ``` + + _Note: If YourBase Test Acceleration specific environment variables - [YOURBASE_AWS_ACCESS_KEY_ID](../reference/configuration-options.md#yourbase_aws_access_key_id) and [YOURBASE_AWS_SECRET_ACCESS_KEY](../reference/configuration-options.md#yourbase_aws_secret_access_key) are set, YourBase Test Acceleration uses them instead of the system credentials._ ### Step 2: Install YourBase Test Acceleration in your CI environment Add YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. diff --git a/docs/test acceleration/reference/common-issues.md b/docs/test acceleration/reference/common-issues.md index ce8b2ac..be78339 100644 --- a/docs/test acceleration/reference/common-issues.md +++ b/docs/test acceleration/reference/common-issues.md @@ -4,7 +4,7 @@ title: Common issues nav_order: 2 parent: Reference grand_parent: Test acceleration -permalink: /test-acceleration/reference/common-issues +permalink: /test-acceleration/common-issues --- # Common issues diff --git a/docs/test acceleration/reference/configuration-options.md b/docs/test acceleration/reference/configuration-options.md index c1114ec..1863c56 100644 --- a/docs/test acceleration/reference/configuration-options.md +++ b/docs/test acceleration/reference/configuration-options.md @@ -12,48 +12,54 @@ permalink: /test-acceleration/reference/configuration-options This page documents the available configuration options. These operate as environment variables that should be set the same way environment variables are set for your deployment. -## Table of contents -{: .no_toc .text-delta } - +
+ + Table of contents + + {: .text-delta } - TOC {:toc} +
-### YOURBASE_ACCEPT_TOS +## YOURBASE_ACCEPT_TOS `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` When set, YourBase Test Acceleration will consider the terms of service permanently accepted for your organization, and will not output terms of service agreement prompts or info messages. This is helpful, for example, when rolling out YourBase Test Acceleration to CI for an organization. -### YOURBASE_ACTIVE_COHORT +--- + +## YOURBASE_ACTIVE_COHORT `Type`: `integer` in the range `[1, $YOURBASE_COHORT_COUNT]` `Default`: `1` -When set alongside YOURBASE_COHORT_COUNT [link], tells YourBase Test Acceleration the cohort ID to run. Used for sharded or otherwise parallelized test suites. +When set alongside [YOURBASE_COHORT_COUNT](#yourbase_cohort_count), tells YourBase Test Acceleration the cohort ID to run. Used for sharded or otherwise parallelized test suites. -See here to learn to use this in your parallelized test-suite [link Advanced usage → Accelerate already parallelized tests]. +See [here](../advanced-usage/accelerate-parallelized-tests.md) to learn to use this in your parallelized test-suite. --- -### YOURBASE_COHORT_COUNT + +## YOURBASE_COHORT_COUNT `Type`: `integer` `Default`: `1` -When set alongside YOURBASE_ACTIVE_COHORT [link], tells YourBase Test Acceleration how many cohorts the tests should be split into. +When set alongside [YOURBASE_ACTIVE_COHORT](#yourbase_active_cohort), tells YourBase Test Acceleration how many cohorts the tests should be split into. -This pair of settings lets YourBase Test Acceleration work with your existing sharding or parallelization setup. See here to learn to use this setting [link Advanced usage → Accelerate already parallelized tests]. +This pair of settings lets YourBase Test Acceleration work with your existing sharding or parallelization setup. See here to [learn to use it in your parallelized test setup](../advanced-usage/accelerate-parallelized-tests.md). --- -### YOURBASE_LICENSE_KEY +## YOURBASE_LICENSE_KEY `Type`: `opaque string` `Default`: `(unset)` -When set to a valid license key, YourBase Test Acceleration acceleration will be unlocked for use after the end of the free trial. Email hi@yourbase.io to obtain a license key. +When set to a valid license key, YourBase Test Acceleration will be unlocked for use after the end of the free trial. Email hi@yourbase.io to obtain a license key. --- -### YOURBASE_OBSERVATION_MODE +## YOURBASE_OBSERVATION_MODE `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` @@ -61,59 +67,58 @@ When on, YourBase Test Acceleration will not skip tests, and instead, only recor When this setting is off, YourBase Test Acceleration will optimize your test run-time as usual. -It’s useful to turn this setting on when you’re testing YourBase Test Acceleration before taking it live with your codebase. See here to learn more [link to Advanced usage → Observation mode]. +It’s useful to turn this setting on when you’re testing YourBase Test Acceleration before taking it live with your codebase. Learn more about [how to use the Observation mode here](../advanced-usage/verify-results.md). --- -### YOURBASE_REMOTE_CACHE +## YOURBASE_REMOTE_CACHE `Type`: `uri` `Default`: `(unset)` -When set, this synchronizes dependency graphs generated only from clean working trees—dependency graphs generated from dirty working trees will not be synchronized as they can poison the cache. Use YOURBASE_SYNC_DIRTY [link] to override this behavior. +When set, this synchronizes [dependency graphs](../how-it-works.md#dependency-graph) generated only from clean working trees—dependency graphs generated from dirty working trees will not be synchronized as they can poison the cache. Use [YOURBASE_SYNC_DIRTY](#yourbase_sync_dirty) to override this behavior. -Example usage: -```bash +```sh # Without a key prefix export YOURBASE_REMOTE_CACHE=s3://my-bucket-name ``` -```bash +```sh # With a key prefix export YOURBASE_REMOTE_CACHE=s3://my-bucket-name/my/key/prefix ``` -This setting is recommended for use when using YourBase Test Acceleration in CI, as the filesystem will not be a dependable store for dependency graphs. See [link: Advanced usage → Accelerate using shared dependency graph on how to use this setting] +This setting is recommended for use when using YourBase Test Acceleration in CI, as the filesystem will not be a dependable store for [dependency graphs](../how-it-works.md#dependency-graph). Learn to [set up a shared dependency graph in your CI here](../advanced-usage/accelerate-tests-in-ci.md). --- -### YOURBASE_AWS_ACCESS_KEY_ID +## YOURBASE_AWS_ACCESS_KEY_ID `Type`: `AWS access key ID` `Default`: `(unset)` -When set alongside YOURBASE_AWS_SECRET_ACCESS_KEY [link], it forces YourBase Test Acceleration to use these credentials over system credentials when interacting with AWS. +When set alongside [YOURBASE_AWS_SECRET_ACCESS_KEY](#yourbase_aws_secret_access_key), it forces YourBase Test Acceleration to use these credentials over system credentials when interacting with AWS. -These environment variables are recommended for use if your AWS system credentials are fudged for the sake of your tests. +These environment variables are recommended for use if your AWS system credentials are fudged for the sake of your tests. Learn to use this [YourBase Test Acceleration specific AWS environment variables here](../advanced-usage/accelerate-tests-in-ci.md#step-1-set-up-shared-dependency-graph). --- -### YOURBASE_AWS_SECRET_ACCESS_KEY +## YOURBASE_AWS_SECRET_ACCESS_KEY `Type`: `AWS secret access key` `Default`: `(unset)` -When set alongside YOURBASE_AWS_ACCESS_KEY_ID [link], it forces YourBase Test Acceleration to use these credentials over AWS system credentials when interacting with AWS. +When set alongside [YOURBASE_AWS_ACCESS_KEY_ID](#yourbase_aws_access_key_id), it forces YourBase Test Acceleration to use these credentials over AWS system credentials when interacting with AWS. -These environment variables are recommended for use if your system credentials are mocked for the sake of your tests. +These environment variables are recommended for use if your system credentials are mocked for the sake of your tests. Learn to use this [YourBase Test Acceleration specific AWS environment variables here](../advanced-usage/accelerate-tests-in-ci.md#step-1-set-up-shared-dependency-graph) --- -### YOURBASE_DEBUG +## YOURBASE_DEBUG `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` When on, YourBase Test Acceleration will report significantly more internal information to stdout, stderr, and XDG (see the file returned by this expression): -```bash +```sh echo ${XDG_STATE_HOME-~/.local/state}/yourbase/python.log ``` @@ -121,35 +126,35 @@ This setting is most beneficial when collaborating with the YourBase Test Accele --- -### YOURBASE_DISABLE +## YOURBASE_DISABLE `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` When on, YourBase Test Acceleration will not load. -Enabling this setting and then manually attaching to a test framework using yourbase.attach produces undefined behavior. +Enabling this setting and then manually attaching to a test framework using `yourbase.attach` produces undefined behavior. --- -### YOURBASE_IGNORE_LOCAL_CACHE +## YOURBASE_IGNORE_LOCAL_CACHE `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` -When on, YourBase Test Acceleration will not look in the local filesystem for a dependency graph. +When on, YourBase Test Acceleration will not look in the local filesystem for a [dependency graph](../how-it-works.md#dependency-graph). -If you’ve set YOURBASE_REMOTE_CACHE [link] to a valid location, YourBase Test Acceleration will look for a dependency graph there, and synchronize with the same location. +If you’ve set [YOURBASE_REMOTE_CACHE](#yourbase_remote_cache) to a valid location, YourBase Test Acceleration will look up and [synchronize the dependency graph](../how-it-works.md#shared-dependency-graph) with the specified location. -Else, if YOURBASE_REMOTE_CACHE [link] is not set, YourBase Test Acceleration will do a cold run of your tests—it will run all the tests since it won’t be able to find any dependency graph. +Else, if [YOURBASE_REMOTE_CACHE](#yourbase_remote_cache) is not set, YourBase Test Acceleration will do a cold run of your tests—it will run all the tests since it won’t be able to find any dependency graph. This setting can be used if the local cache is expected to be poisoned. For instance, this can happen if cohorting is used against a local cache. --- -### YOURBASE_SYNC_DIRTY +## YOURBASE_SYNC_DIRTY `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` -When on, YourBase Test Acceleration will synchronize graphs even if the Git working tree is dirty. +When on, YourBase Test Acceleration will [synchronize dependency graphs](../how-it-works.md#shared-dependency-graph) even if the Git working tree is dirty. This setting is not recommended for use when you run YourBase Test Acceleration locally, as it will poison the remote cache. @@ -158,18 +163,20 @@ If that situation does not apply to you, do not enable this setting. --- -### YOURBASE_TELEMETRY +## YOURBASE_TELEMETRY `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `on` -When on, YourBase Test Acceleration will send anonymized telemetry data to api.yourbase.io over HTTPS for the purposes of improving the product. +When on, YourBase Test Acceleration will send anonymized telemetry data to `api.yourbase.io` over HTTPS for the purposes of improving the product. Note that, telemetry data never includes your code. Turn it off, if you want to opt out of sending usage statistics and error reports to YourBase Test Acceleration. +[Learn more about telemetry data here](../security.md#telemetry). + --- -### YOURBASE_TIMID +## YOURBASE_TIMID `Type`: `bool-ish (0, false, off, 1, true, on)` `Default`: `off` @@ -179,7 +186,7 @@ We recommend reaching out to support@yourbase.io if you are encountering this sc --- -### YOURBASE_WORKDIR +## YOURBASE_WORKDIR `Type`: `absolute or relative path` `Default`: `.` diff --git a/docs/test acceleration/releases.md b/docs/test acceleration/releases.md index 76eceaf..0c0a9f2 100644 --- a/docs/test acceleration/releases.md +++ b/docs/test acceleration/releases.md @@ -7,6 +7,6 @@ has_children: false permalink: /test-acceleration/releases --- -#Releases +# Release history -See YourBase Test Acceleration release history on PyPI: https://pypi.org/project/yourbase/#history \ No newline at end of file +See YourBase Test Acceleration [release history on its PyPI page](https://pypi.org/project/yourbase/#history). \ No newline at end of file diff --git a/docs/test acceleration/security.md b/docs/test acceleration/security.md index abb0f93..f3d4100 100644 --- a/docs/test acceleration/security.md +++ b/docs/test acceleration/security.md @@ -8,10 +8,14 @@ permalink: /test-acceleration/security --- # Security -YourBase Test Acceleration library is secure. Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. Only metadata about your usage of the library would ever be shared with YourBase Test Acceleration. +YourBase Test Acceleration library is secure. -By default, YourBase Test Acceleration collects anonymized data about how the YourBase Test Acceleration library is behaving to help resolve technical issues and develop the product with people's use--cases in mind. The information we collect via telemetry is documented at https://yourbase.io/data-usage. +## Code +Under no circumstance do your code or your dependency graphs ever touch YourBase Test Acceleration servers. -If YourBase Test Acceleration changes the type of data that it collects, it will always inform you at runtime. +## Telemetry +By default, YourBase Test Acceleration collects anonymized data about how the YourBase Test Acceleration library is behaving to help resolve technical issues and develop the product with people's use-cases in mind. The [data collected using telemetry is documented here](https://yourbase.io/data-usage). + +You can opt out of sending telemetry data to the YourBase Test Acceleration team, by setting the environment variable [YOURBASE_TELEMETRY](reference/configuration-options.md#yourbase_telemetry) to `false`. -You can opt-out of sending any crash data or usage statistics to the YourBase Test Acceleration team, by setting the environment variable YOURBASE_TELEMETRY to false. +If YourBase Test Acceleration changes the type of data that it collects, it will always inform you at runtime. From 4546006a6a4c33a8f46302e48f1b3860a428cef8 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 17:57:25 +0530 Subject: [PATCH 19/24] removed cli doc --- docs/cli/index.md | 3 ++- docs/index.md | 9 --------- docs/test acceleration/index.md | 2 +- 3 files changed, 3 insertions(+), 11 deletions(-) delete mode 100644 docs/index.md diff --git a/docs/cli/index.md b/docs/cli/index.md index e75dda4..8baf54f 100644 --- a/docs/cli/index.md +++ b/docs/cli/index.md @@ -1,7 +1,8 @@ --- layout: default title: CI and CLI -nav_order: 3 +nav_exclude: true +nav_order: 2 has_children: true permalink: /ci-cli --- diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index f855964..0000000 --- a/docs/index.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -layout: default -title: Home -nav_order: 1 -permalink: / ---- - -# YourBase -Some text talking about test acceleration & CI CLI \ No newline at end of file diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md index 62859c0..15a4cf8 100644 --- a/docs/test acceleration/index.md +++ b/docs/test acceleration/index.md @@ -1,7 +1,7 @@ --- layout: default title: Test acceleration -nav_order: 2 +nav_order: 1 has_children: true has_toc: false permalink: /test-acceleration From d2f1fee28e955f99cbe744104b90701689e557a0 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 19:50:10 +0530 Subject: [PATCH 20/24] Changes to rollout recos --- docs/test acceleration/deploying.md | 72 ++++++++++++++++++++--------- 1 file changed, 50 insertions(+), 22 deletions(-) diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying.md index b9886aa..30bff6e 100644 --- a/docs/test acceleration/deploying.md +++ b/docs/test acceleration/deploying.md @@ -21,7 +21,7 @@ This section contains guides to deploy the YourBase Test Acceleration library. ## Rollout recommendation -To safely accelerate tests on production, we recommend that you first run YourBase Test Acceleration in its Observation Mode [link to Advanced usage → Observation mode], and manually verify the output logged [link to Advanced usage → Observation mode → Checking output]. +To safely accelerate tests on production, we recommend that you first run YourBase Test Acceleration in its [Observation Mode](advanced-usage/verify-results.md), and manually [verify the output logged](advanced-usage/verify-results.md#verification-steps). We propose the rollout phases to look like the following: @@ -30,41 +30,69 @@ We propose the rollout phases to look like the following: This phase will synchronize your code’s dependency graph on your local machine storage. Hence, after completing this phase, you’ll see the tests on your branch running faster only on your local machine. -1. Install YourBase Test Acceleration locally [link to Installation] -2. Run YourBase Test Acceleration in Observation Mode [link to Advanced usage → Observation mode] -3. Ensure that YourBase Test Acceleration accelerates your tests correctly [link to Advanced usage → Observation mode → Checking output], or resolve any issues that arise -4. Disable Observation Mode [link to Reference → Configuration → Observation mode] -5. Run YourBase Test Acceleration +1. [Install YourBase Test Acceleration](install.md) locally. +2. Run YourBase Test Acceleration in [Observation Mode](advanced-usage/verify-results.md). +3. Ensure that YourBase Test Acceleration [accelerates your tests correctly](advanced-usage/verify-results.md#verification-steps), or resolve any issues that arise. +4. [Disable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). +5. Run your tests. -### Phase 2: Test in CI +### Phase 2: Run in CI {: .no_toc } This phase will synchronize your code’s dependency graph on remote storage for use by CI. Hence, after completing this phase, you’ll see the tests on your branch running faster on your CI as well. +#### Step 1. Configure Remote Cache +{: .no_toc } + In your CI environment: + 1. Set up a [Shared Dependency Graph for use in CI](advanced-usage/accelerate-tests-in-ci.md). + + - Tip: it may be easier to set up and debug the remote cache from your local environment before configuring it in the CI + + 2. Set up the following configuration variables for your CI environment: + - [YOURBASE_LICENSE_KEY](reference/configuration-options.md#yourbase_license_key) + - [YOURBASE_ACCEPT_TOS](reference/configuration-options.md#yourbase_accept_tos) + +#### Step 2. Install in test branch +{: .no_toc } + +In your test branch, do the following: + 1. Install YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. + 2. [Enable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). + 3. Run your tests as usual. + 4. Ensure that [YourBase Test Acceleration accelerates these tests correctly](advanced-usage/verify-results.md#verification-steps), or resolve any issues that arise. + 5. [Disable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). + 6. Run your tests as usual. + + +#### Step 3: Install in main branch +{: .no_toc } + +In your main branch, before enabling Yourbase Test Acceleration for full production, we recommend executing the following steps for a subset of builds, for example in a canary environment or as a percentage experiment: + + 1. Install YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. + 2. Set `YOURBASE_DISABLE=true` and ensure CI continues to run as expected. + 3. [Enable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). + 4. Set `YOURBASE_DISABLE=false`. + 5. Run your tests as usual. + 6. Ensure that [YourBase Test Acceleration accelerates these tests correctly](advanced-usage/verify-results.md#verification-steps), or resolve any issues that arise. + 7. [Disable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). + 8. Run your tests as usual. + + +_Note: Once YourBase Test Acceleration is launched to production, we recommend continuing to run the full test suite occasionally, for example, in advance of major releases._ -1. Set up the Shared Dependency Graph for use by your CI [link to Advanced usage → Accelerate Using Shared Dependency Graph] -2. Install YourBase Test Acceleration to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your CI environment -3. Enable Observation Mode [link to Advanced usage → Observation mode] -4. Set up the following configuration variables for your CI environment: - - YOURBASE_LICENSE_KEY [link to configuration options section] - - YOURBASE_ACCEPT_TOS [link to configuration options section] -5. Run YourBase Test Acceleration for a subset of tests -6. Ensure that YourBase Test Acceleration accelerates these tests correctly [link to Advanced usage → Observation mode → Checking output], or resolve any issues that arise -7. Roll out to the remainder of tests in increments, resolving any potential issues that arise -8. Disable Observation Mode [link to Reference → Configuration → Observation mode] -9. Run YourBase Test Acceleration for all your tests in CI ### Phase 3: Enable local test acceleration across your development team, starting with a group of beta users. {: .no_toc } After this phase, you’ll see tests run faster for developers across the team because of them sharing their dependency graph. -1. Set up the Shared Dependency Graph for use by your local machine [link to Advanced usage → Accelerate Using Shared Dependency Graph] -2. Install YourBase Test Acceleration to your project via requirements.txt or whatever other mechanism you use to install your dependencies in your local environment +1. Set up the [Shared Dependency Graph for use by your local machine](advanced-usage/accelerate-tests-across-developers.md). +2. Install YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your local environment 3. Set up the following configuration variables for your local environment: - - YOURBASE_LICENSE_KEY [link to configuration options section] - - YOURBASE_ACCEPT_TOS` [link to configuration options section] + - [YOURBASE_LICENSE_KEY](reference/configuration-options.md#yourbase_license_key) + - [YOURBASE_ACCEPT_TOS](reference/configuration-options.md#yourbase_accept_tos) --- From 2748efee4f7754d4ce0065daaef65d9542b8456a Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 21:20:02 +0530 Subject: [PATCH 21/24] Fixed deploying pages --- docs/test acceleration/deploying/disable.md | 17 +++++ docs/test acceleration/deploying/index.md | 12 ++++ docs/test acceleration/deploying/logging.md | 13 ++++ .../phased-rollout.md} | 70 +++++++------------ 4 files changed, 67 insertions(+), 45 deletions(-) create mode 100644 docs/test acceleration/deploying/disable.md create mode 100644 docs/test acceleration/deploying/index.md create mode 100644 docs/test acceleration/deploying/logging.md rename docs/test acceleration/{deploying.md => deploying/phased-rollout.md} (66%) diff --git a/docs/test acceleration/deploying/disable.md b/docs/test acceleration/deploying/disable.md new file mode 100644 index 0000000..9e8d70c --- /dev/null +++ b/docs/test acceleration/deploying/disable.md @@ -0,0 +1,17 @@ +--- +layout: default +title: Disable +nav_order: 3 +parent: Deploying +grand_parent: Test acceleration +permalink: /test-acceleration/disable +--- + +# Disable YourBase Test Acceleration +If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable [YOURBASE_DISABLE](../reference/configuration-options.md#yourbase_disable) to true: + +```sh +export YOURBASE_DISABLE=true +``` + +To uninstall the package, [see the uninstall instructions](install.md#uninstall). \ No newline at end of file diff --git a/docs/test acceleration/deploying/index.md b/docs/test acceleration/deploying/index.md new file mode 100644 index 0000000..2aba57d --- /dev/null +++ b/docs/test acceleration/deploying/index.md @@ -0,0 +1,12 @@ +--- +layout: default +title: Deploying +nav_order: 4 +parent: Test acceleration +has_children: true +permalink: /test-acceleration/deploying +--- + +# Deploying +This section contains guides to smoothly deploy the YourBase Test Acceleration library to your production. + diff --git a/docs/test acceleration/deploying/logging.md b/docs/test acceleration/deploying/logging.md new file mode 100644 index 0000000..c0b335a --- /dev/null +++ b/docs/test acceleration/deploying/logging.md @@ -0,0 +1,13 @@ +--- +layout: default +title: Logging +nav_order: 2 +parent: Deploying +grand_parent: Test acceleration +permalink: /test-acceleration/logging +--- + +# Logging +All logs are prefixed with `[YB]`. + +By default, minimal logs are printed. To obtain more detailed debugging information, [set the YOURBASE_DEBUG environment variable](../reference/configuration-options.md#yourbase_debug). diff --git a/docs/test acceleration/deploying.md b/docs/test acceleration/deploying/phased-rollout.md similarity index 66% rename from docs/test acceleration/deploying.md rename to docs/test acceleration/deploying/phased-rollout.md index 30bff6e..e2468b8 100644 --- a/docs/test acceleration/deploying.md +++ b/docs/test acceleration/deploying/phased-rollout.md @@ -1,59 +1,57 @@ --- layout: default -title: Deploying -nav_order: 4 -parent: Test acceleration -permalink: /test-acceleration/deploying +title: Phased rollout +nav_order: 1 +parent: Deploying +grand_parent: Test acceleration +permalink: /test-acceleration/rollout --- -# Deploying -{: .no_toc } - -This section contains guides to deploy the YourBase Test Acceleration library. +# Rollout recommendation +{:.no_toc}
Table of contents {: .text-delta } -- TOC +1. TOC {:toc}
-## Rollout recommendation To safely accelerate tests on production, we recommend that you first run YourBase Test Acceleration in its [Observation Mode](advanced-usage/verify-results.md), and manually [verify the output logged](advanced-usage/verify-results.md#verification-steps). -We propose the rollout phases to look like the following: - -### Phase 1: Test locally: On your development branch -{: .no_toc } +We propose the rollout to be done in the following 3 phases: +## Phase 1: Test locally This phase will synchronize your code’s dependency graph on your local machine storage. Hence, after completing this phase, you’ll see the tests on your branch running faster only on your local machine. -1. [Install YourBase Test Acceleration](install.md) locally. +In your development branch: + +1. [Install YourBase Test Acceleration](install.md) locally. 2. Run YourBase Test Acceleration in [Observation Mode](advanced-usage/verify-results.md). 3. Ensure that YourBase Test Acceleration [accelerates your tests correctly](advanced-usage/verify-results.md#verification-steps), or resolve any issues that arise. 4. [Disable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). 5. Run your tests. -### Phase 2: Run in CI -{: .no_toc } +--- +## Phase 2: Test in CI This phase will synchronize your code’s dependency graph on remote storage for use by CI. Hence, after completing this phase, you’ll see the tests on your branch running faster on your CI as well. -#### Step 1. Configure Remote Cache +### Step 1. Configure Remote Cache {: .no_toc } In your CI environment: 1. Set up a [Shared Dependency Graph for use in CI](advanced-usage/accelerate-tests-in-ci.md). - - Tip: it may be easier to set up and debug the remote cache from your local environment before configuring it in the CI + - Tip: It may be easier to set up and debug the remote cache from your local environment before configuring it in the CI. 2. Set up the following configuration variables for your CI environment: - [YOURBASE_LICENSE_KEY](reference/configuration-options.md#yourbase_license_key) - [YOURBASE_ACCEPT_TOS](reference/configuration-options.md#yourbase_accept_tos) -#### Step 2. Install in test branch +### Step 2. Install in test branch {: .no_toc } In your test branch, do the following: @@ -65,12 +63,12 @@ In your test branch, do the following: 6. Run your tests as usual. -#### Step 3: Install in main branch +### Step 3: Install in main branch {: .no_toc } In your main branch, before enabling Yourbase Test Acceleration for full production, we recommend executing the following steps for a subset of builds, for example in a canary environment or as a percentage experiment: - 1. Install YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. + 1. [Install YourBase Test Acceleration](../install.md) to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your CI environment. 2. Set `YOURBASE_DISABLE=true` and ensure CI continues to run as expected. 3. [Enable Observation Mode](reference/configuration-options.md#yourbase_observation_mode). 4. Set `YOURBASE_DISABLE=false`. @@ -82,32 +80,14 @@ In your main branch, before enabling Yourbase Test Acceleration for full product _Note: Once YourBase Test Acceleration is launched to production, we recommend continuing to run the full test suite occasionally, for example, in advance of major releases._ +--- -### Phase 3: Enable local test acceleration across your development team, starting with a group of beta users. -{: .no_toc } - +## Phase 3: Enable local test acceleration across your development team After this phase, you’ll see tests run faster for developers across the team because of them sharing their dependency graph. -1. Set up the [Shared Dependency Graph for use by your local machine](advanced-usage/accelerate-tests-across-developers.md). -2. Install YourBase Test Acceleration to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your local environment +Do the following starting with a group of beta users, slowly rolling out to the entire team: +1. Set up the [shared Dependency Graph for use by your local machine](advanced-usage/accelerate-tests-across-developers.md). +2. [Install YourBase Test Acceleration](../install.md) to your project via `requirements.txt` or whatever other mechanism you use to install your dependencies in your local environment 3. Set up the following configuration variables for your local environment: - [YOURBASE_LICENSE_KEY](reference/configuration-options.md#yourbase_license_key) - [YOURBASE_ACCEPT_TOS](reference/configuration-options.md#yourbase_accept_tos) - ---- - -## Logging -We prefix all logs with `[YB]`. - -By default, minimal logs are printed. To obtain more detailed debugging information, [set the YOURBASE_DEBUG environment variable](reference/configuration-options.md#yourbase_debug). - ---- - -## Disable YourBase Test Acceleration -If for any reason, you need to disable YourBase Test Acceleration, you can simply set the environment variable [YOURBASE_DISABLE](reference/configuration-options.md#yourbase_disable) to true: - -```sh -export YOURBASE_DISABLE=true -``` - -To uninstall the package, [see the uninstall instructions](install.md#uninstall). \ No newline at end of file From d87b46398e84b0b9b3bfad65c87a26dcde732016 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 23:06:43 +0530 Subject: [PATCH 22/24] Added url changes for hosting on GH pages on priyanshu's account --- docs/_config.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_config.yml b/docs/_config.yml index 15d84cd..9ca673a 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -25,8 +25,8 @@ email: hi@yourbase.io # line in _config.yml. It will appear in your document head meta (for # Google search results) and in your feed.xml site description. baseurl: "" # the subpath of your site, e.g. /blog -url: https://docs.yourbase.io -domain: docs.yourbase.io +url: https://prichhazed.github.io/yb +domain: prichhazed.github.io twitter_username: YourBaseIO github_username: yourbase matomo_url: https://web-analytics.yourbase.io/ From 7d6f45dd41345fb887806370815b271b684e6a6c Mon Sep 17 00:00:00 2001 From: prichhazed Date: Mon, 23 Aug 2021 23:30:22 +0530 Subject: [PATCH 23/24] Added index.md in docs folder --- docs/index.md | 9 +++++++++ docs/test acceleration/index.md | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..416519d --- /dev/null +++ b/docs/index.md @@ -0,0 +1,9 @@ +--- +layout: default +title: Home +nav_order: 1 +permalink: / +--- + +# YourBase +Some text talking about test acceleration & links to CI CLI. \ No newline at end of file diff --git a/docs/test acceleration/index.md b/docs/test acceleration/index.md index 15a4cf8..62859c0 100644 --- a/docs/test acceleration/index.md +++ b/docs/test acceleration/index.md @@ -1,7 +1,7 @@ --- layout: default title: Test acceleration -nav_order: 1 +nav_order: 2 has_children: true has_toc: false permalink: /test-acceleration From 355fdcfb8c263753eb42678d530ff489abdb8c50 Mon Sep 17 00:00:00 2001 From: prichhazed Date: Tue, 24 Aug 2021 16:45:15 +0530 Subject: [PATCH 24/24] Fixed nav order --- docs/test acceleration/advanced-usage/index.md | 2 +- docs/test acceleration/deploying/index.md | 2 +- docs/test acceleration/getting-started/index.md | 2 +- docs/test acceleration/getting-started/pytest.md | 2 +- docs/test acceleration/how-it-works.md | 2 +- docs/test acceleration/reference/index.md | 2 +- docs/test acceleration/releases.md | 2 +- docs/test acceleration/security.md | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/test acceleration/advanced-usage/index.md b/docs/test acceleration/advanced-usage/index.md index c8529eb..27116df 100644 --- a/docs/test acceleration/advanced-usage/index.md +++ b/docs/test acceleration/advanced-usage/index.md @@ -1,7 +1,7 @@ --- layout: default title: Advanced usage -nav_order: 5 +nav_order: 6 parent: Test acceleration has_children: true permalink: /test-acceleration/advanced-usage diff --git a/docs/test acceleration/deploying/index.md b/docs/test acceleration/deploying/index.md index 2aba57d..75c1372 100644 --- a/docs/test acceleration/deploying/index.md +++ b/docs/test acceleration/deploying/index.md @@ -1,7 +1,7 @@ --- layout: default title: Deploying -nav_order: 4 +nav_order: 5 parent: Test acceleration has_children: true permalink: /test-acceleration/deploying diff --git a/docs/test acceleration/getting-started/index.md b/docs/test acceleration/getting-started/index.md index 7d53699..21ed94a 100644 --- a/docs/test acceleration/getting-started/index.md +++ b/docs/test acceleration/getting-started/index.md @@ -1,7 +1,7 @@ --- layout: default title: Getting started -nav_order: 2 +nav_order: 3 parent: Test acceleration has_children: true permalink: /test-acceleration/getting-started diff --git a/docs/test acceleration/getting-started/pytest.md b/docs/test acceleration/getting-started/pytest.md index e258b08..86666cf 100644 --- a/docs/test acceleration/getting-started/pytest.md +++ b/docs/test acceleration/getting-started/pytest.md @@ -117,7 +117,7 @@ Look for the logs traced by YourBase Test Acceleration. You’ll see a message t [YB] Function-level dependency graph found 1 test affected ``` -You can see form the logs that the existing dependency graph [link to How it works → Dependency graph section] was used to decide that only one test was affected by your code-change and that test was the only one that was run, while the remaining tests were skipped. +You can see form the logs that the existing [dependency graph](../how-it-works.md#dependency-graph) was used to decide that only one test was affected by your code-change and that test was the only one that was run, while the remaining tests were skipped. --- diff --git a/docs/test acceleration/how-it-works.md b/docs/test acceleration/how-it-works.md index dd6d261..32be032 100644 --- a/docs/test acceleration/how-it-works.md +++ b/docs/test acceleration/how-it-works.md @@ -1,7 +1,7 @@ --- layout: default title: How it works -nav_order: 3 +nav_order: 4 parent: Test acceleration permalink: /test-acceleration/how-it-works --- diff --git a/docs/test acceleration/reference/index.md b/docs/test acceleration/reference/index.md index 856cff2..61e55fe 100644 --- a/docs/test acceleration/reference/index.md +++ b/docs/test acceleration/reference/index.md @@ -1,7 +1,7 @@ --- layout: default title: Reference -nav_order: 6 +nav_order: 7 parent: Test acceleration has_children: true permalink: /test-acceleration/reference diff --git a/docs/test acceleration/releases.md b/docs/test acceleration/releases.md index 0c0a9f2..a5957ef 100644 --- a/docs/test acceleration/releases.md +++ b/docs/test acceleration/releases.md @@ -1,7 +1,7 @@ --- layout: default title: Releases -nav_order: 8 +nav_order: 9 parent: Test acceleration has_children: false permalink: /test-acceleration/releases diff --git a/docs/test acceleration/security.md b/docs/test acceleration/security.md index f3d4100..b129e8d 100644 --- a/docs/test acceleration/security.md +++ b/docs/test acceleration/security.md @@ -1,7 +1,7 @@ --- layout: default title: Security -nav_order: 7 +nav_order: 8 parent: Test acceleration has_children: false permalink: /test-acceleration/security