install_dev

• install from sources
  • use the devcontainer
  • install manually
    • 1. install OS dependencies
    • 2. clone repositories
    • 3. create and enter virtualenv
    • 4. prepare directories and configuration
    • 5. install gulp
    • 6. run
    • 7. optional: installing extra plugins
  • install using the setup script
  • install the client
  • troubleshoot
  • extras
    • coding style tips

install from sources

use the devcontainer

this is the recommended dev install method!

gulp have its own devcontainer setup ready to use with i.e. vscode.

1. install the devcontainers extension if you do not have already.

2. clone the repositories

mkdir ./repos && cd ./repos
git clone --recurse-submodules https://github.com/mentat-is/gulp.git
# clone other repos, i.e. the devteam may want to clone also the paid-plugins repo...
git clone https://github.com/mentat-is/gulp-paid-plugins
# i.e. this is needed to develop gulp clients or just run the tests
git clone https://github.com/mentat-is/gulp-sdk-python
# to develop slurp
git clone https://github.com/mentat-is/slurp

3. in vscode, open the gulp workspace and it should prompt you to reopen it in the dev container: the devcontainer will be built first time only.

4. on the host, start the main docker-compose to start postgresql and opensearch.

docker compose --profile dev up -d

5. go back in the container and develop as normal.

you may also want to update/edit extensions installed in the container... just do it from vscode in the extensions -> dev container: gulp dev container tab.

install manually

use this if you do not want to use the devcontainer ...

1. install OS dependencies

This depends on your OS, on EndeavourOS(arch):

# tested with python 3.12, *may* work with 3.13....
sudo pacman -S rust python=3.12.7-1 python-virtualenv docker docker-compose docker-buildx jq libpqxx git-lfs

2. clone repositories

mkdir ./repos && cd ./repos
git clone --recurse-submodules https://github.com/mentat-is/gulp.git

3. create and enter virtualenv

cd ./gulp
# also ensure to start with a clean .venv
rm -rf ./.venv
virtualenv --python=/usr/bin/python3.12 ./.venv
source ./.venv/bin/activate

4. prepare directories and configuration

# create configuration directory (ensure its empty)
rm -rf ~/.config/gulp
mkdir -p ~/.config/gulp

# copy template configuration, edit it in case (pay attention to the debug options!)
cp ./gulp_cfg_template.json ~/.config/gulp_cfg.json

5. install gulp

install all packages as editable

# install all packages as editable (-e)
pip3 install -e . && pip3 install -e ./muty-python

# i.e. if you need the api client (tests, develop bridges, ...)
pip3 install -e ../gulp-sdk-python

6. run

you may need to ensure proper docker cleanup first (i.e. previous installation) with reset_docker

# start postgresql and opensearch
# if you find any problem, remove -d and check docker logs (and check our troubleshooting guide)
docker compose --profile dev up -d

# run gulp first time (will create collab database "gulp" on postgresql and the default index/operation "test_operation" on opensearch)
GULP_BIND_TO_ADDR=0.0.0.0 GULP_BIND_TO_PORT=8080 gulp

7. optional: installing extra plugins

plugins are just files, so it is enough to copy/symlink extra plugins in $GULP_WORKING_DIR/plugins and $GULP_WORKING_DIR/mapping_files

install using the setup script

THIS IS CURRENTLY BROKEN AND NOT SUPPORTED

installation of a development environment can be done using the setup.sh script.

curl https://raw.githubusercontent.com/mentat-is/gulp/refs/heads/develop/setup.sh -o gulp_setup.sh
chmod +x gulp_setup.sh
sudo ./gulp_setup.sh --dev -d ./gulp

if your OS is not supported please refer to the manual installation instructions below.

install the client

follow the instructions to install the web client.

troubleshoot

troubleshoot

extras

coding style tips

• use blackformatter with default options

• use the following settings for pylint

  vscode settings.json

  {
      "pylint.args": [
      "--jobs=0",
      "--disable=useless-parent-delegation",
      "--disable=no-member",
      "--disable=no-name-in-module",
      "--disable=import-error",
      "--disable=logging-not-lazy",
      "--disable=consider-using-f-string",
      "--disable=line-too-long",
      "--disable=invalid-name",
      "--disable=global-statement",
      "--disable=broad-exception-raised",
      "--disable=broad-exception-caught",
      "--disable=missing-function-docstring",
      "--disable=broad-except",
      "--disable=no-name-in-module",
      "--disable=global-variable-not-assigned",
      "--disable=bare-except",
      "--disable=too-many-arguments",
      "--disable=unused-argument",
      "--disable=import-outside-toplevel",
      "--extension-pkg-whitelist=lxml",
      "--disable=fixme",
      "--disable=too-many-positional-arguments",
      "--disable=too-many-locals",
      "--disable=missing-class-docstring",
      "--disable=logging-fstring-interpolation",
      "--disable=wrong-import-position"
    ]
  }