Create default trees automatically during setup

[alesan99]

Is your feature request related to a problem? Please describe.
With #6429, there is now a way to create default populated trees. They should be created automatically with the setup tool.

Notes:

• There is code to create root notes in at least 3 different places.
• Chronostrat trees should have FullNameDirection = -1

Describe the solution you'd like
The setup tool should automatically create the Geography, Chronostrat (If applicable), and the appropriate taxon tree.

[alesan99]

Insight from @melton-jason

(optional, performance)

Currently, all database tasks are happening serially. While this is required when creating the main Hierarchy tables (Institution, Division, Discipline, Collection), there are plenty of interim steps that can happen in parallel.

Below is what I think a basic dependency chain looks like for specify setup:

flowchart TD
    A[Institution] --> B(Storage Tree) & C(Division) & D(Geography Tree) & E(Geologic Time Period Tree)
    C & D & E --> F(Discipline)
    F --> G(Taxon Tree)
    F --> H(Lithostrat Tree)
    F --> I(Tectonic Unit Tree)
    F --> L(Collection)
    L --> M(Prep Types)
    L --> N(PickLists)
    L --> O(COTs)
    F --> J(Schema Config)
    F --> K(Uniqueness Rules)
    C --> P(Agent)
    P --> Q(SpecifyUser)

    classDef one stroke:#FF0000
    classDef two stroke:#FFA500
    classDef three stroke:#FFFF00
    classDef four stroke:#00FF00
    classDef five stroke:#0000FF
    class A one;
    class B,C,D,E two;
    class F,P three;
    class G,H,I,L,J,K,Q four;
    class M,N,O five;

Loading

Very loosely, nodes of the same color can be created at the same time. In terms of implementation, this means each node can be a separate task. Once the Institution task is completed, we can queue the tasks for Division, and the Storage, Geography and Chronostrat trees. Once the Division, Chronostrat tree, and Geography tree tasks are completed, we can technically queue the Discipline creation task. Once the Discipline task completes we can queue the tasks for Collection, Schema Config, Uniqueness Rules, and other trees, etc.

Although this would assume our worker can handle concurrent tasks in the first place so it can leverage the parallel tasks: which is not the case in our default setup.

Our current Celery default is to only allow a single task at any given time, so the current implementation is pretty much identical to the above suggestion anyways 🤷.

@specify/developers I would love discussing increasing the default concurrency maximum we provide with the team!

Celery itself actually defaults to the number of available cores: https://docs.celeryq.dev/en/stable/userguide/workers.html#concurrency, but Specify explicitly limits the maximum number of processes in our defaults to 1.

Increasing the concurrency limit would enable the worker to execute more than one task at a given time: meaning more than one Backup Task, Merge, and WorkBench operation can happen at the same time (of course, we'd still be somewhat limited by database locks).

We might actually consider setting something like autoscaling, which dynamically increases or decreases the concurrency limit based on current load to some specified maximum and minimum: https://docs.celeryq.dev/en/stable/userguide/workers.html#autoscaling.

At the very least, I think we should consider increasing the concurrency maximum for database setup, if that's something we want to speed up. A lot of the work can be performed concurrently!