Examples for running multiple workers

[IgnisDa]

Hi! I am trying to upgrade from apalis 0.7 to 1.0.0-rc.2.

I am not sure if an example for this was there earlier, but the main branch no longer has any example of using Monitor::new(). Instead, all examples show WorkerBuilder::new("...").....run().await. But this does not help me understand how I can run multiple workers.

I do have this working, but it looks wrong and finicky:

 let monitor = Monitor::new()
        .register({
            let ss = supporting_service.clone();
            let scheduler = infrequent_scheduler.clone();
            move |_runs| {
                WorkerBuilder::new("infrequent_cron_jobs")
                    .backend(CronStream::new_with_timezone(scheduler.clone(), tz))
                    .enable_tracing()
                    .catch_panic()
                    .data(ss.clone())
                    .build(run_infrequent_cron_jobs)
            }
        })
        .register({
            let ss = supporting_service.clone();
            let scheduler = frequent_scheduler.clone();
            move |_runs| {
                WorkerBuilder::new("frequent_cron_jobs")
                    .backend(CronStream::new_with_timezone(scheduler.clone(), tz))
                    .enable_tracing()
                    .catch_panic()
                    .data(ss.clone())
                    .build(run_frequent_cron_jobs)
            }
        })
        .register({
            let storage = single_application_job_storage.clone();
            let ss = supporting_service.clone();
            move |_runs| {
                WorkerBuilder::new("perform_single_application_job")
                    .backend(storage.clone())
                    .catch_panic()
                    .enable_tracing()
                    .concurrency(1)
                    .data(ss.clone())
                    .build(perform_single_application_job)
            }
        })
        .register({
            let storage = hp_application_job_storage.clone();
            let ss = supporting_service.clone();
            move |_runs| {
                WorkerBuilder::new("perform_hp_application_job")
                    .backend(storage.clone())
                    .catch_panic()
                    .enable_tracing()
                    .data(ss.clone())
                    .build(perform_hp_application_job)
            }
        })
        .register({
            let storage = mp_application_job_storage.clone();
            let ss = supporting_service.clone();
            move |_runs| {
                WorkerBuilder::new("perform_mp_application_job")
                    .backend(storage.clone())
                    .catch_panic()
                    .enable_tracing()
                    .rate_limit(10, Duration::new(5, 0))
                    .data(ss.clone())
                    .build(perform_mp_application_job)
            }
        })
        .register({
            let storage = lp_application_job_storage.clone();
            let ss = supporting_service.clone();
            move |_runs| {
                WorkerBuilder::new("perform_lp_application_job")
                    .backend(storage.clone())
                    .catch_panic()
                    .enable_tracing()
                    .rate_limit(40, Duration::new(5, 0))
                    .data(ss.clone())
                    .build(perform_lp_application_job)
            }
        })
        .run();

I would really appreciate your help!

[geofmureithi]

Sure I will add more examples. Meanwhile you can look at this discussion:
https://github.com/orgs/apalis-dev/discussions/666

[IgnisDa]

Thanks for this. Just to be sure, Monitor is no longer in common usage, I take it? I like the worker builder approach anyway.

[geofmureithi]

I think you should use Monitor if:

1. You need a common point for event collection
2. You need to restart workers (and a common point to manage that)
3. You need a single point to Gracefully shutdown all workers.

All these features are possible with WorkerBuilder but will involve more code and decentralized management.

I would still use it in your case

[IgnisDa]

Makes sense. Thanks!