Distributed Task Scheduler (Node.js + MongoDB)

A lightweight, persistent task orchestration engine built with:

• Node.js
• Express
• MongoDB
• Mongoose

It supports:

• Submitting tasks via HTTP API
• Dependency-aware execution (DAG-style)
• Concurrency limits
• Persistent state & crash recovery

---

1. Running the Server

Prerequisites

• Node.js (>= 18 recommended)
• MongoDB running locally and DB named "taskarch"

Install Dependencies

npm install

Running Server

node src/app.js

Concurrency Model

The concurrency limit is enforced using a simple counter: runningCount < MAX_CONCURRENT_TASKS.

Each iteration calculates how many workers are currently free (availableWorkers) and dispatches that many tasks immediately.

Tasks are picked from a shared in-memory buffer (availableTasks). This avoids repeatedly querying the database for every new task.

Actual execution happens asynchronously via runTask(task) while the loop continues polling — enabling non-blocking processing.

short sleep(100) creates a lightweight scheduling cycle, allowing the system to check for new tasks frequently without overwhelming resources.