Control Synthesis on Dynamical Systems

Project Overview

Implementation of various system environments ranging from linear time-invariant to nonlinear control affine. Allows development of control methods using Control Barrier Functions (CBFs) combined with Quadratic Programming (QP) and also support for Reinforcement Learning (RL).

• 🎯 Safety-Critical Control: Implemented real-time safety filtering using Control Barrier Functions
• 🧠 Deep Reinforcement Learning: Integrated SAC and PPO algorithms with custom Gym environments
• 📊 Comprehensive Evaluation: Validated approaches on multiple dynamical systems

---

📁 Repository Structure

system experiments
├── linear_time_invariant/          # LTI systems with linearized dynamics
│   ├── CartPole.ipynb              # Classic inverted pendulum control
│   ├── Drone3D.ipynb               # Fully-actuated 3D drone system
|   ├── LaneChange.ipynb            # testing lateral control on car
├── nonlinear_control_affine/        # Nonlinear systems with control-affine form
│   ├── Drone2D.ipynb               # Planar quadrotor with thrust/torque
│   ├── ObstacleCar.ipynb           # Vehicle obstacle avoidance
│   ├── SafeACC.ipynb               # Adaptive cruise control system

---

🔧 Technical Implementation

Control Theory Foundation

• LQR Controller: Classical control baselines for comparison and nominal control input
• Control Barrier Functions (CBF): Mathematical framework for guaranteeing forward invariance of safe sets
• Control Lyapunov Functions (CLF): Stability guarantees through control Lyapunov functions
• Quadratic Programming (QP): Real-time safety filtering using optimization-based control

Machine Learning Integration

• Stable Baselines 3: SAC and PPO implementations for policy learning
• Custom Gym Environments: RL-ready environments for all dynamical systems
• Policy Optimization: Training and evaluation of deep RL agents
• Safety Constraints: Embedding formal safety guarantees in learning frameworks

Computational Framework

• Scientific Computing: NumPy and SciPy for numerical methods
• Optimization Solvers: CVXOPT for quadratic programming solutions
• Control Systems Library: Python-Control for system analysis
• Visualization Tools: Matplotlib for simulation and results presentation

---

System Implementations

1. 2D Drone (Nonlinear Control-Affine)

Planar Quadrotor System with thrust and torque inputs implementing:

• Nonlinear dynamics modeling with state-space representation
• CBF-QP safety filter for angle rate constraints
• CLF-QP stabilization with Lyapunov-based control
• SAC and PPO policy learning with custom Gym environment
• Physics-based animation and trajectory visualization
• Real-time simulation with obstacle avoidance

2. 3D Drone (Linear Time-Invariant)

Fully-Actuated Aerial Vehicle featuring:

• Position and attitude control in three-dimensional space
• Linearized dynamics with LQR optimal control
• CBF-QP collision avoidance with static obstacles
• Custom Gym environment for reinforcement learning
• PPO policy implementation with pre-trained models
• 3D trajectory planning and visualization capabilities

3. CartPole System (Linear Time-Invariant)

Classic Control Benchmark demonstrating:

• Inverted pendulum stabilization on moving cart
• Linearized system dynamics with state-space model
• LQR controller design for optimal performance
• CBF-QP safety constraints for state bounds
• SAC and PPO policy learning integration
• Real-time animation with physics simulation

4. Obstacle Avoidance Car (Nonlinear Control-Affine)

Vehicle Navigation System with:

• Nonlinear vehicle dynamics modeling
• Learned CBF from expert demonstrations using neural networks
• Neural Network-based CBF approximation and training
• Safety-critical control with real-time QP optimization
• SAC policy optimization for path planning
• Multi-obstacle environment simulation

5. Adaptive Cruise Control (Nonlinear Control-Affine)

Intelligent Vehicle Following implementing:

• Vehicle following dynamics with distance-based safety
• CBF-QP for collision avoidance in traffic scenarios
• CLF-QP for speed regulation and comfort optimization
• SAC and PPO policy learning for adaptive behavior
• Highway driving scenario simulation
• Emergency braking with safety constraints

---

📊 Technical Approach

Control Barrier Function Framework

ḣ(x) ≥ -γ h(x)  ⟹  Safety constraint ensuring forward invariance

CBFs ensure that if a system starts in a safe set (h(x) ≥ 0), it remains safe for all future times by enforcing the above condition through control inputs.

Safety-Critical Control QP

min ||u - u_nom||² + p*δ
s.t. Lfh(x) + Lgh(x)*u ≥ -γ*h(x) + δ
     δ ≥ 0

Where:

• u_nom: Nominal (desired) control input
• Lfh(x), Lgh(x): Lie derivatives of the safety function
• γ: Class-K function for constraint convergence
• δ: Slack variable for constraint feasibility

Control Lyapunov Functions (CLF)

For stability guarantees:

V̇(x) ≤ -α(V(x))  ⟹  Stability constraint ensuring convergence

Combined CLF-CBF QP formulation ensures both stability and safety simultaneously.

Reinforcement Learning Integration

• Environment: Custom Gym environments for all systems
• Algorithms: SAC (Soft Actor-Critic) and PPO (Proximal Policy Optimization)
• Observation Space: System states including positions, velocities, angles
• Action Space: Control inputs bounded by physical actuator limits
• Reward Function: Task-specific objectives with safety violation penalties

---

🎯 Skills Demonstrated

Control Systems Expertise

• Advanced Control Theory: CBFs, CLFs, LQR design
• Nonlinear Systems: Control-affine forms, Lie derivative computation

Machine Learning Proficiency

• Reinforcement Learning: SAC, PPO, DDPG algorithm implementation
• Deep Learning: Neural network architectures for control functions
• Gym Integration: Custom environment development and reward engineering

Software Engineering Capabilities

• Scientific Computing: NumPy, SciPy, Matplotlib for numerical computing
• Optimization: Convex optimization, QP solvers for real-time control