Device (ESP32-S3) Documentation

This document provides instructions for setting up and using the firmware for the ESP32-S3 device component of this AIoT project.

✨ Features

• AI Model Deployment: Remotely deploy new AI models to the device.
• AI Model Switching: Switch between multiple AI models stored on the device without reflashing.
• Over-the-Air (OTA) Updates: Update the device's firmware remotely.

🛠️ Setup

1. ESP-IDF Environment

This project is built using esp-idf, the official development framework for Espressif's SoCs. You must have it installed and configured first.

• Setup Guide: ESP-IDF Extension for VSCode

2. Web Service

Before running the device, ensure the web service (backend/frontend) is running and you have created a user account. The device will need these credentials to connect, the server repo is here.

3. Menuconfig Configuration

You must configure the project using idf.py menuconfig. Set the following parameters:

General Device Config

• Serial Flash Configurations -> Flash size
• Serial Flash Configurations -> Enable Octal Flash
• Partition Table -> Custom partition table CSV
• Component config -> ESP PSRAM -> Support for external, SPI-connected RAM
• Component config -> SPI RAM config -> Octal Mode PSRAM
• Component config -> SPI RAM config -> 80MHz clock speed
• ESP System Settings -> CPU frequency -> 240MHZ
• Component config -> FAT Filesystem support -> Long filename support -> Long filename buffer on stack
• ESP HTTPS OTA -> Allow HTTP for OTA (WARNING: ONLY FOR TESTING PURPOSE, READ HELP)
• Compiler options -> optimization for performance
• Component config -> Power Management -> Support for power management
• Component config -> Power Management -> Enable dynamic frequency scaling (DFS) at startup

Server Config

This section is for connecting the device to your backend server.

• SERVER Configuration -> Server Host: The host IP address or domain name of your backend server. For example: 192.168.0.155:8000.
• Auth Provisioning -> Username: The username of the account on the web service that you want this device to be paired with.
• Auth Provisioning -> Password: The password for the corresponding username.

Wifi Config

• WiFi -> SSID
• WiFi -> PASSWORD

⚠️ Limitations

• Hardware Support: This project currently only supports the ESP32-S3 series. The AI inference functionality relies on the esp-idf/dl component, and it has not been tested on other series like the ESP32-P4.
• Model Support: Only Object Detection models are supported at the moment. Image Classification models are not yet implemented.
• Custom Models: The detection postprocessor parameters are currently hardcoded for the default model. If you wish to use your own custom-trained model, you will need to modify these parameters. You can find them in: main/model/model_factory.cpp

🔌 Hardware

This project was developed and tested using the Freenove ESP32-S3 CAM board.

• Product Link: Freenove ESP32-S3 CAM
• Product Link: Freenove ESP32-S3 CAM for Shopee

If you intend to use a different ESP32-S3 development board, pay close attention to the camera and SD card pin configurations. You will need to modify the corresponding pin definitions in main.cpp to match your hardware.