Detection: Being able to set the recorded_at field when recording a detection.

[fe51]

Motivation

Currently, the /api/v1/detections/ POST endpoint allow to create detections but the creation date (created_at) is imposed (= now)

While this may have been satisfactory for a while, detections can sometimes be cached on the engine side, meaning that when they are posted on the API, they get published almost all at the same datetime, even though each detection occured at different times.

Bonus point : As of today in the envdenv, this is not straight forward to create alerts in the past (for demonstration purpose), it will be easier once this issue is solved !

What should be done

• Adds recorded_at field to detections table.

• Updates endpoints of the api adding recorded_at in :

  • /api/v1/detections/ POST (in order to be able to set date as recorded_at as parameter )
  • /api/v1/detections/{detection_id} GET
  • /api/v1/detections/ GET
  • /api/v1/sequences/{sequence_id}/detections GET

• Updates how detections are linked to sequences : Currently based on the created_at field, must be related to the recorded_at field

  • mainly in pyro-api/src/app/api/api_v1/endpoints/detections.py
  • updates creation rule and sequence creation/update of started_eat and last_seen_at fields

• Add migration script (in migrations directory).

  • add upgrade (create created_at field if does not exists, and copy data as described below) and downgrade function (drop field) -> you can check other migration scripts !
  • Migrations rules : copy data from created_at field if recorded_at is empty, probably something like

UPDATE detections
SET recorded_at = created_at
WHERE recorded_at IS NULL;

• do not forget to add tests ;)

Happy to discuss it :)

[frgfm]

I agree and thought about this regularly. I'll share my thoughts:

1. The reason I named the column "created_at" in the first place is because it leaves room for "taken_at", only the creation date of the database entry, which is the information we can be adamant about
2. I think the real solution here would be to make sure we created picture metadata whenever we create one on the device. Once uploaded, we can retrieve that information in the backend.
3. passing the information in the payload can be good with I think it comes with lots of data inaccuracies and constraints on TZ management 😅

On several projects I've worked with in the past, we used the picture metadata from smartphone cameras to determine the timestamp accurately

[fe51]

Ok so what you suggest is to keep the created_at field as is it and create a new field “taken_at” that could be filled from, according which strategy we decide to implement:

• Either from the payload but with issues mention above (and I caan agree with)
• Or extract the metadata from the picture
  • It is a good idea thank you for suggesting it ! As long I have never done it, i have a few questions
  • are we sure will have these metadata in all situations with different caméras type/way to access data ( and not necessarily our hardware) ?
  • Are we sure to avoid tz constrains ?
  • Also, would it be as fast/fast enough to get in the db as a payload input ?

Still, if the idea discussed above is implemented, do we will still have a use of the field created_at ? Otherwise why we should keep it ?

[frgfm]

Still, if the idea discussed above is implemented, do we will still have a use of the field created_at ? Otherwise why we should keep it ?

imo yes. We can always trust the timestamp of when the payload was received by our backend API, but taken_at can be null in some case or less accurate. And the diff between the two would be quite helpful to spot cameras with major connection issues 😅

[MateoLostanlen]

Sorry for the late reply!

Yes, my idea was to keep both fields? one created_at that doesn’t change, and one recorded_at.

For recorded_at, the engine can handle that easily: as soon as it captures an image, it saves the timestamp in UTC zero, and we’re all good