[Gesture Lock] Create Testing App

[RKBoss6]

This app adds a ML model to the Bangle.js that has a twist out gesture that locks the bangle. Give it a try here!
Current Data Contributors:

• @bobrippling
• @thyttan
• me :)

Edit by @thyttan: Removed the link pointing to the app since it will turn Bangles to illuminated bricks, not for eternity but for a while.

[thyttan]

Just tried it - but it just made my Bangle lock up! It was even hard to trigger the watchdog.

No debugging info right now. Could try gathering if you want.

[RKBoss6]

@thyttan Oh, wow, I'm so sorry about that! I hope your bangle is back to normal! It could be because it's flashing model.lite onto the watch directly as .tfmodel. The other gesture app has it uploaded as a .js file which is then decompressed, but according to the thread I opened, @gfwilliams said it should be fine... Any clue as to how I should proceed?

No debugging info right now. Could try gathering if you want.

@thyttan If you feel comfortable, maybe try gathering some info, but I would completely understand if you don't want to!

[thyttan]

I'll see if there's some output to the web ide 👍

And no worries, I got the watch back to normal. I managed to reboot and uninstall via app loader without too much trouble.

[RKBoss6]

@thyttan Glad to hear! Thanks!

[thyttan]

Failed when trying to install now with these errors:

OUT OF MEMORY    at :1:41

...age').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABA...

                              ^

Uncaught SyntaxError: Got UNFINISHED STRING expected EOF

    at REPL (:1:42)

...ge').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABAA...

                              ^

Execution Interrupted

New interpreter error: LOW_MEMORY,MEMORY

Uncaught ReferenceError: "NHy9zY89bksK7NR37EyWxO6CgP5EbbaBNHjpQo1ug1VMkdL9fTruxQ58DVY4VX9" is not defined

    at REPL (:1:77)

...TruxQ58DVY4VX9PEKvFijVyCbdT/32OANOrrzOVeDpWjoavSoC28sFviqxza...

                              ^

> 

[RKBoss6]

@gfwilliams
The first upload was with the binary, and that also ran out of memory I assume. Do you know how to tell GitHub it's a binary file?

[thyttan]

I attempted to upload it through the it for me, but the same issue popped up, and I cannot access the watchdog, and the bangle is absolutely frozen. @thyttan how did you recover the bangle? Bluetooth is disabled, is there a fix? Otherwise I'll just let it drain, and attempt a clean boot.

Yeah, that sounds like what happened to mine.

For my case, I was connected to my phone via bluetooth. Disconnecting the bangle from within Gadgetbridge (or disabling phone bluetooth I guess) made it so the watch became responsive to the watchdog. I think that was what did it at least.

[thyttan]

If you can't do the same to toggle the bluetooth connection, maybe see if connecting and disconnecting the charger nudges the Bangle in a similar way?

[RKBoss6]

@thyttan sigh...
I tried all those, but no luck. I'll just wait for a complete discharge. Thanks!

[thyttan]

Failed when trying to install now with these errors:

OUT OF MEMORY    at :1:41

...age').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABA...

                              ^

Uncaught SyntaxError: Got UNFINISHED STRING expected EOF

    at REPL (:1:42)

...ge').write(".tfmodel",atob("IAAAAFRGTDMAAAAAFAAgABwAGAAUABAA...

                              ^

Execution Interrupted

New interpreter error: LOW_MEMORY,MEMORY

Uncaught ReferenceError: "NHy9zY89bksK7NR37EyWxO6CgP5EbbaBNHjpQo1ug1VMkdL9fTruxQ58DVY4VX9" is not defined

    at REPL (:1:77)

...TruxQ58DVY4VX9PEKvFijVyCbdT/32OANOrrzOVeDpWjoavSoC28sFviqxza...

                              ^

> 

And to clarify, this was during upload via the RKBoss6's app loader and not after the app was installed and in use.

[RKBoss6]

So as @gfwilliams said, it's probably the upload of the model, and evaluating it, which takes up memory. How do we bypass that? I'll try reverting the upload back to the first version, which has it as the raw model, not the atob(...). @thyttan or @gfwilliams If you feel up for doing another test with that to collect some debug info, that would be really great. Be careful though...

[thyttan]

It just so happens I have a second Bangle 2 laying around - perfect for this ;)

[thyttan]

Sadly still the same problem occurs on my other watch. Latest cutting edge firmware. Made sure to charge it very little so it should flatten the battery soon.

Edit: Hm, as I was writing this it became responsive again. Maybe again a disconnect of bluetooth did something. Not sure.

[RKBoss6]

Huh, I guess we have to just wait until someone knows something, if you guys know someone who might be able to help, feel free to ping them...

[RKBoss6]

It just so happens I have a second Bangle 2 laying around - perfect for this ;)

I only have one haha! Do you know whether a full drain and clean boot solves it? Getting a bit worried, as mine is still unresponsive...

[thyttan]

Thomas Vikström was at least active on the previous incarnation of the forum, maybe you can contact him via youtube comments: https://www.youtube.com/watch?v=Drn6Nd0vjbo

I'm not sure of his github handle or if he has one.

Maybe helpful: https://github.com/edgeimpulse/expert-projects/blob/main/accelerometer-and-activity-projects/gesture-recognition-banglejs-smartwatch.md

https://github.com/orgs/espruino/discussions/2926

https://github.com/orgs/espruino/discussions?discussions_q=is%3Aopen+%40thomasvikstr%C3%B6m+

Maybe you can contact him via LinkedIn: https://www.linkedin.com/posts/thomas-vikstr%C3%B6m-71b6767_banglejs2-deeplearning-ai-activity-6893241261231357953-gKBh

[thyttan]

It just so happens I have a second Bangle 2 laying around - perfect for this ;)

I only have one haha! Do you know whether a full drain and clean boot solves it? Getting a bit worried, as mine is still unresponsive...

When your battery has run flat and you've charged it (I suggest only a little) keep holding the button when turning the Bangle on. That will trigger the recovery screen where you can choose to factory reset the watch. I can't imagine that would not work.

If you don't want to reset the watch, I think it would also do if you leave the watch laying flat on the table (so you don't accidentally trigger the gesture and problem) and then turn it on normally. Then you'll probably be able to connect to either Web IDE or the app loader to uninstall/remove the app.

Depending on the battery level it may take a while though...

[RKBoss6]

@thyttan Would a clean boot work instead of a factory reset? I'm not sure if the model would be started with that, but maybe it would work?

Edit: As far as I can tell, it only stops calling the boot scripts. It would be a good debug to see whether the boot is at fault, since when I tried reloading the watch after uploading that from the IDE I ran into the memory issue...

[thyttan]

Would a clean boot work instead of a factory reset?

I think so yes. Also thinking some more I believe you should be able to connect to Web IDE or app loader also when the recovery screen is on.

[bobrippling]

Yeah, watchdog (holding the button) should reset it, odd that it isn't for this. Maybe the watchdog is disabled during the file upload? Or its timeout altered?

[thyttan]

Maybe the watchdog is disabled during the file upload? Or its timeout altered?

Clarifying again, the hangup happens when the app is installed and does not produce any error message.

The errors during install from app loader was fixed.

Re timeout I don't know.

[gfwilliams]

I cannot access the watchdog

This is very odd - it should pretty much always work. Not the ~2sec press, but the 10 second one that totally reboots the watch.

The only things that don't allow reboot are storage writes/compaction (as that might corrupt storage) but as far as I can see Tensorflow doesn't mess with the watchdog.

Please can you run E.CRC32(require("Storage").read(".tfmodel")) on your working version? That might help us to tell if the version the app loader is installing is the same, or corrupted

[RKBoss6]

@gfwilliams yep, nothings seems to be working for me, I tried holding it down for a full minute! Just waiting until it discharges, which might take weeks... it was at 70%

[RKBoss6]

Alright, my bangle is back in business! It's charged a fair bit now, so I'll wait for it to get to 10%, so I won't need to wait... Any updates from you, @thyttan?

[gfwilliams]

Just took a look at this, it's a strange one. When I upload your code, I can still use the Bangle, but whenever I physically move it, it tries to load the model and crashes (but the watchdog auto-reboots it). So if it's sitting on the desk it's fine.

But I think the main issue is actually the size of the model. The one in the gesture app is 3616 bytes, but the one in gesturelock is 54768 bytes.

The tensorflow arena size that gets automatically created when handling gestures is 4000 bytes, and I'm not sure but I guess that the 54768 model might use more than that (although it should fail with an error if so).

Did the model ever work for you, or is it just the upload to the app store that causes issues? If it's the app store, please can you upload your model file here, and I'll see if I can spot any difference between what's not on my watch and what you have

[RKBoss6]

Hey, sorry for the pause - the model worked fine, it's just the app loader thats the issue - github doesn't support .lite files, so try this

[RKBoss6]

Hey, just wanted to get back to this. From what we discussed earlier, it looks like it might be a problem with the model size during upload, although it works perfectly through an IDE upload. @gfwilliams does the IDE uploading system do anything differently than the app loader, and if so what? Hopefully we can get this up and running soon...

[gfwilliams]

Well, at the moment the two do update slightly differently, but to be honest I would have expected any upload issues would have surfaced a long time ago if there were some. I'm afraid the link to the model you uploaded has gone now, but I MD5 summed the one on GitHub and got: b9a7ecd8b4665523dc92bbf5e1ca46b7

Can you try MD5 summing the one on your PC and see if it's the same? I still wonder if maybe Git is treating it as a text file or something and trying to change what it thinks are 'line endings'

[gfwilliams]

... is model.js supposed to be an exact copy of model.lite? Because I just copy/pasted it into Espruino and did require("fs").writeFile("model.lite",a) and then md5summed the file, and got 1a78582000eaba8a6ee024d7159e5e21

So they're definitely different, despite being the same size.

In fact I just 'diffed' the two files, and they are different. The model in model.js is actually missing chunks (which are 0xFF) so the upload of those chunks appears to have failed. I'd have said that was the corrupt model, not the model.lite file - and it appears to have been uploaded with an older version of the app loader (as the new version wouldn't do that)

[RKBoss6]

I think I made the js file from inspo of the other gesture ai model app. Can you try uploading both js and lite to a bangle and see which one works? I'll work on getting the MD5 on my machine, but it decided to be difficult and freezes whenever I try. I'll try restarting to see if it works.