Reestablishing Stream and Anonymous Download

atonino

Hi everyone.

I am writing a script which connects to several MMRL (Firmware 1.5.1, HW 0.5) sensors configures them and streams quaternion data.
To increase stability with many sensors connected I reduced the connection interval as well as the data rate of the quaternions by using a passthrough and a timer. Due to range limitation and interference we still experience some disconnects. To not loose to much data I added a logger which starts logging the output of the passthrough when the sensor disconnects from the host.

With this I can reconnect to the sensor and download the missing data.
This works quite well. However, if the reconnect fails due to a *** buffer overflow detected ***: terminated error or similar, I need to reinitialize the state of the sensor and start from scratch.

I have written and attached a demonstrator script. There is a menu which lets you step though different steps in communication. I'll add the steps to reproduce the behavior to each problem.

With this background I do have the following problems:

1. I am not able to use the anonymous download function to download the recorded data:
When I log data from the passthrough I get following error: terminate called after throwing an instance of 'std::out_of_range' what(): _Map_base::at when I try to read any anonymous data signals.

Steps to reproduce:
1. Connect to sensor,
2. "3": Configure,
3. "2": Disconnect (Wait for red LED starts to flash, this indicates that the sensor started logging)
4. "n": New State/Connection (if it fails just restart the script and continue)
5. "9" Anonymous Download

If you change the logger to log quaternion data directly by changing "self.processor" to "signal" in line 109 the script and the anonymous download does work. However, this would mean to log 100Hz data which limits the memory and increases download time.

Is there anything I can do differently with the data processor or the anonymous download to download the logged data from the passthrough?

2. How to recreate the received_unknown_entry messages:
If I do not use the anonymous download method and instead reconfigure the sensor (get a new Logger ID and a new data processor) and log some data and start the normal download I get all the previous logged data as "unknown_entry" messages: Download: {epoch: 1642598903745, id: 3, length: 4 value: <mbientlab.metawear.cbindings.LP_c_ubyte object at 0x7f5010e6fa40> : 2}

Steps to reproduce:
1. Connect to sensor,
2. "3": Configure,
3. "2": Disconnect (Wait for red LED starts to flash, this indicates that the sensor started logging)
4. "n": New State/Connection (if it fails just restart the script and continue)
5. "3": Configure again (add new logger and second processor to configuration on the sensor)
6. "6": Start Logging
7. "7": Stop Logging
8. "8": Download Data

There are 4 packages for each of the log entries (id0-3). How can I recreate the original message?
This does only work once due to limitations of processors and loggers on the sensor.

3. Is it possible to somehow reconnect to the processor created from the previous session and restart streaming?:
If I monitor the bluetooth connection using btmon (sudo btmon in a different terminal) I can see that after a reconnection to a configured sensor the sensor still streams data
Steps to reproduce:
1. open btmon
2. Connect to sensor
3. "3": Configure,
4. "4": Start Streaming (you can see all messages streamed from the sensor)
5. "2": Disconnect (you need to be quick with hitting enter on this one)
6. "n": New State/Connection (if it fails just restart the script and continue)
7. you can still see all messages streamed from the sensor

Is it possible to attach a new callback function to this data stream?
I also could disable the stream using the disconnect event on the sensor. Is there a way to reconnect to the created processor?
I tried to save the processor id and getting the corresponding processor using mbl_mw_dataprocessor_lookup_id but this also failes with terminate called after throwing an instance of 'std::out_of_range' what(): _Map_base::at
Right now I can stream the quaternion data only when I use the direct signal, which again raises the data rate to 100Hz (you can do the same by disabling line 152).

Thank you for your help.
Alex

Everything tested on Ubuntu 20.04.3 LTS, Python 3.8.10, BT ver: 5.53, Intel AX200, Metawear 1.0.2 , some of it tested on Raspbian: Python 3.7.3, Metawear 1.0.2 using Realtek BLE dongles

atonino

Here is the script

atonino

Anyone who can help me?
Do you need more information?
Anything basic I've missed?

ryanferrell

@atonino

I might be completely off base — and my background is in Swift — but I had to poke at a similar sounding crash when writing the Swift Combine SDK that involved downloading data processor-gated or event-timer-polled logger.

I believe the fix for me, but perhaps not for you, is using mbl_mw_anonymous_datasignal_subscribe and not mbl_mw_logger_subscribe in the download method. At least in my case, I fuzzily remember the logger id coming back with some URL query items. If that doesn't fix it, can you confirm what array of logger IDs are present before you perform the download?

Ryan

atonino

Hi Ryan,
thank you for your input. I tried to switch this two commands. However, I cannot see any difference in behavior of the script.
For Problem 1: The script fails at while waiting for self.anonymous_signal_event set in the self.anonymous_signal_handler. I am not able to get any loggers after a fresh reconnect. mbl_mw_logger_lookup_id or similar functions just crash with a Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) Error.

As an identifier I get quaternion:passthrough?id=1 for the logger, which makes a lot of sense.

Using serialize and deserialize does not work with the processor setup.

Is there a way to reattach to a already present processor or logger signal on the MMR sensor when reconnecting?
Thanks
Alex

atonino

Can anyone help me?
Reconnecting to a already running processor/logger is quite a important step for our project. This enables us to not have any (major) data loss.

ryanferrell

@atonino

Quite important to our own projects, too. It's possible this is a bug in the C++ library. See if the attachment to my response in this post is a fix. There's a PR in review for it.

atonino

Hi ryan,

thanks. I was able to use the most current state of the c++ library. At first glance it looks promising. I was able to use anonymous download. I need to do more testing. I will follow up next week.

thanks
Alex

atonino

Hi ryan,

I am able to use the anon. download function with the most current cpp and py revision on github.

Also, mbl_mw_logger_lookup_id does work. But only if I used anonymous download first. Otherwise it still errors with Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

alex

Laura

That sounds correct @atonino, you can check out our example apps, they are open source and use anonymous downloads (especially the new MetaWear iOS one).