Streaming Issues

rhristov

We have around 15 MetawearR sensors. We are using node.js + bleno to program them as this was the most reliable option to achieve the application that we want.

We are continuously streaming data from the accelerometer at 50Hz, but sometimes the streaming just stops. Even if we use the metawear app, we still can't stream acceleration, even though we can connect to the sensor. We found one solution to this problem - do a soft reset, either in the app or through our nodejs code.

However, sometimes the soft reset leads to unresponsive board. A hard reset (shortening the pins) brings it back to working state, but we can't afford to do this.

We discovered when the issues with the streaming happen: All the times when the streaming stops is when the sensor is on the boundary of the range of the receiver. The frequency of these events is around 1 every 2 hours.

Please let us know if you have any ideas or if you want more information!

Laura

First and foremost, you should build in the reset and data connection error handling into your node code. 

Your node code should be "crash proof" because as we all know, Bluetooth devices and stacks can become corrupted!

Clean up your code, handle errors and report back on your system once this is done.

The only other info/feedback I can give you, considering you haven't given us much to go on, is to try to use multiple dongles and to only handle about 4 to 5 connections per dongle at a time. Try to handle connections in parallel using separate dongles/antennas as much as possible (so it is less likely for the ble stack to get corrupted).

Finally, test only a few sensors at a time. 

Test 5 sensors with 1 dongle, make sure it's reliable. 

Then add another dongle and test 10 sensor connections...and so on. 

Testing 15 at once is a recipe for disaster.

rhristov

Thanks for the quick reply.

We are not using the 15 sensors at once. The problem appears even if there is 1 sensor connected to the bluetooth card. Also, we handle errors in our code, but there is nothing indicating that the streaming has stopped. The way we detect there is some issue is when we don't get packets for around 20-30 seconds and there is no disconnect event.

I tried reducing the max connection interval to 7.5ms (as suggested in http://community.mbientlab.com/discussion/1887/gyro-accelerometer-stops-sending-data-until-soft-reset), but it did not help.

Also I tried streaming and logging through both the MetaWear app and the MetaHub app, but none of them work. 

Currently, I am wondering whether I can reset just the accelerometer from software instead of doing a soft reset of the whole board?

Laura

I saw similar behavior when I was working on the Python SDK using pygatt. Pygatt corrupted the BLE link and the MetaSensor would crash. It could only be recovered via a hard reset or a firmware upgrade.

Once I updated to the Bluez ble lib, I didn't have any issues.

So can you try to figure out a few things:

1. Isolate the issue to a specific ble function.

2. Does the same issue happen when you use our Python + Bluez sdk?

Report back.

Matt

@rhistov

An issue with this failure signature was previously identified and fixed -- a corner case could cause our low priority TX queue to lockup.  Notification type messages like accelerometer streaming data would get squelched, but high priority message like direct API reads would still work.

What firmware release version are you running?  That way I can confirm whether the fix is present or if this is another issue.

rhristov

Laura, unfortunately it is hard for me to change to python + bluez as there are compatibility of the software I am running. 

I guess, there are two separate problems: 

1) Acceleration stops streaming and I need to soft reset. This happens when I am simply streaming at 50Hz and there is no specific ble function that causes it.

2) Soft reset brings the device to a state, which needs a hard reset. Here, I initiate the soft reset in a manual way by looking at: https://github.com/mbientlab/MetaWear-SDK-Cpp/blob/21b15b4d78ccdd1c0c9b23f53fdbe14d4340edc8/src/metawear/core/cpp/debug.cpp

I was executing debug_reset, but I realized there is debug_reset_after_gc, so I switched to it this afternoon.

Matt:

We are running 1.2.5, which I believe is the latest? We are using Metawear R sensors.

Matt

@rhristov

r1.2.5 is the latest public release for MetaWearR.  We have an upcoming release across all models that is in the final stages of testing.  I reviewed the commit log, and there is one change that should fix your streaming lock up.

As for the soft reset, it is a very simple operation and should not cause a device to lock up.  We would again prefer to see if this issue persists in the new release.

debug_reset_after_gc only allows the sensor an opportunity to erase logger module flash pages that have already been read out before it resets.  This requires the device to be disconnected from first by other means because it cannot perform erase operations while in a radio connection.  It is not necessary to use this reset type unless you are logging.

rhristov

Matt,

Thanks for the response. Do you have any ETA on the new release? Any way I can apply it to my sensors before you publicly release it? I am really eager to solve this issue as I am supporting around 15 sensors, which are distributed at different physical locations and each lock up requires me to manually hard reset it.

Eric

The firmware should be available next week after we have updated the various SDKs and tested the new features.

rhristov

Eric, Is there any updates or new ETA for the firmware upgrade?

Eric

Firmware update is now available.

rhristov

I upgraded the firmware on some of the boards and will monitor to see whether the issue will appear again. Thank you