›

Board stops sending data after few sequential connect-disconnect cycles (possibly a firmware bug)

mykolajmykolaj
edited September 2016 in Android
Hello!
I'm developing Android application which connects to as many MetaWear boards as possible on a device it is launched. After a connection is established this program listens to 3 sensors on every board (Accelerometer, Gyroscope and Magnetometer) and saves data as CSV. Nothing new there. There's a switch which can be used to start and stop data capturing. After this switch is set to OFF the program unsubscribes from streams but leaves all boards connected. Every time this switch is set to ON the program starts those streams again. What is bothering me is that after a some number of those start-stop cycles some boards stop sending any data until the board is reset. Then it starts to send the data again. The number of ON-OFF cycles to cause such behaviour can sometimes be 3, and sometimes it can be 10.

Screenshot: image

The program creates a folder '/sdcard/MetaWearDemo' where a file for each device data is stored. Empty file contains only a header and has a size of 63 bytes.

In short the program's flow can be described like this:

1. Connect to a board. This is done when a user checks a checkbox in a list:
2. Connect to sensors. This is done when a user turns switch to ON:
3. Start listen to data:
4. Stop listening to data. This is done when a user set a switch to OFF

After a few repetition of steps from 2 through 4 some devices stop sending data until manually reset. This can be indicated by their respective csv file being empty (63 bytes, contains only a header)

What i've tried so far to solve this issue:
1. Tried to call board.disconnect() every time a switch gets set to OFF. This does not help.
2. Tried to call board.tearDown() before call to module.routeData(). This does not help either.
3. Tried to call board.tearDown() every time a switch gets set to OFF. No help.

Only thing which helps is to reset a device before every connect, but this is not a solution:
Debug debugModule = board.getModule(Debug.class);
debugModule.resetDevice();

-------------------------------------------
Android version i'm testing with is '5.0.2'.
Phone is 'HTC One M9'.

Devices affected by described behaviour has next parameters:
Model number:      2
Firmware revision: 1.2.3
Hardware revision: 0.2

Maybe there are others too, but i don't know that.
-------------------------------------------

I've prepared a demo application for you to check this behaviour. Link: git clone https://mykolaj@bitbucket.org/mykolaj/metawear-firmware-bug-demo.git
Please try to test with it, and help me to identify is there a problem on firmware's side, or it's on my app's side.

PS: I was trying to include more code in this post but validator does not allow that much text.

Comments

  • asadasad
    Hi,
    We are trying to stream gyro data at 100hz and log in to a file. We are using buffering and queues so that Process callback functions gets completed within 10ms. However sometimes it take more than 10ms (around 15-17) ms to complete the callback. We narrowed down this issue when we noticed our sensors go in freeze mode (LED etc work but no streaming) after some recording sessions and we needed to reset them to get them back to work. 
    I want to ask following now. 
    What could be possibilities to recover from this issue except resetting sensor? 
    Can I call tearDown() method after couple of delayed callbacks? will it help us in this scenario?
     Will i need to stop previously configured gyro module and reconfigure after calling teardown?

    Sensor hardware version is 0.3
    SDK version is 2.6 (Latest I guess)
    I am currently using high frequency mode to stream at 100hz.


  • EricEric
    • What Android OS and device are you using?
    • How does this issue happen exactly?  Do you have a stream going and then it randomly stops without the user doing anything?
      • Can this be reliably reproduced with your setup
    The tearDown function does not interact with any sensors so no, it won't help.
  • mgdbgjmgdbgj
    edited September 2016
    I've experienced the same "lack of streaming" as well ...  I'm actually streaming accelerometer (50Hz) and gyro (50Hz) from High Frequency Stream and Barometer and Magnetometer (20Hz), but I experienced the problem just streaming accelerometer and gyro at 20Hz from regular stream. 

    Everything seems to work for as long as 15 minutes, but then eventually the streaming stops; the app continues to run and Android Studio does not log any errors as far as I can tell (although I have to confirm).  Closing my app and switching to Google Play store Metawear app does not reinitialize the streaming either.  BLE communication continues to work and LED can be turned on and off, but neither of the sensors will stream data until battery is removed and reinserted.  

    I had a hypothesis that battery power drops below some required level and disables the sensors or just the interrupts for streaming the data, or simply resets the board ...  but it does not explain why reinserting the battery is necessary, or while reinserting the same battery solves the problem, or why LED and BLE in general continues to function normally.

    Metawear CPRO firmware 1.2.3
    Android 6 on Nexus 5
    SDK 2.6

  • siroborissiroboris
    Hi, I am experiencing essentially the exact same issue as mgdbgj and asad. I am also using the High Frequency Stream of the Accelerometer and Gyro. I just need to let the stream run and it will break down after anywhere between 30s and 2 minutes.

    Removing the battery then allows me to reconnect and continue the stream, however killing my app and restarting it also allows me to reconnect and stream.

    I have the same setup as mgdbgj, only running Android 7 on the Nexus 5x.

    If anyone has any updates on how to resolve this issue, I would be very much appreciative.
    Cheers!
  • mgdbgjmgdbgj
    edited September 2016
    After additional testing ...

    (CPRO Hardware Version 0.2)

    - Android Studio does not report any errors.
    - Stopping app (Force Stop) does not solve the problem.

    But ...

    - Calling soft reset mwBoard.getModule(Debug.class).resetDevice()); does seem to reset the streaming capability.
  • mykolajmykolaj
    Screenshot can be found here: http://joxi.net/LmGEPb9FRqebzA
  • EricEric
    edited September 2016
    Talked with the firmware dev and we're getting a fix out for this issue.
  • mgdbgjmgdbgj
    Fix will be appreciated ...

    Another data point  just in case...

    I did NOT record any difference with regards to this issue between streaming continuously and turning sensors on and off as fast as 0.3Hz.

    -- Magda


  • siroborissiroboris
    Since updating my boards to firmware 1.2.4 I am still experiencing this same issue.

    @Magda: can you confirm that updating the firmware resolved the issue for you? If so my problem must lie somewhere else...

    Cheers!
  • mgdbgjmgdbgj
    My initial tests still show the loss of data streaming ... with firmware 1.2.4. 

    I recently switched to a fresher battery batch (still Energizer #ECR2032BP) and installed the CPRO in a little case to avoid contact while it is running, and the issue seems to appear less often than originally but it still does happen, and is unpredictable anywhere from 2-30 minutes of runtime.

  • EricEric
    Do you have other devices you can test on?  It is very odd that 2 newer Android devices are having Bluetooth issues.
  • mykolajmykolaj
    edited October 2016
    Thank you for attention to this post.
    Sorry but i can not test earlier than October 23.
  • nightdnightd
    I have this issue too, running BLE in a Linux host with C++ SDK, after running for a while the sensor handler were stopped being called, resetting device will make it works again but the issue continue. Have used latest 1.2.3 firmware though.
  • nightdnightd
    edited October 2016

    Firmware: 1.2.4
    C++ SDK: 5.22
    Host: Linux 3.10
    BlueZ: 5.39

    After several minutes running our program which subscribes to a CPRO accelerometer, gyro and magnetometer sensors, apparently it stop receiving sensor data, in this case all signal handlers were stopped being called. We have had this issue since the previous firmware and SDK. With device reset, the stream shall run again then unfortunately stop after some times.

    We tested using latest 1.2.4 firmware since as the release notes say it may have addressed this issue.

  • EricEric
    For those who have not provided the following details:
    • How fast are you running the sensors?
    • Does this occur for all frequencies or are these some frequencies which give a working stream?
    • How long does it take for the stream to stop?

  • nightdnightd
    Hi Eric,

    Answering your questions:

    1. It was the lowest possible rate, the program utilize configuration file for sensor's range and output data rate (ODR). In test, the rate was 4Hz for Accelerometer, 25Hz Gyro and LOW_POWER preset Magnetometer, but even if I only subscribe Accelero, the stream was also stopped. Restarting the program (create board connection, reinitialize, re-enable and subscribe) didn't help making the signal handler called.

    2. I have tested with slightly upper freqs, same result I believe.

    3. Like 10 minutes or so, sometime less. I presume the behavior much faster after several stop and go of the program.

    Let me know if you have more questions.
  • EricEric
    I tested data streaming today in the office on our Nexus 9 (Android N), 7 (Android 5.1), and Win10 laptop and could not hit this bug on firmware v1.2.4 while streaming gyro and acc at 100Hz each for 1hr.  Not saying there isn't anything wrong with the firmware but I was not able to reproduce with our dev devices.  Unfortunately we don't have a nexus 5/5x on hand so I cannot check using those devices.

    In the meantime, can you guys try reducing the max connection interval to 7.5ms before you start the stream?  The decreased connection interval will increase the amount of times the host device asks for data from the peripheral.
  • nightdnightd
    Hi Eric,

    * Did you test to run acc and gyro at 100Hz for each? that means to subscribe/stream acc and gyro at the same time?
    * Have you tried to stop the Win10 program and run again? you may need to do it after several times.

  • EricEric
    edited October 2016
    • Yes, both sensors are streaming at 100Hz each and use the high frequency data signals.
    • No, I just let it run for an hour and print the # of received samples every second.  I did test just now stopping and starting it every few minutes but again, nothing out of the ordinary.
    Give the reduced connection interval a try and see if that imrpoves performance.
  • kkflfkkflf
    edited October 2016 
    Hello!

    Device information:
    Model number: 1
    Serial Number: 004F20
    Firmware Revision: 1.2.4
    Hardware Revision: 0.3
    Battery Level : 99

    I am experiencing the same problems as previous posters have described.

    I have designed an Android application where the user can start and stop the stream multiple times in a session. The applications logs the accelerometer and gyroscope at 50hz each.

    I have tested the applications multiple times, both with sessions of continuously streaming and sessions where the user have started and stopped the stream mid session.

    At some point the sensors stops streaming and I am forced to reset the sensor before it can stream data again.

    I have tested the application with both Android version 6 and 7.

    I have not changed the max connection interval. As far as I know, 11.25ms is the fastest connections interval allowed in Android version 6+.

    I have uploaded some sample code of how I log the     accelerometer and gyroscope:

    http://pastebin.com/kwQN7kns

    Eric, I noticed that you are streaming accelerometer and gyroscope at 100hz.Is this 100hz per sensor or in total?
    If it is 100hz per sensor, how do you achieve this?

    I have tried multiple times to stream 100hz per sensor without success. I have since discovered your FAQ, which states that Bluetooth LE can handle data throughput of up to 100hz. I am therefore wondering how you manged to stream at 100Hz per sensor.
  • EricEric
    @afzaal
    Your issues are not relevant to the issue described in this thread.  I am putting your posts into their own thread, here.
  • EricEric
    @kkflf
    Also, that faq refers to the throughput of the btle transmission which is not necessarily equal the frequency at which sensors create the data.
  • NathanNathan
    Have there been any more issues with streaming data to a Nexus 5 since the firmware was updated to v.1.2.5? I'm hoping to buy a phone for my research project, and want to select one with no streaming issues.
  • EricEric
    Haven't had any issues using either a Nexus 6 or 9.
This discussion has been closed.