Logging Sensor Fusion and Raw Data
Hello there,
To start off, my setup is as follows: 7 MetaMotionC , 1 Rpi 4, 2 BLE Dongle (4 sensors on first Ble donlge , and 3 on the second), Code in Python. The purpose of the project is to record specific exercise data such as Acc, Gyro, Mag, Pressure, and Quaternion while the user is doing specific sets of exercises. I have all 7 sensors attached to the body and the Rpi (server that starts the sensors) on the ground or on a table. The Rpi and sensors aren't far apart from one another (Less then a meter apart).
Now, from what I have learned from searching through the forums and testing with my code are the following points related to logging:
1.There is a limited number of loggers that can be initiated by mbl_mw_datasignal_log()
2. Many raw sensors outputs can be logged at once by mbl_mw_datasignal_log() due to each having a small size (ex,raw Acc & raw Gyro & raw Mag & pressure all in the same session)
2. A maximum of 2 Sensor Fusion elements (Quaternion, Corrected Acc, Corrected gyro, etc.) can be logged any one time by mbl_mw_datasignal_log()
3. When it is needed to log both Raw data and Sensor Fusion, the only possible combination is 1 sensor fusion element and 2 raw data elements (ex, Quaternion & raw acc & raw gyro)
4. The number of elements that you are able to log with mbl_mw_datasignal_log() doesn't change even if you try to use the fuser processor to group all raw data (ex. raw acc & raw gyro & raw mag & pres) together in one logger object and another with a sensor fusion element (ex. Quaternion)
5. Many of the common processors don't work on sensor fusion elements for sensor fusion data manipulation is not supported
I would appreciate to know if everything I have stated as if now is correct.
So, I had initially wanted to log 5 components (Corrected Acc, Corrected Gyro, Corrected Mag, Quaternions, and Pressure). This is what I ideally want to do. However, due to the fact that only 2 elements can be logged from the sensor fusion module, I had to reduce it to only the Quaternions and switch back to the Raw Acc, Raw Gyro, Raw Mag as they are smaller in size and can all be logged. Next, I realised that it is was only possible to log 1 sensor fusion with only 2 additional raw data elements. I tried to group all of the 4 raw data elements into one data fuser processor to have only 2 log objects (the fuser processor and the Quaternions) but did not work as it seems that the same size is still being registered by the logger.
Now I am stuck as I need to log all 5 elements but can't think of any other solutions. I have thought up of a work around which is LOGGING all 4 raw elements (Raw Acc, Raw Gyro, Raw Mag, Pressure) and STREAMING the 1 sensor fusion element (Quaternions) and it works. However, I would like log all of them if possible so are there any solution I can try??
I have had a lot of trouble with streaming as I have been getting many disconnections. I have been able to create a reconnect function to handle a sudden disconnection is detected but I don't know how to make it continue from where it left off. For example, I would not know did it get disconnected before streaming so I would just need to connect it and make it wait at stand by, or did it disconnected while streaming which would take some time to connect and start recording again and thus lead to many of the data being lost which would ruin my session. Restarting the whole system would be a better choice and redoing the session than writing a complicated re connection algorithm. I had used the maximum BLE parameters to try to have a stable connection but it still isn't that reliable:
libmetawear.mbl_mw_settings_set_ad_parameters(self.device.board, 1000, 0, 0)
libmetawear.mbl_mw_settings_set_tx_power(self.device.board, 4)
libmetawear.mbl_mw_settings_set_connection_parameters(self.device.board, 7.5, 7.5, 0, 6000)
sleep(1.5)
The problem is not the number of sensors connected to each BLE dongle( 4 - 3 sensors) for it is mentioned in the documentations that this number of sensors should have a reliable connection. I do restart the sensors when I disconnect to it manually which of course means that the previous mentioned connection settings (such as Tx-power) are reset to default but I reinitialise them again when I first connect to a sensor which should strengthen the connection after it has been connected.
To summarise, here are my main questions:
1. Is there anyway you can think of to log Corrected Acc, Corrected Gyro, Corrected Mag, Quaternion, and Pressure?
2. If not, how about logging Raw Acc, Raw Gyro, Raw Mag, Quaternion , and Pressure?
3. if not, then I would need to stream the Quaternion. Is there anyway to strengthen the connection stability so that a disconnection doesn't occur. Additionally, in a different post it was mentioned that you could reduce the Quaternion frequency (default 100 Hz) by using the processors (Accumulator/Averager) but that is not possible as the processors don't work on sensor fusion elements as previously discussed. So, is there any additional techniques to reduce the Quaternion frequency from 100Hz to something 25Hz because I will need to stream a maximum of 4 sensor Quaternions to one BLE Donge and as it is known the Maximum data throughput for a BLE connection is at 100-120 Hz.
I know you will ask what is the need to record of these elements but they are all necessary for my research project.
Comments
1.There is a limited number of loggers that can be initiated by mbl_mw_datasignal_log()
True
2. Many raw sensors outputs can be logged at once by mbl_mw_datasignal_log() due to each having a small size (ex,raw Acc & raw Gyro & raw Mag & pressure all in the same session)
True
3. A maximum of 2 Sensor Fusion elements (Quaternion, Corrected Acc, Corrected gyro, etc.) can be logged any one time by mbl_mw_datasignal_log()
False, only 1
4. When it is needed to log both Raw data and Sensor Fusion, the only possible combination is 1 sensor fusion element and 2 raw data elements (ex, Quaternion & raw acc & raw gyro)
True, you might be able to push it to quat + acc + gyro + mag but the log memory will fill very quickly.
4. The number of elements that you are able to log with mbl_mw_datasignal_log() doesn't change even if you try to use the fuser processor to group all raw data (ex. raw acc & raw gyro & raw mag & pres) together in one logger object and another with a sensor fusion element (ex. Quaternion)
Do not use the fuser for logging
5. Many of the common processors don't work on sensor fusion elements for sensor fusion data manipulation is not supported
False
You can log quat + raw acc + gyro + mag + pressure as long as you keep the raw data at low frequency. Again, you will fill up your memory VERY FAST.
Same as above.
That is incorrect. You can use any non-math related processor on the sensor fusion signals. You can use the time filter to down-sample the quaternions.
https://github.com/mbientlab/MetaWear-SDK-Cpp/blob/master/cppdocs/source/dataprocessor.rst#time
Thank you for the quick reply.
a. So my question is then how would you down sample the raw acc, raw gyro and raw mag? For in sensor fusion it internally configures the ODR of the Acc and Gyro to 100 Hz and the mag to 25 Hz when NDOF is picked as the sensor fusion mode. Would I be able to override the NDOF configuration with a future configurations for each sensor such as this:
b. Also to be clear, my first question was related to the corrected version of the raw data such Corrected Acc and Corrected Gyro that is within the sensor fusion module. You had gracefully mentioned that only 1 sensor element can be logged, so I guess what I originally wanted to do is impossible as I would need 4 sensor fusion elements to log (Corrected Acc, Corrected, Gyro, Corrected Mag, Quaternion). So I will have to try to log the raw Acc,raw Gyro, and raw mag.
c. Also shouldn't down sampling the raw data also result in the down sampling of the Quarternion as they are used to calculate the Quaternion?
d. You said I would be able to log all the those data elements (Raw acc, raw gyro, raw mag, pressure, and Quaternion) if I down sample the raw data (Acc, gyro, mag) but why is that if I am logging? Shouldn't it not matter the frequency I am sampling the data in when I am logging as it will eventually all be transferred after recording to the server (Rpi) through a constant connection speed even if it takes longer than it took to record?
EDIT
e. So I have tried to down sample the Quaternions, Raw Acc, Raw Gyro, and Raw Mag (without pressure) with the time processor at a sampling frequency of 20 Hz for each but I am still unable to log these 4 elements. I am able to log with 3 elements (Quaternion, Raw Acc, and Raw Gyro) but that also works even without the time processor. So it seems as if now that the time processor doesn't really have any effect.
a. That's correct. You would need to downsample with the processors. To be honest, this just isn't something that we recommend doing. You should do sensor fusion OR raw sensor data. We didn't enable sensor fusion + raw data by default because the BOSCH sensor fusion library does take over all the settings and processing power. You are definitely outside the realm of our recommended scope. At most, we recommend sensor fusion + 1 data signal.
b. Yes, again, only one sensor fusion output at a time. That is a BOSCH library rule.
c. No, the sensor itself isn't downsampling, only the items sent over the BLE link are.
d. It's possible but we honestly you shouldn't do it. You are operating outside of the recommended realm of our sensors. At most we can support one sensor fusion + 1 raw data. I don't know and can't support you doing more than that.
e. As I mentioned, we don't support this so it isn't necessarily going to work.
Okay thank you for all of your replies.
Then how about my question about establishing a more stable connection for streaming? Is there anything additional I can do to create the BLE connection between the RPi and my sensors more stable other than the 3 line of codes that I have already previously posted (ad parameters, tx power, set connection parameters)?
Unfortunately not, you have hit the limits of BLE.
I would like to point out that the main problem is NOT BLE bandwidth. I have some experience with a comparable product (Moticon Insole3), which happily logs 3D acc, 3D gyro, timestamp, temperature, and 16 channels of pressure data, at 100 Hz. Oh, and we can reliably stream that data from two feet/insoles at the same time.
It amazes me how well the mbientlab APIs have been built, but how they expose the BLE data layer at the same time. Each packet can contain just a single type of data in your architecture, and at best a number of samples OR a sample number can be packed into a single packet using some difficult configuration options.
As a user of a sensor board I just want the data, at the highest sample rate possible, synchronized in my client. Sensor fusion is a nice extra that could be implemented on the server side or on the client side, but if I go down that route I want orientation (quaternions/euler angles) AND linear acceleration, not one OR the other.
I think mbientlab sensors could be much more competitive if they could just provide raw 10DOF data @ 100Hz, which should be easy to obtain: take 16 bit values for each channel, and you are looking at 160 bits * 100 Hz = 16 kbit/s, which is WAY below the theoretical maximum even for BLE 4.1, let alone 4.2 or 5.0. Probably there is headroom for sending packets more than once to reduce data loss, or even implement resends on demand in the case of packet loss.
Heck, while you're at it, implement a nice sensor fusion algorithm on the client side, including a zero-velocity update (ZUPT) algorithm and you're on your way to lead the field of Pedestrian Dead Reckoning applications.
The thing is, we don't use our own BLE libraries. We rely on third party Bluetooth libraries and we are completely at their mercy. I will agree some BLE libraries are better written than others.
The names of functions to config Acc and Gyro are a little bit confusing: for Acc, it is called libmetawear.mbl_mw_acc_set_odr; while for Gyro it is libmetawear.mbl_mw_gyro_bmi160_set_odr.
For Acc ODR, the input parameter is a direct float value; while for Gyro, it is an enum GyroBmi160Odr._25Hz as in the Python example code.
In file MetaWear-SDK-Cpp/blob/master/bindings/python/mbientlab/metawear/cbindings.py, there is
class AccBmi160Odr:
_0_78125Hz = 0
_1_5625Hz = 1
_3_125Hz = 2
_6_25Hz = 3
_12_5Hz = 4
_25Hz = 5
_50Hz = 6
_100Hz = 7
_200Hz = 8
_400Hz = 9
_800Hz = 10
_1600Hz = 11
My questions are
(1) Why this AccBmi160Odr enum is not used in the Python example code to configure Acc?
(2) What are the first 4 values in AccBmi160Odr?