What is the best latency possible using SensorFusion APIs with MMR?

Segotia

Hello mbientlabbers!

We are doing a latency-critical audio App that needs to connect to and receive data from an orientation sensor. We are currently evaluating the MMR.

We are observing very poor latencies and so we neen to understand all the contributing sources of this total end-to-end latency.

I now all I need to know about the latencies within my App's audio path. For the MMR I see 2 main sources of latency:
1. The Sensor Fusion data latency: that is, the speed with which Sensor fusion data are captured/processed/ready-to-go
2. The **BLE transmission latency **: that is, the time it takes for the SensorFusion data to transmit via BLE from the MMR HW to my Android Phone's BLE HW and on to the mbientlab APIs

Can you please clarify the following:

Sensor Fusion data latency
We are using sample code out of the 3D Cube App and, in particular we use the following:

public void onServiceConnected(ComponentName componentName, IBinder service) 
{
        BtleService.LocalBinder binder = (BtleService.LocalBinder) service;
        board = binder.getMetaWearBoard(btDevice);

        final SensorFusionBosch sensorFusion = board.getModule(SensorFusionBosch.class);
        sensorFusion.configure()
                .mode(SensorFusionBosch.Mode.NDOF)
                .accRange(SensorFusionBosch.AccRange.AR_16G)
                .gyroRange(SensorFusionBosch.GyroRange.GR_2000DPS)
                .commit();

        sensorFusion.eulerAngles().addRouteAsync(source -> source.limit(1).stream((data, env) -> {
            onSensorFusionDataChanged(data.value(float[].class));
        })).continueWith((Continuation<Route, Void>) ignored -> {
            sensorFusion.eulerAngles().start();
            sensorFusion.start();
            return null;
        });
}

As you can see, we set source.limit(1) , i.e. its lowest possible value.
QUESTION: Will the above configure SensorFusion to give me orientation data as fast as is possible? Or is there another, better way to minimise the SensorFusion data generation latency?

BLE transmission latency
I know that a large portion of BLE transmission latency is out of mbientlab's control, however:
QUESTION: Can you advise what, if anything, we can do with mbientlab APIs to minimise this latency and what in your experience is this latency on typical "bought-in-the-last-year" Android phones?

Thanks!
Team Segotia

Matt

Hi @Segotia

Let me try to answer all your questions.

1. 100Hz is the typical sampling rate fed into sensor fusion, which represents 10ms sample to sample. The sensor fusion itself may appear as a low pass element to orientation change.
2. BLE transmission latency comes down the BLE "Connection Interval". The theoretical min is 7.5ms, but typical phones will be more in the 12.5ms to 20ms range. On average you should experience around half this latency, but it will vary sample to sample unless the connection interval matches the sampling rate.
3. I am not the android expert, but if you are using the fastest setting available there likely is not much more to do from the fusion API side. If faster than 100Hz operation is necessary it may be better to develop a custom orientation model/predictor and work with the raw data.
4. You will want to have a look at the BLE Connection Parameters, there is a call to have the MetaMotion device attempt to renegotiate connection parameters with the client. Actual results vary based on the phone, other connected devices, etc. https://mbientlab.com/documents/metawear/android/latest/com/mbientlab/metawear/module/Settings.BleConnectionParameters.html
5. Typical minimum connection intervals allowable vary from 7.5ms to 20ms, it depends on the limits set by the phone manufacturer, some chipsets have wifi coexistence features, etc. 12.5ms used to be fairly standard on android, 20ms on iphone.

Segotia

Thanks Matt for this useful information.

Can we come back please to the code I posted, based on the Cube Sensor Fusion project and, in particular:

sensorFusion.eulerAngles().addRouteAsync(source -> source.limit(1).stream((data, env) -> {

If I read up on what limit(1) means I find:
/**
* Reduce the amount of data allowed through such that the output data rate matches the delay
* @param period How often to allow data through, in milliseconds (ms)
* @return Object representing the output of the limiter
*/
RouteComponent limit(int period);

And so this is why I formed the opinion that I can get a 1ms "limit", which corresponds to information received from Mbientlab that we can get 1ms latency or, to put it another way, 1kHz data rate.

But this is possibly at odds with your statement: "100Hz is the typical sampling rate fed into sensor fusion," - can you clarify what is the impact of the limit above on the data rate/latency I can hope to achieve, taking other factors (eg BLE) out of the equation?

Thanks!

Segotia

@Matt said:
Hi @Segotia

Let me try to answer all your questions.

1. 100Hz is the typical sampling rate fed into sensor fusion, which represents 10ms sample to sample. The sensor fusion itself may appear as a low pass element to orientation change.
2. BLE transmission latency comes down the BLE "Connection Interval". The theoretical min is 7.5ms, but typical phones will be more in the 12.5ms to 20ms range. On average you should experience around half this latency, but it will vary sample to sample unless the connection interval matches the sampling rate.
3. I am not the android expert, but if you are using the fastest setting available there likely is not much more to do from the fusion API side. If faster than 100Hz operation is necessary it may be better to develop a custom orientation model/predictor and work with the raw data.
4. You will want to have a look at the BLE Connection Parameters, there is a call to have the MetaMotion device attempt to renegotiate connection parameters with the client. Actual results vary based on the phone, other connected devices, etc. https://mbientlab.com/documents/metawear/android/latest/com/mbientlab/metawear/module/Settings.BleConnectionParameters.html
5. Typical minimum connection intervals allowable vary from 7.5ms to 20ms, it depends on the limits set by the phone manufacturer, some chipsets have wifi coexistence features, etc. 12.5ms used to be fairly standard on android, 20ms on iphone.

Hello Matt
If you have some time, could you please clarify on our follow-up question?
Thanks

Matt

@Segotia

I did not recognize the android syntax initially. limit(1) is invoking a firmware downsampling filter on the data source. As called it allows one sample to pass per 1ms, so signals with sampling rates greater than 1kHz will be reduced to 1kHz. For data sources with sampling rates less than 1kHz, it has no effect.

The sensor fusion data rate is 100Hz so it has no effect in this case.

Matt