TimeoutException: Creating a Metawear timer timed out after 3000ms

vthompson

My app utilizes a MetaWear (Model #2, firmware rev: 1.2.5, Hardware rev: 0.2) C, I think.  The app uses the temperature module, accelerometer module, LED module, GPIO module, and does a periodic battery check.

The gpio module is connected to 3 pressure sensors.  When motion is detected, the app activates the voltage on the gpio module and begins reading pressure information (value between 0 and 3000 which I assume are millivolts) for a 10 second period, then deactivates the voltage.  The process occurs again once motion is detected.

Roughly 40-50% of the time, I get the following exception when attempting to create the timers for the gpio board:

Error starting timer task for sensor 1

java.util.concurrent.TimeoutException: Creating a MetaWear timer timed out after 3000ms

                                                              at com.mbientlab.metawear.impl.DefaultMetaWearBoard$TimerImpl$1.run(DefaultMetaWearBoard.java:4969)

                                                              at android.os.Handler.handleCallback(Handler.java:739)

                                                              at android.os.Handler.dispatchMessage(Handler.java:95)

                                                              at android.os.Looper.loop(Looper.java:148)

                                                              at android.app.ActivityThread.main(ActivityThread.java:5417)

                                                              at java.lang.reflect.Method.invoke(Native Method)

                                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)

                                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

Once this exception is thrown, the app must be restarted before the gpio timers will work again.  Between calls that activate the gpio board, I clear the timers using the clearTimers() method on the timer module.  Having said that, there are times the app starts fresh and I get the timeout exception.

What do I need to do to ensure the gpio module and timers will work reliably?

Eric

The removeTimers method only removes timers the API is aware of so if you close the app without saving its state, it won't remove anything next time you call it.  You should remove timers and routes when you are done using them.  

You can also use the debug module to reset the board.

vthompson

Eric

Thank you for your response.  Once motion is detected, the gpio board and timers are activated for a period of 10 seconds.  After that, I turn off the voltage and then call removeTimers.

A few minutes ago I allowed the app to continue running and would occasionally verify, by moving the sensor, the timers were working as expected.  About 30 minutes in, I received the following exception when motion was detected:

Background task reported an error                                                                 java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException 

I'd give you the full stack trace but apparently there's a character limit on the posts.  Almost immediately after this exception I received the timeout exception on a timer.  The app had not been closed, the screen remained on and unlocked, and the app was the only app used during the test.

I don't think I mentioned I am using Android API v2.8.8

Eric

I don't need the entire stack trace but it should contain at least all of the MetaWear API calls.

Based on your specific usage, you should not be constantly removing timers.  Create the timer once and have it execute its task a fixed number of times.  When you need it to execute its task again, call the start method.

vthompson

Can you provide an example of timer reuse?

Eric

When you schedule a task, the AsyncOperation returns a Controller object which is used to control the timer.  Save that controller object so you can restart the timer.

https://mbientlab.com/androiddocs/2/timer.html#controller

vthompson

Thanks.  I've noticed when the timers return the timeout exception, they are done.  In other words, they don't respond and the only recourse I've found is to restart the app or pull the battery from the sensor.  What can I do to smoothly recover and get the gpio/timers going again?

vthompson

java.util.concurrent.TimeoutException: Creating a MetaWear timer timed out after 3000ms at

com.mbientlab.metawear.impl.DefaultMetaWearBoard$TimerImpl$1.run(DefaultMetaWearBoard.java:4969)

at android.os.Handler.handleCallback(Handler.java:739)

at android.os.Handler.dispatchMessage(Handler.java:95)

at android.os.Looper.loop(Looper.java:148)

at android.app.ActivityThread.main(ActivityThread.java:5551)

at java.lang.reflect.Method.invoke(Native Method)

at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:730)

at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:620)

I've removed individual timers (controller.remove()), called teardown(), even debug.reset().  No matter what, once that error occurs on a gpio pin, I cannot read on that pin until the app is restarted.  I really need to get this resolved.

vthompson

I have some improvement to report.  I can now recover from the error by removing the timer (Controller.remove()) and removing the route (RouteManager.remove()), however, I get the timeout error every other time I call start on the controller.  Is there something I need to do after a timer completes?

Eric

Can you post the code that you are currently using?  Please include the setup, clearn up, and error handling code

vthompson

public class StandDetectionModule implements PressureModule.PressureModuleObserver {

final static Byte VOLTAGE_PIN = 3;

private MbientManager _mbientManager;
private ArrayList _standDetectionObservers;
private Short _pressureThreshold;
private Gpio _gpio;
private Timer _mbientTimer;
private PressureModule _toeSensor;
private PressureModule _heel1Sensor;
private PressureModule _heel2Sensor;
private static boolean _isDetecting;
private static StandDetectionModule _instance;

private StandDetectionModule(Short pressureThreshold, MbientManager mbientManager) {
setPressureThreshold(pressureThreshold);
_standDetectionObservers = new ArrayList<>();
_mbientManager = mbientManager;
_isDetecting = false;
}

public static StandDetectionModule getInstance(Short pressureThreshold, MbientManager manager) {
if (_instance == null) {
_instance = new StandDetectionModule(pressureThreshold, manager);
}

return _instance;
}

public synchronized void startDetecting() {

if (_isDetecting) {
return;
}

if (!_mbientManager.isBoardConnected()) {
Log.w(ComponentConstants.APP_TAG, "Could not start detecting stands. The mbient board is not connected");
return;
}

try {
_gpio = _mbientManager.getMetaWearBoard().getModule(Gpio.class);
_mbientTimer = _mbientManager.getMetaWearBoard().getModule(Timer.class);
}
catch (UnsupportedModuleException e) {
Log.e(ComponentConstants.APP_TAG, "GPIO module is unsupported", e);
return;
}

if (_toeSensor == null) {
_toeSensor = new PressureModule(PressureModule.SENSOR_INDEX_TOE, _gpio, _mbientTimer);
_toeSensor.addPressureSensorObserver(this);
}

_toeSensor.startPressureReading();

if (_heel1Sensor == null) {
_heel1Sensor = new PressureModule(PressureModule.SENSOR_INDEX_HEEL_1, _gpio, _mbientTimer);
_heel1Sensor.addPressureSensorObserver(this);
}

_heel1Sensor.startPressureReading();

if (_heel2Sensor == null) {
_heel2Sensor = new PressureModule(PressureModule.SENSOR_INDEX_HEEL_2, _gpio, _mbientTimer);
_heel2Sensor.addPressureSensorObserver(this);
}

_heel2Sensor.startPressureReading();

_gpio.clearDigitalOut(VOLTAGE_PIN);
_isDetecting = true;
}

public synchronized void stopDetecting() {

if (!_isDetecting) {
return;
}

if (_gpio != null) {
_gpio.setDigitalOut(VOLTAGE_PIN);
}

_isDetecting = false;

if (_toeSensor != null) {
_toeSensor.stopPressureReading();
}

if (_heel1Sensor != null) {
_heel1Sensor.stopPressureReading();
}

if (_heel2Sensor != null) {
_heel2Sensor.stopPressureReading();
}

_isDetecting = false;
}

public Short getPressureThreshold() {
return _pressureThreshold;
}

public void setPressureThreshold(Short pressureThreshold) {
_pressureThreshold = pressureThreshold;
}

public void addStandDetectionObserver(StandDetectionObserver observer) {
_standDetectionObservers.add(observer);
}

@Override
public void onPressureRead(Byte sensorIndex, Short data) {
Log.i(ComponentConstants.APP_TAG, "Data read from sensor " + sensorIndex.toString() + ": " +
data.toString());

if (data >= _pressureThreshold) {
stopDetecting();
onStandDetected();
}
}

@Override
public void pressureReadingStarted(Byte sensorIndex) {
onStartDetecting();
}

@Override
public void pressureReadingStopped(Byte sensorIndex) {
onStopDetecting();
}

@Override
public void pressureError(Byte sensorIndex, String message, @Nullable Throwable error) {
// The timer module will on occasion throw a timeout. Sometimes the timer recovers,
// sometimes not. Try to reset and reconnect.
if (error != null && error.getClass().equals(TimeoutException.class)) {

if (sensorIndex == PressureModule.SENSOR_INDEX_TOE) {
_toeSensor = null;
}

if (sensorIndex == PressureModule.SENSOR_INDEX_HEEL_1) {
_heel1Sensor = null;
}

if (sensorIndex == PressureModule.SENSOR_INDEX_HEEL_2) {
_heel2Sensor = null;
}
// Nuclear option
/*
if (_gpio != null) {
_gpio.setDigitalOut(VOLTAGE_PIN);
}

try {
_toeSensor = null;
_heel1Sensor = null;
_heel2Sensor = null;
_isDetecting = false;

Debug debugModule = _mbientManager.getMetaWearBoard().getModule(Debug.class);
debugModule.resetAfterGarbageCollect();
Log.i(ComponentConstants.APP_TAG, "Mbient board reset...");
onError(message, error);
_mbientManager.reconnectToMetaWearBoard();
}
catch (UnsupportedModuleException e) {
Log.e(ComponentConstants.APP_TAG, "Debug module is not supported", e);
}
*/
}
}

private void onStandDetected() {
for (StandDetectionObserver observer : _standDetectionObservers) {
observer.standDetected();
}
}

private void onStartDetecting() {
for (StandDetectionObserver observer : _standDetectionObservers) {
observer.standDetectionStarted();
}
}

private void onStopDetecting() {
for (StandDetectionObserver observer : _standDetectionObservers) {
observer.standDetectionStopped();
}
}

private void onError(String message, @Nullable Throwable error) {
for (StandDetectionObserver observer : _standDetectionObservers) {
observer.sensorError(message, error);
}
}

public interface StandDetectionObserver {
void standDetected();
void standDetectionStarted();
void standDetectionStopped();
void sensorError(String message, @Nullable Throwable error);
}
}

vthompson

I am having trouble posting the other bit.  The forum software is complaining about the length...

vthompson

public class PressureModule {
public static final Byte SENSOR_INDEX_TOE = 0;
public static final Byte SENSOR_INDEX_HEEL_1 = 1;
public static final Byte SENSOR_INDEX_HEEL_2 = 2;

private Gpio _gpioModule;
private Timer _sensorTimer;
private Byte _sensorNumber;
private RouteManager _sensorRouteManager;
private final String _routeTag;
private byte _timerControllerId;
private Timer.Controller _timerController;
private Short _pressureData;

private ArrayList _pressureModuleObservers;

public PressureModule(Byte sensorNumber, Gpio module, Timer timer) {
_pressureModuleObservers = new ArrayList<>();
_sensorNumber = sensorNumber;
_routeTag = "pressure" + _sensorNumber.toString();
_gpioModule = module;
_sensorTimer = timer;
}

public void addPressureSensorObserver(PressureModuleObserver observer) {
_pressureModuleObservers.add(observer);
}

protected void onPressureRead(Byte sensorIndex, Short data) {
for (PressureModuleObserver observer : _pressureModuleObservers) {
observer.onPressureRead(sensorIndex, data);
}
}

protected void onStartReading(Byte sensorIndex) {
for (PressureModuleObserver observer : _pressureModuleObservers) {
observer.pressureReadingStarted(sensorIndex);
}
}

protected void onStopReading(Byte sensorIndex) {
for (PressureModuleObserver observer : _pressureModuleObservers) {
observer.pressureReadingStopped(sensorIndex);
}
}

protected void onPressureError(Byte sensorIndex, String message, @Nullable Throwable error) {
for (PressureModuleObserver observer : _pressureModuleObservers) {
observer.pressureError(_sensorNumber, message, error);
}
}

public synchronized void startPressureReading() {

Log.d(ComponentConstants.APP_TAG, "Starting to read pressure on sensor " + _sensorNumber.toString());

if (_timerController != null && !_timerController.isActive()) {
_timerController.start();
onStartReading(_sensorNumber);
return;
}

_gpioModule.routeData().fromAnalogIn(_sensorNumber, Gpio.AnalogReadMode.ABS_REFERENCE)
.stream(_routeTag)
.commit().onComplete(new AsyncOperation.CompletionHandler() {

@Override
public void success(RouteManager result) {
super.success(result);

_sensorRouteManager = result;
_sensorRouteManager.subscribe(_routeTag, _dataHandler);

AsyncOperation taskResult = _sensorTimer.scheduleTask(new Timer.Task() {
@Override
public void commands() {
_gpioModule.readAnalogIn(_sensorNumber, Gpio.AnalogReadMode.ABS_REFERENCE);
}
}, 1000, false, (short) 10);

taskResult.onComplete(new AsyncOperation.CompletionHandler() {
@Override
public void success(Timer.Controller controller) {
super.success(controller);
controller.start();
_timerController = controller;
onStartReading(_sensorNumber);
Log.d(ComponentConstants.APP_TAG, "Successfully started pressure monitoring for sensor " + _sensorNumber.toString());
}

@Override
public void failure(Throwable error) {
super.failure(error);
Log.e(ComponentConstants.APP_TAG, "Error starting timer task for sensor " +
_sensorNumber.toString(), error);

if (_timerController != null) {
_timerController.remove();
}

if (_sensorRouteManager != null) {
_sensorRouteManager.unsubscribe(_routeTag);
_sensorRouteManager.remove();
}

_sensorRouteManager = null;
_timerController = null;
onPressureError(_sensorNumber, "Error starting mbient timer task", error);
}
});
}

@Override
public void failure(Throwable error) {
super.failure(error);

Log.e(ComponentConstants.APP_TAG, "Error setting up pressure reading for sensor " +
_sensorNumber.toString(), error);
}
});
}

public synchronized void stopPressureReading() {

if (_timerController != null) {
_timerController.stop();

onStopReading(_sensorNumber);
Log.d(ComponentConstants.APP_TAG, "Stopped pressure monitoring on sensor " + _sensorNumber.toString());
return;
}

Log.w(ComponentConstants.APP_TAG, "Could not stop pressure module " + _sensorNumber.toString() +
". The timer controller is null");
}

private final RouteManager.MessageHandler _dataHandler = new RouteManager.MessageHandler() {
@Override
public void process(Message message) {
Short data = message.getData(Short.class);

if (!data.equals(_pressureData)) {
_pressureData = data;
onPressureRead(_sensorNumber, data);
}
}
};

public interface PressureModuleObserver {
void onPressureRead(Byte sensorIndex, Short data);
void pressureReadingStarted(Byte sensorIndex);
void pressureReadingStopped(Byte sensorIndex);
void pressureError(Byte sensorIndex, String message, @Nullable Throwable error);
}
}

vthompson

The first bit is a controller of sorts. The second bit is a container for a sensor/gpio pin.

Eric

Seems like you have an unwanted negation operator when checking if the controller is active in the PressureModule class.

  if (_timerController != null && !_timerController.isActive()) {

   _timerController.start();

   onStartReading(_sensorNumber);

   return;

  }

vthompson

What does it mean when a controller is in an Active state?  I mistakenly assumed this meant it was currently executing a timer's task.

vthompson

Further testing following the change recommended (isActive)...

The gpio board worked flawlessly for about 20 minutes.  When movement was detected the attached sensors were activated and data flowed as expected.  I committed the code, made a change that renames the MetaWear board, then ran via the debugger.  Each pin on the gpio board times out.  I've restarted the app several times.  A battery pull is about the only thing I know to do to get the board working again.  Once I do the battery pull the gpio board works as expected.

For what it's worth, whenever the board connects I call the teardown method which I hope clears timers and routes.

Eric

Active means the controller object can still be used.  It's only used with the remove method so you don't accidentally use an object corresponding to a removed timer.

Teardown, in api v2, only works with timers it is aware of and is intended for cleanup after you are done with your board.  Use the debug module to reset the board but you should only do that if you can't schedule a task.

vthompson

So, if I get the timeout error the way to overcome it is to debug.reset the board?

vthompson

Does v3 of the api have a less "nuclear" option for instances when a task cannot be scheduled?

Eric

Timeout exceptions most commonly occur when there are no more resources available to be allocated on the board.  Make sure you release all resources by calling tearDown before you reschedule tasks / setup more routes.

You can try the tearDown function in api v3 which functions similar in v2 but will blindly clear all timers.