Unexpected exception with time processor

robin · January 2022

Hello all,

I’m new to using MbientLab sensors and am running into an issue where I’m wondering what I’m doing wrong. I am on macOS with the Swift API, and would like to record fusion sensor values at a low frequency for a while, like a few per minute; and then later download the recorded data. For that, I’m combining the fusion signal with a time data processor. But after downloading, I’m getting exceptions:

libc++abi: terminating with uncaught exception of type std::out_of_range: unordered_map::at: key not found

Backtrace:

#0  0x0000000105b6a0f8 in std::out_of_range::out_of_range(char const*) at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/stdexcept:165
#1  0x0000000105b69c24 in std::__1::__throw_out_of_range(char const*) at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/stdexcept:267
#2  0x0000000105b5a730 in std::__1::unordered_map<ResponseHeader, MblMwEvent*, std::__1::hash<ResponseHeader>, std::__1::equal_to<ResponseHeader>, std::__1::allocator<std::__1::pair<ResponseHeader const, MblMwEvent*> > >::at(ResponseHeader const&) at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/unordered_map:1789
#3  0x0000000105bcd6b4 in guessLogSource(MblMwMetaWearBoard*, ResponseHeader&, unsigned char, unsigned char) at /project/Pods/MetaWear/MetaWear/MetaWear-SDK-Cpp/src/metawear/core/cpp/logging.cpp:155
#4  0x0000000105bccecc in processor_synced(MblMwMetaWearBoard*, std::__1::stack<ProcessorEntry, std::__1::deque<ProcessorEntry, std::__1::allocator<ProcessorEntry> > >&) at /project/Pods/MetaWear/MetaWear/MetaWear-SDK-Cpp/src/metawear/core/cpp/logging.cpp:276
#5  0x0000000105b5b89c in dataprocessor_config_received(MblMwMetaWearBoard*, unsigned char const*, unsigned char) at /project/Pods/MetaWear/MetaWear/MetaWear-SDK-Cpp/src/metawear/processor/cpp/dataprocessor.cpp:522
#6  0x0000000105c1e798 in char_changed_handler(void const*, unsigned char const*, unsigned char) at /project/Pods/MetaWear/MetaWear/MetaWear-SDK-Cpp/src/metawear/impl/cpp/metawearboard.cpp:431
#7  0x0000000105cd1708 in closure #2 in MetaWear.peripheral(_:didUpdateValueFor:error:) at /project/Pods/MetaWear/MetaWear/Core/MetaWear.swift:525
[...]

[...]

Below’s the main part of the code, stripped down & simplified a bit to trigger the problem directly. record_test() begins recording for 10s, then starts the download. The download() function still completes, but then the exception occurs immediately afterwards. If I skip the time processor and use the fusion signal directly instead, it works just fine. I am wondering what I’m doing wrong?

func record_test() {
    record_begin()

    DispatchQueue.main.asyncAfter(deadline: .now() + 10) {
        record_end()
        download()
    }
}

func record_begin() {
    print("recording - begin")

    let accelRange = MBL_MW_SENSOR_FUSION_ACC_RANGE_16G
    let gyroRange = MBL_MW_SENSOR_FUSION_GYRO_RANGE_2000DPS
    let sensorFusionMode = MBL_MW_SENSOR_FUSION_MODE_NDOF

    let device = devices.connected!.meta!

    mbl_mw_sensor_fusion_set_acc_range(device.board, accelRange)
    mbl_mw_sensor_fusion_set_gyro_range(device.board, gyroRange)
    mbl_mw_sensor_fusion_set_mode(device.board, sensorFusionMode)

    mbl_mw_sensor_fusion_write_config(device.board)

    let signal = mbl_mw_sensor_fusion_get_data_signal(device.board, MBL_MW_SENSOR_FUSION_DATA_QUATERNION)!

    mbl_mw_dataprocessor_time_create(signal, MBL_MW_TIME_ABSOLUTE, 1000, bridgeRetained(obj: device)) { (context, signal) in 
        let device: MetaWear = bridgeTransfer(ptr: context!)

        mbl_mw_datasignal_log(signal, nil) { (context, logger) in
            let id = mbl_mw_logger_generate_identifier(logger)!
            print("identifier for logger: ", id as Any)
        }

        mbl_mw_logging_start(device.board, 1)

        mbl_mw_sensor_fusion_clear_enabled_mask(device.board)
        mbl_mw_sensor_fusion_enable_data(device.board, MBL_MW_SENSOR_FUSION_DATA_QUATERNION)
        mbl_mw_sensor_fusion_start(device.board)
    }
} 

func record_end() {
    print("recording - end")

    let device = devices.connected!.meta!

    mbl_mw_logging_stop(device.board)
    mbl_mw_sensor_fusion_stop(device.board)
    mbl_mw_sensor_fusion_clear_enabled_mask(device.board)
}

func download() {
    print("download - begin")

    let device = devices.connected!.meta!

    device.createAnonymousDatasignals().continueWith(.mainThread) { t in
        if let signals = t.result {
            for signal in signals {
                print("- signal ", signal)

                mbl_mw_anonymous_datasignal_subscribe(signal, nil) { (context, data) in
                    print("data")
                }

                devices.download_handlers.received_progress_update = { (context, remainingEntries, totalEntries) in
                    print(remainingEntries, " of ", totalEntries)
                }

                devices.download_handlers.received_unknown_entry = { (context, id, epoch, data, length) in
                    print("received_unknown_entry")
                }

                devices.download_handlers.received_unhandled_entry = { (context, data) in
                    print("received_unhandled_entry")
                }

                mbl_mw_logging_download(device.board, 10, &devices.download_handlers)
            }
        }
    }

    print("download - end")
}

ryanferrell · January 2022

@robin

Thanks for providing simplified code, a clear explanation, and the backtrace. Super helpful.

First off, this is a bug in the C++ library. We will push an updated C++ library shortly. A quick fix replacement file is attached to get you moving before then.

The fix is adding next->remove = false; to line 281 of MetaWearCpp/src/metawear/core/cpp/logging.cpp When collecting anonymous data signals prior to a download, the data processor gets removed. You can breakpoint line 382 of dataprocessor.cpp to watch the crime in progress.

Second, a couple tips.

For createAnonymousDatasignals().continueWith(.mainThread), you’ve got a potential thread safety issue. Use the apiAccessQueue, which the MetaWear exposes, to stay on the C++ library’s serial queue.

For the simplified example code you’re using bridgeRetained(obj:) against the MetaWear instance. In your app’s code on the callback you’re probably using bridge(obj:) and bridge(ptr:) against the concrete class that contains these methods, I assume, but just wanted to check.

Finally, in future crash reports, if you set the MetaWear’s logDelegate to use the ConsoleLogger.shared , we can see the actual bytes sent back and forth. This lets us pinpoint exactly what’s going on. In this case, we saw a logger deletion call, which led us to the bug.

ryanferrell · January 2022

@robin

If you have any interest in a beta Combine SDK, today or tomorrow we'll push a version with sugar around some data processing. Example:

metawear
    .publishWhenConnected()
    .log(.sensorFusionEulerAngles(mode: .ndof), withProcessing: { mutableSignal in
        mutableSignal.throttle(rate: .hz1)
    })
    .delay(for: 10, tolerance: 0, scheduler: metawear.bleQueue)
    .downloadLogs(startDate: mockCachedDate)
    .drop(while: { $0.percentComplete < 1 })
    .map(\.data)
    ._sinkNoFailure(&subs, finished: { }) { dataTables in
        let rowCount = dataTables.first?.rows.endIndex ?? 0
        XCTAssertEqual(rowCount, 10, accuracy: 2)
        exp.fulfill()
    }

robin · January 2022

@ryanferrell

Awesome, I just did a quick test with your change and indeed, it works now. Many thanks for looking into it and the quick fix.

Thanks for your tips as well, appreciate it.

I did notice the Combine SDK the other day, that looks pretty neat, I'll definitely give that a try.

Thanks,

Robin

robin · February 2022

@ryanferrell

I'm playing with the Combine SDK now, that's definitely much nicer! But I can't get the throttling to work. It looks like withProcessing has been renamed to afterProcessing, but the following code doesn't build for me:

metawear.
    .publish()
    .log(.sensorFusionEulerAngles(mode: .ndof), afterProcessing: { signal in
        signal.throttle(rate:. hz1)
...
})

The error message suggests that it's resolving the call to throttle to Combine.Publisher.throttle(for:scheduler:latest:) instead of the one for the data signal. I've pinned the MetaWear package to commit 6a0e7d9fe742 at the moment, so the new code should be available. Any idea what I'm missing? (Btw, I can't get current main to build: mbl_mw_debug_spoof_button_event is reported as unknown).

I got a 2nd question as well actually. I'd like to start/stop logging by pressing the button: 1st press starts logging, 2nd stops logging. I can get the 1st one to work through a recorded event, but how would I go about switching the behavior of the next button press to now stop logging?

Thanks,

Robin

ryanferrell · February 2022

@robin

Glad you're having some fun!

Does the new head commit — 33e689ec tag 0.5.2 — resolve the throttle and build issues mentioned? It renames throttle to throttleData. The build issue was the C++ SDK mainlined a new functionality yesterday and I hadn't yet set the repo's public C++ submodule.

FYI — The download methods are about to change to be more flexible with timing, post-processing, and tolerance to unexpected data types that might result from data processors that change from the sensor output type to some other type (e.g., simd_quatf to Int). At the moment you'd have to manually mix in some C or fill a dictionary with the exact logger identifier string. Yuck. We're super open to feedback to make this SDK more compassionate and straightforward, both in code and in documentation. If you have feedback, even if it's just a personal aesthetic, feel free to PM.

Regarding starting and stopping logging, do you mind describing your exact use case? This is a very useful functionality, so we'll expose a preset like .recordEvents(.buttonDownOdds) { ... } sometime next week.

Documentation is needed for custom event creation and recording. In short, the data processor chain should be something like:

// upstream AnyPublisher<MetaWear,MWError>
  .getLoggerMutablePointer(.mechanicalButton)
  .math(.modulus, rhs: 2, signed: false)
  .filter(.equals, reference: 0)
// return AnyPublisher<MWDataProcessorSignal,MWError>

To pair that with event recording, let's look at the existing SDK code a bit.

The recordEvents API accepts an enum option, which maps to a pipeline that asynchronously creates a data processed logger signal, just like the snippet above.

recordEvents then zips this pipeline to publish an Output that is a tuple (MetaWear, OpaquePointer). The latter is the C++ data processor signal struct whose type information is not exposed to Swift. For clarity, it is type aliased as MWDataProcessorSignal.

Once that data processor pointer arrives, the next call is what you'd need to use... a much less discoverable API matched to that (MetaWear,OpaquePointer) tuple. It's got the same closure for recording commands (e.g., .loggersPause if you don't mind keeping the sensors warm and using some battery).

That mbl_mw_debug_spoof_button_event function is super useful in this context.

In a work-in-progress branch of Streamy, the button is made to:
— lazily start loggers on first button down (not immediately when the device is programmed)
— shine the LED red while the button is depressed
— later, after downloading, reading button down/up times splits CSV files into separate "trial runs"

What that code doesn't do is loggersPause because, once paused, the .loggersStart command would come too late to record that button event... which is the signpost needed for CSV splitting. Grrr....

The fix for this conundrum is the new spoof function .command(.logUserEvent(flag:). It writes a UInt8 of choice to the button's logger (if active). This will let the next iteration of Streamy pause and restart loggers to reduce log size and download times while recording trial runs. (FYI, the ML models in that branch are also not properly trained, hence tutorial is not published.)

Annoyance: If you're starting/stopping the magnetometer, its hardware is a bit of a weekend teenager. It has a slow-wake up behavior that can result in "start" commands being missed without some troubleshooting. One workaround is, oddly, telling it to "stop" before "starting" as in MWMagnetometer/streamSignal.

ryanferrell · February 2022

@robin

Re throttling, see if these two test cases work for you.

To run the tests, launch the SwiftCombineSDKTestHost.xcproj. So tests can be run automatically against specific local devices with specific sensors, they are constrained to UUID identifiers that won't exist on your machine.

Add your machine's MetaWear UUIDs to TestDevices. Then set the hostMachine to you. If you're one of those people who do not have your MetaWear's CBPeripheralIdentifier UUID memorized, you can just run the Test Host app without a test and click on one of the devices listed. It'll turn blue and its UUID will be copied to the clipboard.

robin · February 2022

@ryanferrell

Thanks a lot for the extensive response. I'll reply more later, but I just did a quick check with my code updated to use 0.5.2 and it still can't get to throttleData(); says'throttleData' is inaccessible due to 'internal' protection level.

More later,

Robin

robin · February 2022

@ryanferrell

To follow up further on this, I made throttleData() public in a local clone of the repository, and that solves the issue. Now throttling works great. I'm now also using current main, and that's likewise working well.

I'll make sure to provide further feedback as I notice things. Part of the challenge here is that I'm new not only to the MetaWear ecosystem, but also to both Swift and SwiftUI, so sometimes it's still a bit hard for me to say where I just don't fully understand things yet vs. where, say, an API change would make things easier.

| Regarding starting and stopping logging, do you mind describing your exact use case?

I'd like to track the orientation of the device offline over extended periods of time (hours), activated and stopped through the button. So my app would set up logging, but not start it immediately. I'd install macros so that one button press starts logging, and a second press (hours later) stops it, with some different LED colors to indicate what's going on. This can be repeated as often as desired. When the app later reconnects to the device, it can download everything recorded in the meantime. While logging is stopped, it'd be great to preserve battery as well, so ideally the sensors would be fully turned off to during that time.

I've got this working now for the first part: set up logging ahead of time, start by button press. For now, I only stop logging when reconnecting & downloading. Nice though on using the modulus to distinguish
button presses; I'll give that a try, thanks for the background there. If you guys could provide that buttonDownOdds... out of the box, that would be cool.

Good to know about that weekend teenager!

Thanks,

Robin

Unexpected exception with time processor

Comments