›

Request for new data processors

minmaxminmax
in Requests & Bug Reports
Hi,

I'd like to propose two new data processors which I think will be useful but are cumbersome to implement with the current DSL.

1. Buffer (or delay) filter
     - When a buffer in an Empty state receives an upstream value, it just stores the data token, changes it's state to Non-Empty and doesn't emit anything to the next processor in the route.
     - When a buffer in a non-empty state receives an upstream value, it yields the previously stored value downstream, and stores the new value.
     - Users can use .setState(..) to control Empty/Non-Empty and stored value


2. Const transformer
    Given an array of constants, the processor ignores the input data and emits the array values upstream - one by one in a cycle
    For example:
       .process(new Const(new byte[]{1}))    will always emit 1 upstream
       .process(new Const(new float[]{1,2,3,4}))   will emit 1,2,3,4,1,...


Thanks

Comments

  • EricEric
    Some offhand thoughts about the proposed filters:
    1. The sample filter (erroneously described in the Android docs, need to fix that) partially does what you are looking for.  It will not allow data through until it has received X samples however, there currently is no way to empty then bin to reset back to empty state.  You could probably leverage the passthrough filter in conditional mode to function as the empty/non-empty controller.
    2. You can use the counter and and math processor to create this effect.  Something along the lines of: counter -> math mod X.
  • minmaxminmax
    Thanks for the hint about Sample. From the docs it looks like it just downsamples the signal.
    So,  Sample(1) does exactly what I wanted for buffer.
    On a side note, hooking Sample directly to a log    .process( new Sample((byte) 1)).log("LOG")  raises exception when downloading the logs. http://pastebin.com/JzTGeX2Q

    Regarding the second one, I wanted it to be useful for directly defining convolution kernels as an array of float values. As you suggested, it can now be implemented as counter -> mod N -> comparison branches, but it doesn't scale well.
  • EricEric
    What is the full code for your logged sample processor?  I logged a sample processor and downloaded it without running into any problems.

    // Using a MetaWear RPro
    multiTempModule.routeData().fromSource(multiTempModule.getSources().get(1)).process(new Sample((byte) 1)).log("V").commit()

    It's possible that either the board was not in a clean state or the app was was not reset when you added the sample processor.  The out of bounds error suggests that there is some conflict between what the API thinks the log data should be versus the byte array it is actually receiving.
  • minmaxminmax
    Yes, it doesn't look like it's caused simply by doing  .process (new Sample).log(..)
    Here are the things I've tried http://pastebin.com/Ct4KrJHM in case it can be reproduced.
    Might as well be specific to my app/board 

  • MattMatt
    @minmax

    I'm looking into the possibilities for enabling these functionalities in firmware.

    Item 1:
    Eric and yourself have already discussed how this is possible with the Sample/Delay filter.  It handles the general case of an n-deep delay.  We will dig deeper to confirm or add the mechanism to reset the delay state to empty or possibly more generally "n-captured".

    Example use case: Log 50 Accelerometer samples before and 50 after a high-g event. Solution: Delay accelerometer output by 50 entries, feed it into a passthrough filter setup to pass 100 entries, then log it.  The high-g event will flip the "pass 100" switch on the pass through filter.

    Item 2:
    • Case 1, Always emit a constant.
      • To always emit "1", the comparison filter would work for this now.
      • The general case, emit "x", I propose adding a constant output mode to the simplemath filter.
    • Case 2, Emit a sequence of outputs.
      • For the sequential case, we had considered adding the "count" to the timer notifications.  This would cover upcounter and downcounter use cases on time delays.
      • For the arbitrary sequence case, I propose adding a "Sequencer" data processor. It could have a mode for sequential runs, and a mode to output arbitrary sequences.
    Regarding your end use case, convolution kernels, we have in the past considered adding a general purpose FIR/IIR DSP style filter which would take a coefficient array/filter kernel as a parameter.  We should start up a new thread to discuss requirements and capabilities.

    The simple math and sequencer filters are something we can intercept for the upcoming firmware release r1.1.0.  The DSP filter will require quite a bit more discussion and thought and will have to be addressed in the future.
  • minmaxminmax
    Hi,

    I included the emit constant example just as a trivial case for the proposed transformer. If you implement the Sequencer DP for arbitrary sequences, it will cover this functionality, so no need to duplicate it in Maths, unless it supports modification of the constant token.
    a use case:
    emit(1) -> Accumulator    will behave like Counter
    but if at some point you change 1 to -1, it will start counting backwards form the current count

    What do you suggest regarding the sequential mode of Sequencer?
    The general case can be handled by Counter, as Eric pointed out. 

    Thanks
  • EricEric
    @minmax

    The cause of the exceptions is most likely due to reusing processor and logger keys.  Each process and logger key must be unique.
  • minmaxminmax
    If you can't reproduce it then it's not an issue.
This discussion has been closed.