Python client using C++ API

hbldh

Hello,

I just wanted to make you aware of the Python package that I uploaded to Github today: https://github.com/hbldh/pymetawear

It is a Metawear communication client written in Python, using the C++ API library. 

It is in a early stage of development and testing, but I have gotten it to connect and blink LEDs on both R Pro and C boards; in my book a decent state after the two days I have hitherto dedicated to it. I have two projects that I want to use the MetaWear boards for so I think I will continue to develop it for some time to come.

Eric

This is sweet.  There have been people wanting to use Python so this will be very helpful for everyone.  If you need any more help with the C++ library, let me know.

hbldh

Oh, I've got some questions, but I need to get the basic ble communication working nicely first before it is reasonable to ask anything!

hbldh

@Eric: After some programming I have now come to trying to subscribe to data signals, but I experience segfaults at some 

points. Last time I asked, it seemed to be because the board had not initialized properly. Now though, I wait for the initialization callback method to be called, and after that I try to subscribe. 

Subscribing to switch notifications works fine, I get the correct responses when pressing switch button.

Regarding battery state and accelerometer data I instead get the following error when running my scipts in gdb:

Program received signal SIGSEGV, Segmentation fault.

0x00007ffff6071a94 in mbl_mw_datasignal_subscribe ()

   from /home/hbldh/Repos/pymetawear/pymetawear/libmetawear.so

Battery code: https://github.com/hbldh/pymetawear/blob/develop/examples/battery.py#L35-L47

Accelerometer code: https://github.com/hbldh/pymetawear/blob/develop/examples/accelerometer.py#L34-L58

Do you have any idea why this happens?

Eric

What is the address of acc_signal and battery_signal?  If my hunch is correct, then they are set to NULL which means something went wrong with the initialization.

hbldh

They are not NULL or None, but I get different values every time I run the script:

battery_signal = libmetawear.mbl_mw_settings_get_battery_state_data_signal(c.board)

battery_signal: -603953344, 1744856896, -1342150848, ...

acc_signal = libmetawear.mbl_mw_acc_get_acceleration_data_signal(c.board)

acc_signal: -1207943488, ...

Seems like a get an uninitialized value for libmetawear. 

When I turn on my logging, which prints every read and write performed I have this:

Discovering nearby MetaWear boards...

Creating new GATTRequester...

Connecting GATTRequester...

Subscribe 0x001f: 

Write  0x001d: 01 80

Read   0x0014: 32

Read   0x001a: 31 2e 32 2e 31

Waiting for MetaWear board to be fully initialized...

Notify 0x001f: 01 80 00 00

Write  0x001d: 02 80

Notify 0x001f: 02 80 00 00

Write  0x001d: 03 80

Notify 0x001f: 03 80 01 00

Write  0x001d: 04 80

Notify 0x001f: 04 80 01 00 00 03 01 02

Write  0x001d: 05 80

Notify 0x001f: 05 80 00 01 03 03 03 03 01

Write  0x001d: 06 80

Notify 0x001f: 06 80 00 00

Write  0x001d: 07 80

Notify 0x001f: 07 80 00 00

Write  0x001d: 08 80

Notify 0x001f: 08 80 00 00

Write  0x001d: 09 80

Notify 0x001f: 09 80 00 00 1c

Write  0x001d: 0a 80

Notify 0x001f: 0a 80 00 00 1c

Write  0x001d: 0b 80

Notify 0x001f: 0b 80 00 02 08 80 2b 00 00

Write  0x001d: 0c 80

Notify 0x001f: 0c 80 00 00 08

Write  0x001d: 0d 80

Notify 0x001f: 0d 80 00 00

Write  0x001d: 0f 80

Notify 0x001f: 0f 80 00 01 08

Write  0x001d: 10 80

Notify 0x001f: 10 80

Write  0x001d: 11 80

Notify 0x001f: 11 80 00 03

Write  0x001d: 12 80

Notify 0x001f: 12 80

Write  0x001d: 13 80

Notify 0x001f: 13 80 00 00

Write  0x001d: 14 80

Notify 0x001f: 14 80

Write  0x001d: 15 80

Notify 0x001f: 15 80

Write  0x001d: 16 80

Notify 0x001f: 16 80

Write  0x001d: 17 80

Notify 0x001f: 17 80

Write  0x001d: 18 80

Notify 0x001f: 18 80

Write  0x001d: fe 80

Notify 0x001f: fe 80 00 00

Write  0x001d: 0b 84

Notify 0x001f: 0b 84 cc 0e 00 00 07

PyBluezBackend, DD:3A:7D:4D:56:F0 initialized.  <--- Initialization callback

New client created: MetaWearClient, DD:3A:7D:4D:56:F0

Getting battery state data signal...

<type 'int'> -671062208

Subscribing to battery state...

Process finished with exit code 139

Don't know if it is useful here, but I can at least provide it in case.

Eric

The write/notifies from your log are correct; that is the expected write/notify exchanges from initialization.  Since you're not getting a null pointer when requesting a data signal, then it would appear that everything is working fine.

The datasignal_subscribe function itself is only 2 lines long (https://github.com/mbientlab/Metawear-CppAPI/blob/0.4.15/src/metawear/core/cpp/datasignal.cpp#L57).  Can you try inserting some couts to see which of the two lines of code causes the segfault?

You say that switch notification is working fine so my question then is what's different between the working switch code and the broken accelerometer/battery?

hbldh

Added a printout before both rows and it segfaults on the first one. Added this:

std::cout << "Before emplace: " << signal << std::endl;

which gave the following printout in my code:

Getting battery state data signal...
<type 'int'> -1409266112
Subscribing to battery state...
Before emplace: 0xffffffffac004e40

The integer printed out is the value returned by mbl_mw_settings_get_battery_state_data_signal
As you can see, it is the same printed out in the C code as well in two's complement. When I tried to access signal -> owner for printing, it chrashed already in the printing step.

I use the same way of calling both switch and accelerometer/battery subscriptions. Should I use something different than the FnDataPtr for the callback method in the battery/accelerometer case?

Eric

Hrm...an invalid pointer is not a good sign.  I have your project clone on an Ubuntu 15.04 VM with gcc 4.9.2 and I will try to see why the memory address is corrupted in your script but not from the unit test.

Eric

I've addressed the seg fault in your code; please see my pull request.

hbldh

@eric: What is the proper way to disconnect from a MetaWear board using the C++ API? Is it sufficient to use
mbl_mw_metawearboard_tear_down(board) mbl_mw_metawearboard_free(board)
Is there any need for the Debug module's mbl_mw_debug_disconnect function in this context? When should that be used?

A related question:
Do I have to manually unsubscribe from all notifications prior to disconnect, or will these be removed by itself. In the mbl_mw_metawearboard_tear_down functon I see removal of DataProcessors, Timers, Events, Logging but nothing about notifications. Is that done here? Can I use the mbl_mw_debug_reset to accomplish that in worst cases?

Eric

Bluetooth LE communication is not managed by the API; connection management needs to be handled by your underlying btle library.  mbl_mw_debug_disconnect is not to be used in this case.

The mbl_mw_metawearboard_tear_down and mbl_mw_metawearboard_free functions are only needed if your app is a long running process that will be interacting with many boards and needs some memory management.  If it's just a simple script that issues a few commands, you won't need either of them i.e. a script that turn son the LED.

The way everything is setup now, you will need to manually unsubscribe from everything though I suppose that can be handled in tear down as well.  mbl_mw_debug_reset only issues the reset command; it does not modify the API state. Functions in the debug header file are meant for helping you debug / develop on your board and are not meant to be used in production code.

fraguada

@hbldh,
Do you know if this should be able to be installed on Debian Jessie?  I'm on a Raspberry Pi 3 and attempted to install it, but got some errors (apologies, away from the pi at the moment, so cannot post the errors) after the gatttools setup.py process.

I am attempting to add this to a python 3 program which uses OpenCV and the Pi Camera.

Thanks,
Luis

hbldh

@fraguada, I have recently been solving that very issue! The Github issue #7 (https://github.com/hbldh/pymetawear/issues/7) of the PyMetWear repo deals with Raspberry Pi 2/3 instalaltions and it is working now, albeit only in the development branch. Checkout the develop branch and install that until I have it merged to the master branch, which I will do this month, as soon as I have finished up some other issues as well.

fraguada

Great!  Always love being on the bleeding edge!  I'm able to build and install from the develop branch, but I get this error uppon running the battery.py example:

python battery.py
Discovering nearby Bluetooth Low Energy devices...
Traceback (most recent call last):
  File "battery.py", line 22, in <module>
    address = scan_and_select_le_device()
  File "/home/pi/pymetawear/examples/discover.py", line 26, in scan_and_select_le_device
    ble_devices = discover_devices(timeout=timeout)
  File "/home/pi/.virtualenvs/cv/local/lib/python3.4/site-packages/pymetawear-0.5.0.dev3-py3.4.egg/pymetawear/client.py", line 74, in discover_devices
    raise PyMetaWearException("Missing capabilites for hcitool!")
pymetawear.exceptions.PyMetaWearException: Missing capabilites for hcitool!

Seems I'm missing something in hcitool?

hbldh

Read the docstring here: https://github.com/hbldh/pymetawear/blob/master/pymetawear/client.py#L29-L50

It will remove the need for sudo when doing hcitool lescan, making it possible to call from PyMetaWear.

fraguada

Thanks a lot.  This works!  Thanks for putting this together.  Any pointers for how to send values to the GPIO?  I have a string of 10 or so Neopixels that I'd like to feed colors to.

fraguada

If I'm reading this correctly, the 'develop' branch uses the libmetawear Metawear C++ API as a submodule and targets what seems to be a fairly recent commit (0b2a3ad).
In this commit, there is a functions.py file that seems to include several methods for working with neopixels such as this and this.  In theory I should be able to use these to control the neopixels.  I've written a small python script and it seems communication is fine.  Next, I'll hook up some pixels!

hbldh

Yes the pymetawear is only a wrapper around the MetaWear C++ api, providing a more pythonic interface to it. Check out the test code for neopixels (https://github.com/mbientlab/Metawear-CppAPI/blob/master/test/testneopixel.py) to get something to work. You can import the libmetawear shared library as 'from pymetawear import libmetawear' and then directly call methods you found in functions.py.

fraguada

Works like a charm!  My main issues were knowing the ordering and speed of my particular neopixels (5mm diffused)...the metawear android app helped me to mess with the parameters...these are slow, RGB.  The code for a strand with one pixel would be like this:


from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from __future__ import absolute_import

from ctypes import byref
from pymetawear import libmetawear
from pymetawear.client import MetaWearClient

#just using one metawear
c = MetaWearClient('XX:XX:XX:XX:XX:XX, 'pygatt', debug=True)
print("New client created: {0}".format(c))

#args board, strand, pin, #pixels, color order
libmetawear.mbl_mw_neopixel_init_slow_strand(c.board, 0, 0, 1, 0)

#args board, strand, pixel id, r, g, b
libmetawear.mbl_mw_neopixel_set_color(c.board, 0, 0, 100, 0, 100)

print('about to disconnect')
c.disconnect()

hbldh

The PyMetaWear Python package, using the MetaWear-CppAPI version 0.5.22 is now available on PyPI: https://pypi.python.org/pypi/pymetawear/

This means that it can finally be installed by

pip install pymetawear

jdjones

I've been trying to figure out how the example script log_acc.py works, and I'm confused by line 76:

libmetawear.mbl_mw_datasignal_log(c.accelerometer.data_signal, cartesian_float_data, logger_ready)

The way this function is called above doesn't seem to match the C++ API definition:

void mbl_mw_datasignal_log (MblMwDataSignal *signal, MblMwFnDataLoggerPtr logger_ready)

What's going on here? Am I missing something?

J.

hbldh

The log_acc.py script should never have made it into the release branch, my bad. I never got it to work properly and I think the example there is from an old C++ API version.

For the purposes I needed the lib for, I was content with streaming the data to my computer and save it continually to an array (by altering the acc_callback in the accelerometer.py example file) so I never got around to finish the logging solution. I will get around to it eventually, or if you want to contribute you are welcome to implement it!

jdjones

Still working on logging data. When the mbientlab python unit tests for logging call mbl_mw_datasignal_log, the second argument (logger_ready) is of type Fn_VoidPtr. I noticed that I get a ctype error if I pass an argument of type Fn_VoidPtr, and the error is resolved if I pass an argument of type Fn_DataPtr. IE the example I mentioned earlier seems to be correct somehow. I installed pymetawear from the PyPI repository. Is it possible that the installation process links to an old version of the C++ API?

Eric

As far as I can tell, the pymetawear package is using the correct C++ API release.  What does the functions.py file in your site-packages folder define for mbl_mw_datasignal_log?

jdjones

functions.py defines mbl_mw_datasignal_log in agreement with the API:

libmetawear.mbl_mw_datasignal_log.restype= None

libmetawear.mbl_mw_datasignal_log.argtypes= [c_void_p, Fn_VoidPtr]

hbldh

Sorry for not responding sooner; have had a lot to take care of elsewhere.

I did get the logging to work somewhat with that script, but I seem to recall that the downloading of data in the logger_ready callback got stuck somehow. I stopped the attempt when I realized that for the purposes that I needed it for right then it was better to just stream the data and store it on the computer during the recording session.

@jdjones How does your test code look like now?

jdjones

Hi all (especially @hbldh),

I found the bug that was causing this error. Part of the __init__ file in the pymetawear root directory is out of date with the current metawear API. Specifically it's this section about 64-bit python compatibility:

# Alleviating Segfault causing pointer errors in 64-bit Python.

if IS_64_BIT:

    libmetawear.mbl_mw_datasignal_subscribe.argtypes = [c_longlong, Fn_DataPtr]

    libmetawear.mbl_mw_datasignal_unsubscribe.argtypes = [c_longlong, ]

    libmetawear.mbl_mw_datasignal_log.argtypes = [c_longlong, Fn_DataPtr]

    libmetawear.mbl_mw_datasignal_read.argtypes = [c_longlong]

Eric

Is that IS_64_BIT section necessary when using the python wrappers provided in v0.5.0 and later?  The functions.py file assigns all pointer types with the c_void_p which should also address the invalid pointer addresses we saw 6 months ago on 64-bit OS.

hbldh

I seem to recall that I tried to remove them during the 0.5 update and that it seemed that they were still needed, so I left it at that. Some quick tests this morning seems to indicate otherwise though...

@jdjones I will evaluate it this morning and hopefully make a release this week.

hbldh

Published a new release to PyPI (PyMetaWear 0.6.0) that has the 64-bit stuff removed. It worked fine without it now; apparently I did some mistake last time I tried to remove it. I also updated to use the C++ API version 0.6.0 at the same time.

@jdjones Try the logging bit now and feel free to send me the logging code or make a pull request so we can add a working logging example to the Github repo and the documentation!