XDK2MAM Status Report: XDK110 can now publish via MAM2.
As we are nearing the end of the XDK2MAM roadmap, we still have a hard challenge to solve: make the XDK110 send MAM messages onboard. To tackle this endeavor, we have chosen to work with MAM2, because it has a C implementation on Entangled. But things are not easy at all: C based MAM2 is still under development and the Build system used by IF (Bazel) does not help a lot when trying to compile the needed code on devices such as the XDK110.
Since our last milestone overlaps with IOT2TANGLE (our new proposal to extend IoT-Tangle integration) we thought it would be a good idea to start by transitioning from Bazel to a more IoT friendly building tool. Considering that some work had already been done on the Entangled cclient by Sam Chen using Cmake on an ESP32, we decided that this was the right tool.
Cmake Entangled MAM2 Library available on IOT2TANGLE repository
The first step was then to be able to build applications to send and receive MAM2 messages on devices such as Raspberry Pi3B and ESP32. So after studying the work done by Bazel, we were able to get all created headers and compose the CMakeLists.txt files needed to get the send-msg and recv applications without having to deal with Bazel limitations.
You can find the IOT2TANGLE MAM Cmake based library here. Keep in mind that this is a work in progress and lots of optimizations will occur in the coming weeks. As always, feedback to improve our work is welcome.
XDK110 can now send datasets via HTTP/MQTT/UDP/USB/BLE using MAM2 Protocol
XDK2MAM provided solutions to send XDK110 sensor’s data to the Tangle using MAM Javascript wrapper and a listener server written in Nodejs. This server listens for the datasets sent by the XDK110 and publishes them via MAM1.
Now that we can easily build MAM2 applications on a Raspberry (where the listening Nodejs server usually runs), we considered that offering solutions for this new protocol was a good idea, so we went for it.
If you take a look at one of the available methods (HTTP/MQTT/UDP/USB/BLE) you will find a new xdk2mam2-nodejs folder for each method. For instance, if you head to the HTTP folder, you now have.
- xdk2mam-c (C code to flash to the XDK110)
- xdk2mam-nodejs (Nodejs code to get and publish sensor’s data via MAM1)
- xdk2mam2-nodejs (Nodejs code to get and publish sensor’s data via MAM2)
As always, the new code comes with step by step guides on each readme explaining how to get everything working. If you are familiar with XDK2MAM code, this should not represent any major challenge. We have made the process as easy as possible.
Please notice that the provided code will only work out of the box on Unix (Linux/MacOS) based Systems but could easily be ported to Windows by doing the Cmake part done by the install.sh routine manually.
Dealing with MAM2 publications is straightforward:
- install.sh downloads Cmake Entangled, compiles needed files (send-msg, recv), moves them to the xdk2mam2-nodejs folder level and deletes all unneeded code to keep things clean.
- server.js listens to the XDK110, gets the datasets and then executes the send-msg binary to send the data, outputting Address, Msg ID and Bundle on console.
- getData.js executes recv and requires a Bundle as argument to fetch the published data.
We know this “wrapper” is somehow a naive implementation of MAM2, but considering that IF developers are working on bindings to make MAM2 available on Golang and other languages it does not make much sense for us to spend effort and time on this.
Challenges ahead
As we work to find a way to make the XDK110 sign MAM transactions onboard we are conducting research to solve two major issues:
- Reduce binaries size to make them fit on 1MB flash devices such as the XDK110. The send-msg produced by the Entangled Cmake Lib results in 1MB size so we are analyzing ways to clean unnecessary code (ie. related to the process of getting messages but not used while sending) in order to shrink its size a bit.
- High CPU load while building bundles. Entangled MAM uses a lot of processing while getting the Bundle ready and this is an important limitation on devices with MCUs that could get stuck. We need to find ways to reduce the work done at this point to allow Onboard transactions.
