Navigate back to the homepage

My experience with porting a library to Arduino

Porting a low-level library to a new platform isn't easy. Let's take a look at how I proceeded.
Vladimír Záhradník
July 26th, 2020 · 19 min read

Porting software to new architectures or platforms has always appealed to me. Imagine that the software that was not written for your device suddenly runs on it. I’ve always admired people who develop emulators for older (but also new) game consoles, do reverse engineering, or write software on the console without access to the devkit.

Specifics of the Arduino platform

My task was much easier, but I still learned a lot. I had to bring the C library, written for Windows, Linux, and macOS platforms, to the Arduino platform. The library’s authors mentioned that porting to new platforms, including microcontrollers, shouldn’t be difficult, but Arduino is quite peculiar. Notably, Arduino is not just another instruction set architecture. It is a framework, which supports multiple development boards across different CPU architectures — MIPS, ARM, RISC. The authors have also defined the essential functions that you must use in your programs, when you code using this framework. Otherwise, you will not be able to print “Hello, world.” into the console. Last but not least, these chips are relatively inefficient; they have barely 2 kB of memory, but they are interesting to us since they allow us to program applications running in real-time.

The microcontroller can guarantee that a critical task will be executed regularly, for example, via interruptions. The Raspberry Pi may be much more powerful, but it doesn’t support applications that must be running in real-time. I was convinced of this already when we made a stopwatch for firefighting competitions on it, and it turned out that RPi was not able to receive a stable flow of pulses. In short, some pulses were not caught because another task was running under Linux for a fraction of a second, and the conversion from pulses to engine speed fluctuated by an order of magnitude give it or take 100 revolutions. The microcontroller has no operating system on top of it; your code runs directly on it. Thus, you have much more control over how to run a time-critical code.

Real-time operating systems

Naturally, there are also real-time operating systems. These were designed to run time-critical tasks, which makes building applications easier. You have more memory, a more mature system, a more powerful processor, and a full-fledged implementation of the C/C++ language, or even others such as Rust or Python/MicroPython. However, I did not currently find anything for the Raspberry Pi. I found some information that FreeRTOS from Amazon should be supported on it, but the information was for the RPi2. I can’t guarantee it would run on the latest RPi4. Besides, there’s a project I’ve been personally interested in for a long time — The Zephyr Project. Unfortunately, it doesn’t have the Raspberry Pi in the list of supported platforms. To sum it up, I believe exploring application programming for real-time operating systems is worth it, so I will definitely look into it in the future.

SimpleMotionV2

I need to bring up communication with the drive that controls the servomotor. Communication must take place very quickly, several hundred times per second. I need to read real data from the servo and correct its movement if I find out that the movement deviates from the norm. In our project, we use the IONI by Granite Devices. Communication takes place via the RS485 bus and the company’s proprietary communication protocol, SimpleMotion. RS485 only defines the transmission line’s physical parameters, and it is up to you how you transmit the information.

This company develops for their IONI drives a tool called Granity. This application allows you to set the drive parameters. However, it only runs on some platforms, and especially for our application, where we need to read data in real-time very often, it is unusable. So I dug deeper and found the SimpleMotion protocol library. It’s available as open-source on their GitHub.

Before we begin

Before we start porting the library to the new platform, we need to gather as much information as possible. In my case, I studied how the RS485 bus works. Next, I looked at everything I could find for the IONI drive and checked the entire wiki for the SimpleMotion protocol. Unfortunately, there wasn’t much information; it was brief and incomplete. You will learn that you can send batch commands to the drive’s buffer, and the IONI will read and run commands from it. Thus, you can avoid the need for real-time hardware. Our second option is that we write commands continuously and in rapid pace. According to the wiki, the upper limit is up to 20,000 commands per second.

The wiki refers to a few examples, which describe how to initiate such communication. But I must add that no one has updated it in five years, and most of the existing examples are commented as not having any functionality.

Non-working examples
Non-working examples

Interestingly, the best source of information was the library’s source code, and that’s not much. In its repository, I found the subfolder doc, where authors described their protocol. Simply put, after you open an RS485 line, you communicate with the devices asymmetrically according to the master/slave model. Your computer or microcontroller is the master. It communicates with one or more slave devices. All devices are connected to the same bus, and each device is assigned a unique node address. You, as the master, can read or write parameters. In both cases, you need to know the parameter name and the node address.

All parameters are defined on the wiki page of the protocol and in the library’s header file.

The Readme file gives us useful information for porting. It lists mandatory files that must be compiled and optional files that add some functionality but are not critical. Also, in the “Porting to new platform” section, we find basic instructions on what needs to be done.

For the purposes of this article, I will present these findings here. I will refer to them later.

Required files

  • simplemotion.c/.h - The core of the entire library. Defines APIs that users call from their code, such as smOpenBus( ... ) function, which initiates communication, or the smRead1Parameter( ... ) function, which reads a value from a slave device
  • sm485.h - Command definitions for communication via RS485
  • sm_consts.c - Pre-calculated CRC tables
  • simplemotion_defs.h - Parameters that can be sent to slave devices
  • simplemotion_private.h - Definitions of internal structures and parameters that are used only inside the library, and the user should not even know about them
  • busdevice.c/.h - Maintains communication via RS485 bus

Optional files

  • bufferedmotion.c/.h - Code that adds the ability to send commands to the drive in batches. The driver stores them in its buffer and executes them step by step
  • devicedeployment.c/.h - Code that allows you to load firmware into the drive or send settings to it
  • files in the drivers/ folder - Drivers for basic platforms that implement the functionality required by the code in busdevice.h and simplify the construction of RS485 communication

I will add that the whole library is written in C. Later, it will be quite an important fact.

Let’s start porting

Whether you’re porting a library like mine or some complex game engine, the fundamental process isn’t that different. In the first step, you need to put together as much information as possible about the host platform, the target platform, and ideally also about the architecture and code structure of the project you are trying to port. I have summarized everything necessary in the point above.

As a next step, I copied the source code repository to my computer:

1git clone git@github.com:GraniteDevices/SimpleMotionV2.git
2Cloning into 'SimpleMotionV2'...
3remote: Enumerating objects: 162, done.
4remote: Counting objects: 100% (162/162), done.
5remote: Compressing objects: 100% (113/113), done.
6remote: Total 1316 (delta 95), reused 103 (delta 49), pack-reused 1154
7Receiving objects: 100% (1316/1316), 2.03 MiB | 3.55 MiB/s, done.
8Resolving deltas: 100% (807/807), done.

Since my computer is among the supported platforms, I tried to compile the library. I wanted to see what the output would be like or what errors the compiler would print. The authors have automation running via Travis CI. I just looked at the .travis.yml file and found what I needed. The library has a Makefile, which we can use, and also some unit tests. We can build it easily:

1make

And we can run the tests (albeit there aren’t many) like this:

1make test

The library was successfully compiled on the host platform. Although the compiler printed out some errors, I assume they were expected based on the code comments.

1...
2drivers/tcpip/tcpclient.c:255:16: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
3 255 | int sockfd=(int)busdevicePointer;//compiler warning expected here on 64 bit compilation but it's ok
4 |

Building a library for the target platform

This should be your first goal. It’s expected that you won’t be able to build the library on a new platform. A library that would be compiled without any modifications is probably quite simple. Arduino IDE, the development environment for Arduino, does not use any Makefiles. If you create a sketch (project file with an .ino extension) in the Arduino IDE, this file goes through several stages before being compiled. It is first preprocessed and transformed into a full-fledged C++ code. The Arduino preprocessor fills in the missing #include lines and includes the header files’ contents, especially Arduino.h. It also automatically creates function declarations that you have defined in your project. It then tries to find the necessary dependencies (libraries) for your project, select the target architecture according to the type of development board, and in the last step, sends all this information to the compiler. It will try to compile the code, and if it succeeds, the firmware will be loaded into the microcontroller.

Given how code compilation works on Arduino, we can’t build the library directly. We need to create some simple sketch that will call a function from this library. It can be anything. Arduino IDE will find a dependency on your library in the project and try to build it. It will probably fail at this step, as it did on my PC.

Of course, you don’t have the library in the Arduino library register yet. Arduino allows you to install a .zip archive with library code via IDE, or you can copy the library yourself. The Arduino IDE searches libraries in specific folders, in my case:

1[vzahradnik@EliteBook ~]$ ls -la Arduino/libraries/
2total 4
3drwxr-xr-x 1 vzahradnik vzahradnik 86 jul 25 23:37 .
4drwxr-xr-x 1 vzahradnik vzahradnik 58 jul 24 18:38 ..
5drwxr-xr-x 1 vzahradnik vzahradnik 234 jul 24 16:06 CONTROLLINO
6-rw-r--r-- 1 vzahradnik vzahradnik 86 jul 24 16:05 readme.txt
7drwxr-xr-x 1 vzahradnik vzahradnik 146 jul 26 11:16 SimpleMotionV2-Arduino
8[vzahradnik@EliteBook ~]$

It doesn’t matter what you name a folder with a library. The Arduino IDE looks for header files in all of these folders.

My very first Arduino sketch looked something like this:

First attempt to build the library on Arduino IDE
First attempt to build the library on Arduino IDE
```

Surprisingly, there were far fewer errors than I expected. I solved the include error simply:

1diff --git a/devicedeployment.c b/devicedeployment.c
2index 0eb954f..2995a41 100755
3--- a/devicedeployment.c
4+++ b/devicedeployment.c
5@@ -1,6 +1,6 @@
6 #include "devicedeployment.h"
7 #include "user_options.h"
8-#include "utils/crc.h"
9+#include "crc.h"^M
10 #include <stdio.h>
11 #include <string.h>
12 #include <stdlib.h>

Blocking functionality that cannot be ported

Surprisingly, after this modification, the library was built successfully. However, this certainly doesn’t mean that it’s functional. Arduino IDE probably compiles only what it needs to. In the sketch, I included the file simplemotion.h. Somewhere in it, there’s an include statement for devicedeployment.h, which in turn includes crc.h. Therefore, Arduino IDE will include all files referenced from the file defined in the project. However, only the functionality you use is compiled. I used only a single function, smGetVersion(), which didn’t depend on any other, so the code was built successfully. Despite the imperfections, this is a good starting position. The simplemotion.h file also includes header files such as the following:

1...
2#include <stdio.h>
3#include <stdint.h>
4...

I was surprised that these files exist for my architecture. The Arduino framework does not implement these files, so they will probably be part of the GCC compiler for the AVR architecture I use. I later confirmed this assumption. In parallel with the Arduino IDE, I also have PlatformIO running, which downloads dependencies, such as this compiler, to the .platformio/packages folder. Directly in it, I found the toolchain-atmelavr folder, which contained the header files provided by the compiler, and also the framework-arduino-avr folder, which provides the Arduino framework header files. The ability to look at these files and look up definitions of functions or C++ classes in them proved to be very advantageous.

My compiler implements most of the standard C library, so the compilation was successful. But there are also calls in the SimpleMotion library that require a pointer to a file and so on. Naturally, we cannot work with files on a microcontroller. The code is compiled, but that doesn’t mean it works. In the next step, I decided to map what functions the SimpleMotion library calls from the standard C library. So, in all the files, I commented out the include lines and watched where the compilation breaks. The compiler genrated over 2000 lines of errors for me, but most were repeated. I clearly mapped the library’s functions and identified the ones I need to replace with a functional Arduino equivalent.

1...
2# stdio.h
3/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.h:106:52: error: unknown type name 'FILE'
4 106 | LIB void smSetDebugOutput( smVerbosityLevel level, FILE *stream );
5/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:1131:18: warning: incompatible implicit declaration of built-in function 'snprintf'
6
7/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:111:17: warning: implicit declaration of function 'fprintf' [-Wimplicit-function-declaration]
8 111 | fprintf(smDebugOut,"%s: %s",smBus[handle].busDeviceName, buffer);
9/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:858:7: warning: implicit declaration of function 'fopen' [-Wimplicit-function-declaration]
10/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:865:5: warning: implicit declaration of function 'fseek' [-Wimplicit-function-declaration]
11/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:866:16: warning: implicit declaration of function 'ftell' [-Wimplicit-function-declaration]
12/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:877:9: warning: implicit declaration of function 'fclose' [-Wimplicit-function-declaration]
13/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:885:15: warning: implicit declaration of function 'fread' [-Wimplicit-function-declaration]
14
15# stddef.h
16/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.h:199:39: error: unknown type name 'size_t'
17 199 | LIB int smDescribeSmStatus(char* str, size_t size, SM_STATUS status);
18/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:21:18: error: 'NULL' undeclared here (not in a function)
19/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:840:13: warning: implicit declaration of function 'strcpy' [-Wimplicit-function-declaration]
20
21# stdarg.h
22/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c: In function 'smDebug':
23/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:95:5: error: unknown type name 'va_list'
24
25# string.h
26/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c: In function 'smOpenBus':
27/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:248:5: warning: implicit declaration of function 'strncpy' [-Wimplicit-function-declaration]
28/home/vzahradnik/Arduino/libraries/SimpleMotionV2/simplemotion.c:967:2: warning: implicit declaration of function 'memcpy' [-Wimplicit-function-declaration]
29
30/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:97:13: warning: incompatible implicit declaration of built-in function 'strlen'
31/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:101:6: warning: implicit declaration of function 'strncmp'
32/home/vzahradnik/Arduino/libraries/SimpleMotionV2/devicedeployment.c:288:5: warning: incompatible implicit declaration of built-in function 'sprintf'
33...

At first glance, I see that only the code for uploading firmware to the drives, devicedeployment.c works with the files. It’s logical; it needs to read the data from somewhere. However, this functionality is not critical, it is only listed as optional, so I blocked it. I used preprocessor directives for this. When you compile a library for Arduino, the ARDUINO flag is defined somewhere. Then I can check if the code is running in this environment. This is done as follows:

1#ifdef ARDUINO
2// Arduino-specific code
3#endif

I decided to add this control right at the beginning of the header. This will ensure that an “empty” object file is created on the Arduino. By discarding the code from the header, its API will not be available from the library. Similarly, we need to add a clause to the matching C file that implements function definitions.

1#ifndef SMDEPLOYMENTTOOL_H
2#define SMDEPLOYMENTTOOL_H
3
4// Disable compilation on Arduino boards
5#ifndef ARDUINO
6
7...
8
9#endif // ARDUINO
10#endif // SMDEPLOYMENTTOOL_H

What can and can’t be ported?

I analyzed the code and continuously modified it for Arduino. Except for the code that wanted to read the commands from the file, the rest could be modified somehow. The most important points are the following:

  • Functionality responsible for bus communication is defined in the busdevice.h file
  • The void smDebug( ... ) function is responsible for writing logs to a file. I remade it to write logs to the console on the Arduino. Yes, it’s slow, but it’s just for debugging.

We will now look at both cases in detail.

Communication with RS485 bus (busdevice.h)

This module is the basis for communication with other devices using the SimpleMotion protocol. It defines four callback functions, and if you implement them for your device, the communication should start working. The rest of the code in busdevice.h/.c is already platform-independent. These four functions ensure opening and closing the communication channel, reading data from the bus, and sending data via the bus. There is another function, which is used, for example, to flush the RX/TX buffer.

The authors of this library also prepared drivers for the most common platforms. They provide communication via RS485 through a serial line or a USB converter with an FTDI chip. These drivers are platform-specific; they work under Windows, Linux, and macOS. By default, these drivers are disabled and must be enabled by the user when compiling with the ENABLE_BUILT_IN_DRIVERS flag. In this case, the user calls the function smOpenBus (const char * devicename ), where he specifies the port name as a parameter (let’s say /dev/ttyS0). The code tries to call individual drivers one by one, and if one of them does not return an error during initialization, the communication is considered successful. The drivers directly implement these four basic callback functions.

As an alternative, you can implement these callback functions directly in your code. In this case, you need to call from your code the smOpenBusWithCallbacks ( ... ) function. This function requires references to callback functions as its parameters. Although this method works, and I have successfully tested it with my Controllino, as we make the Arduino library, we can afford to incorporate the bus initialization code directly into the library as a new driver. So far, I have implemented a simple driver for the industrial Controllino Maxi/Mega that I am currently using and also for the ESP32 Grey by M5Stack.

If we could successfully compile the library to this point, this will change after enabling the ENABLE_BUILT_IN_DRIVERS flag. All of a sudden, all the drivers from the drivers folder will start to compile, and while the compiler provided some basic header files for the standard C, with drivers’ code you’ll hit the wall. For example, they work with sockets, have calls to pthread, and so on. First, I tried to modify the code using #ifdef ARDUINO #endif blocks just enough to compile the code. It turned out that the code was already so specific that it was pointless. Finally, I did something similar to the devicedeployment.h file above and blocked all existing code for each driver.

1#ifndef SM_D2XX_H
2#define SM_D2XX_H
3
4#ifndef ARDUINO
5// Original code
6#endif // ARDUINO
7#endif

After this adjustment, the compilation was successfull again. I could start writing a driver for RS485 communication for Controllino and ESP32 Grey. As I wrote, the basis is to implement a few functions that the library will call. Both boards have sample examples for RS485 communication. You can add the bus support just with a few lines of code. Both controllers are very similar, the operation will be explained on the driver for Controllino. To give you an idea, this is how easy it is to read the data:

1smint32 controllinoRs485PortRead(smBusdevicePointer busdevicePointer, smuint8 *buf, smint32 size)
2{
3 smint32 n = 0;
4
5 // Read only when there's data
6 if (Serial3.available() > 0) {
7 n = Serial3.readBytes(buf, size);
8
9 }
10
11 return n;
12}

Compared to other drivers, this code is minimalist. The Arduino framework solves everything for us. I similarly implemented other functions. However, I will mention one thing that has proved problematic. Arduino implements many things as C++ objects (see the Serial3 object). Since the library is written in C, I encountered interoperability issues between C and C++. We will discuss these issues in a separate section.

Logging

Logging is implemented using the internal function void smDebug ( ... ). It writes logs to a file, which is not possible in the case of Arduino. I replaced all calls to the printf function by writing directly to the console via the Serial object. It is available globally throughout the Arduino project after including the Arduino.h library. The adjustment looked something like this:

1...
2if(smIsHandleOpen(handle)==smtrue)
3{
4 #ifdef ARDUINO
5 Serial.print(smBus[handle].busDeviceName);
6 Serial.print(": ");
7 Serial.print(buffer);
8 #else
9 fprintf(smDebugOut,"%s: %s",smBus[handle].busDeviceName, buffer);
10 #endif
11... }

Note that I replaced one call to fprintf(...) with several calls to the print(...) function. It does not allow you to specify a format and combine multiple variables into a single string. I thought of allocating a buffer to it, where I would write the final strings using the snprintf(...) function, but ultimately, I decided to save some memory. It’s already limited, and we don’t want to lose speed. Printing to the console is very slow and only suitable during program debugging. In the production program, you can and should disable these prints by clearing the ENABLE_DEBUG_PRINTS flag. You can also do this directly in your Arduino sketches, as follows:

1#undef ENABLE_DEBUG_PRINTS

Let’s take a look at some of the problems I’ve been solving for hours. They point out how treacherous the development can be at times.

Arduino IDE build environment

As I mentioned, the Arduino IDE does many things automatically. It modifies your code, detects dependencies, and builds binary firmware, which is then uploaded to the development board. This works quite well for simple projects, but you’ll run into quite some problems if you use a library like mine.

The most common was the linker problem.

The library won't build
The library won't build

I was solving this problem for at least an hour. I had all the files in the folder, and the code called them correctly, so why couldn’t the Arduino linker find them? It was probably related to the library specification. Back in the day, the code for Arduino libraries was stored directly at the root of the repository. When Arduino finds code organized in this way, it automatically assumes that all C files are in the same hierarchy, and the code in the subfolders does not compile at all. The solution was to reorganize the code to fit the newer specification. It expects the library code in the src folder and a defined manifest with a library description at the root folder of the repository. After I made these simple modifications, the code had finally compiled.

I encountered this problem when compiling the project via the Arduino IDE. When I compile a sketch and include my library, the print to serial line doesn’t work. For a long time, I researched if I had something in the code that would conflict with the basic printing functionality of the Arduino framework, but I didn’t find anything. I guess Arduino IDE just didn’t include some crucial code during the compilation. I tried the same project in the PlatformIO environment, where the print worked flawlessly. I think it’s a good idea to avoid as many problems as possible when porting, so I recommend a stable environment where you have more control over the build process. You should debug the library for the Arduino IDE at the very end, once you know the code works.

Interoperability between C and C++ code

The SimpleMotion library was written in pure C. It does not contain any C++ stuff. I tried to interfere with the library as little as possible so that my changes could be easily modified when incorporating new code from the upstream. I wanted to keep the possibility of building the library on existing platforms, just with Arduino’s addition among the supported platforms.

The hard lesson I learned here was this:

If the source file has a .c extension, it is compiled in C, and if it has a .cpp extension, it is compiled in C++.

I thought the file extension was irrelevant and that Arduino automatically used a compiler ready to compile C++ code. But I was wrong. This compiler also distinguishes what it compiles. Because my code was compiled as C-like, I got strange errors, such as the unknown keyword class. All C++ objects defined by the Arduino framework, are unusable in pure C mode. The solution was to rename the file extension from .c to .cpp, but this had revealed other problems. When a file is compiled as C++, things that are still allowed in C suddenly cause a warning at best or an error at worst in C++.

1.pio/libdeps/uno/SimpleMotionV2-Arduino/src/simplemotion.cpp:1107:258: warning: ISO C++ forbids converting a string constant to 'char*' [-Wwrite-strings]
2 smDebug(handle,SMDebugLow,"Previous SM call failed and changed the SM_STATUS value obtainable with getCumulativeStatus(). Status before failure was %d, and new error flag valued %d has been now set.\n",(int)smBus[handle].cumulativeSmStatus,(int)stat);

Specifically, I resolved this error by modifying the function declaration to expect the string constant char const * as a parameter. However, I thought it would be best to separate all the Arduino C++ code into a separate module, whose API can also be called from C-code. I created the arduino_helper.h/.cpp module and placed it directly next to the Controllino and M5stack drivers. Suddenly, my changes to the library became much more readable. The original simplemotion.c file has minimal changes, and everything important happens in this module.

Linker problems and the extern keyword

I thought I knew what the extern keyword does. After hours of debugging, however, I realized that my knowledge was far from enough. But now I understand it well.

This keyword has three meanings: First, if you specify it as a variable modifier, then you make this variable available to other modules. Usually, you place such variables in a header file so that other modules can easily access them.

Example:

1/* Header main.h */
2extern int variable;
3
4/* Main module */
5#include <stdio.h>
6#include "main.h"
7
8int main() {
9 variable = 0;
10}
11
12/* Another module */
13#include "main.h"
14
15void someFunction() {
16 variable = 1;
17}

By default, variables are visible only in the module where you define them. The extern keyword changes that. We already use this variable directly in the main C file. Other modules can access this variable after including the header file. The variable name must be unique throughout the library; otherwise, the linker will complain.

Second meaning: If you specify extern in function declaration, then you make this function available to other modules. For functions, the extern keyword is specified by default, and you do not need to explicitly specify it. Here too, however, the function name must be unique throughout the library. The C language does not recognize namespaces.

Example:

1/* Header main.h */
2extern int doSomeStuff();
3
4/* Main module */
5#include <stdio.h>
6#include "main.h"
7
8int main() {
9 doSomeStuff();
10}
11
12/* Another module */
13#include "main.h"
14
15void someFunction() {
16 doSomeStuff();
17}

Third meaning: extern "C" This example is the most important with regards to the C/C++ interoperability. The C++ compiler mangles function names. It does this to distinguish several functions with the same name but different parameters (typical for function overloading). However, in order to call C++ functions from the C code (and vice versa), with this command you explicitly tell the compiler not to mangle function names. In other words, in a library, you can have C++ code in one module, C code in other modules, yet they can both communicate through functions.

In practice, this block is added using preprocessor directives. If the header file loads a module written in C++, it knows that the functions should have names compatible with C-code. And the module written in C will not see the extern directive at all.

1#ifndef SIMPLEMOTION_H
2#define SIMPLEMOTION_H
3
4...
5
6#ifdef __cplusplus
7extern "C"{
8#endif
9
10// Functions, which names will be compatible with what's expected by the C code
11LIB int smDescribeStatus(char* str, size_t size, int32_t status);
12
13#ifdef __cplusplus
14}
15#endif
16#endif // SIMPLEMOTION_H

The __cplusplus directive is defined only if the file that includes the header is compiled in C++ mode. In practice, this means that it must have a .h or .cpp extension.

External code incompatible with C

When I was adding support for ESP32 boards, I had to include several additional header files to the library to compile the project. It turned out that after adding those files, the compiler started complaining about errors such as unknown type name 'class'. Apparently, the library code for ESP32 does not contain the extern "C" directive and defines some classes within itself. When you add such header files to the library, files compiled in C mode won’t build.

1/home/vzahradnik/.platformio/packages/framework-arduinoespressif32/cores/esp32/Stream.h:38:13: error: expected '=', ',', ';', 'asm' or '__attribute__' before ':' token
2 class Stream: public Print
3 ^
4In file included from .pio/libdeps/m5stack-grey/SimpleMotionV2-Arduino/src/drivers/arduino/arduino_helper.h:22:0,
5 from .pio/libdeps/m5stack-grey/SimpleMotionV2-Arduino/src/simplemotion.h:24,
6 from .pio/libdeps/m5stack-grey/SimpleMotionV2-Arduino/src/simplemotion_private.h:8,
7 from .pio/libdeps/m5stack-grey/SimpleMotionV2-Arduino/src/sm_consts.c:1:
8/home/vzahradnik/.platformio/packages/framework-arduinoespressif32/libraries/Wire/src/Wire.h:38:1: error: unknown type name 'class'
9 class TwoWire: public Stream

The solution to this problem has proved to be simple. I knew that object code could only be compiled in C++ mode and that errors only occur when compiling files in C mode. I used a directive that verifies in which mode I compile the module. However, this time, I did not add the keyword extern “C” but the problematic header files. The build process worked after this change without any problems.

1#ifdef __cplusplus
2#ifdef ESP32
3#include <Wire.h>
4#include <FS.h>
5#include <SPIFFS.h>
6#include <SD.h>
7#include <HTTPClient.h>
8#endif
9
10extern "C" {
11#endif
12...

Functions that have C++ objects as parameters

If you have functions somewhere in your code that expect a pointer to an object as a parameter, such code will not compile. As I have already shown, the term class is unknown for C modules, and the compiler declares an error with this type. Momentarily, I bypassed the problem by defining a function that expects a pointer to a void block, and later in C++ code, I cast this pointer to a C++ object. I know it’s not perfect, but it works.

First, I defined a new type via typedef:

1typedef void* ArduinoSerial;

Subsequently, I could use this type as a function parameter:

1LIB void smSetDebugOutput( smVerbosityLevel level, ArduinoSerial serial );

In the Arduino sketch, I simply pass a reference to the Serial object:

1smSetDebugOutput(SMDebugLow, &Serial);

This object is then cast to ArduinoSerial, and it’s passed through the C-code all the way to the arduino_helper module, where I cast it to the pointer to the Print object. Then I can print to the serial line as I’m used to doing.

1/*
2 * arduino_helper.cpp
3 *
4 */
5Print* consoleOut = NULL;
6
7#ifdef ENABLE_DEBUG_PRINTS
8void arduinoPrintMessage(const char* message) {
9 if (consoleOut != NULL) {
10 consoleOut->print(message);
11 }
12}
13#endif // DEBUG PRINTS

Bit operations, macros, and alerts

Although the compilation works at this point, I’ve noticed a lot of errors in the macros. By analysis, I found that macros are used in only three functions designed mainly for debugging. By disabling the ENABLE_DEBUG_PRINTS flag I will get rid of most alerts. However, this can’t be a permanent solution.

1...
2pio/libdeps/uno/SimpleMotionV2-Arduino/src/simplemotion.cpp: In function 'int smDescribeFault(char*, size_t, int32_t)':
3.pio/libdeps/uno/SimpleMotionV2-Arduino/src/simplemotion_defs.h:84:25: warning: left shift count >= width of type [-Wshift-count-overflow]
4 #define BV(bit) (1<<(bit))
5 ^
6.pio/libdeps/uno/SimpleMotionV2-Arduino/src/simplemotion.cpp:1189:20: note: in definition of macro 'APPEND_IF'
7 if (((source) & (name)) != 0) { \
8...

After a deeper analysis, I found a problem. Arduino boards built on the AVR platform define the size of an int type as 16 bits. I confirmed this by compiling a simple sketch where I added the call to sizeof(int). I got a value of 2 bytes, i.e., 16 bits. The SimpleMotion library defines bitmasks, and for some values, we exceed the range of 16 bits. The problem is located in two places — the definition file simplemotion_defs.h and the file where the code for calculating CRC is, crc.c.

For most of the warnings, only a single line was responsible:

1#define BV(bit) (1<<(bit))

I fixed the error by using long instead of int, which has 4 bytes on the AVR platform.

1#ifdef __AVR__
2/* Int on Arduino AVR boards is only 2 bytes, which is not enough
3 * for all values defined here. We'll use long instead.
4 */
5#define BV(bit) (1L<<(bit))
6#else
7#define BV(bit) (1<<(bit))
8#endif

Why didn’t I use #ifdef ARDUINO? Well, the Arduino framework runs on many different platforms. It turned out, for example, that I don’t get warnings on the ESP32 platform because the int type has enough bytes. Therefore, it is better to modify the macro definition only for those platforms where it is necessary. I made similar adjustments to the CRC calculation code. The library compiles at this point without errors.

Compilation differences between PlatformIO and the Arduino IDE

I have already mentioned several compilation problems under the Arduino IDE. However, if you look at alternatives, you may encounter other problems. In general, however, I adapt the library to run under the PlatformIO IDE. The compilation process is much clearer, and I have more control over it. At the same time, all the library code is compiled, not just the selected parts, depending on how the Arduino IDE decides. When I compile under PlatformIO, I get rid of several issues. Debug prints are much more readable. And most importantly, in such a compiled project, printing to the serial line also works flawlessly. It’s something I still haven’t solved in the Arduino IDE.

Project in PlatformIO IDE
Project in PlatformIO IDE

Hard fork

Soon after I started working on the library port for Arduino, it was clear that it would not be possible to incorporate the changes into the upstream. It is not enough to add Arduino-specific code. To make the library compatible, you need to rearrange the code.

  • All source code must go to the src subfolder
  • It is necessary to write usage examples in the examples folder
  • It is necessary to define a library manifest with the current description
  • You also need to send a request for adding the library to the official repositories for Arduino IDE and PlatformIO
  • It would also be wise to set up an automated build for Arduino boards using Travis CI and the Arduino CI testing framework

I created the fork under the organization I work with. You can find it here. At this point, I don’t know yet to what extent I will maintain it. However, I would like to make the library functional enough to work under both Arduino IDE and PlatformIO. Since I still have support mainly for Controllino and one board with the ESP32 chip, it is definitely possible to add drivers for other boards or expansion modules.

Conclusion

This project taught me one thing: each port is specific, and you always need to know what you are doing. I am sure that some of the problems I have encountered would not surprise others. As a developer who writes code in several programming languages, I am glad that I could adapt relatively quickly and successfully complete the project. My knowledge is now a little bit bigger, and I believe it is transferable to other projects that I will work on in the future.

This library port was an interesting puzzle for me. It wasn’t an easy task, and also I didn’t have anyone to ask questions, especially due to a shortage of time. Because of that, I am all the more proud of the result.

Do you have experience with porting code? Share your thoughts in the comments below.

More articles from Vladimír Záhradník

My First Webinar

What do you need to train people over the Internet?

June 23rd, 2020 · 9 min read

How our club transformed to online

The COVID crisis brought unexpected challenges, our club found itself at a crossroads.

June 23rd, 2020 · 11 min read
© 2018–2020 Vladimír Záhradník
Link to $https://github.com/vzahradnikLink to $https://medium.com/@vladimir.zahradnikLink to $https://www.youtube.com/channel/UCogZ6qxqKa_WIsw7NnU2IaALink to $https://twitter.com/VladoZahradnikLink to $https://www.linkedin.com/in/vladimirzahradnikLink to $https://www.facebook.com/vzahradnikLink to $https://www.instagram.com/vladimir.zahradnik