We are in a period of transition from an older browser based web application space to an emerging Mobile technology space and the online Embedded device or Internet of Things. Whether it is a Mobile, Embedded device or driver, the resultant programming domain is known as a system programing. Applications built for this space often need to be fast and use resources with a greater efficiency when compared to a traditional application. C is the archetypal Systems language and is ideally suited both for embedded system/device programming and what is known in the Mobile web application domain as a native application. There is a strong dominant trend emerging of the mobile native web application over the mobile web browser HTML 5 application. Major parts of the Windows and Linux are written in C. C provides access to both low level system layers such as the assembly layer for an operating system and higher level layers such as the Graphical User Interface.
For the reasons above C continues its domination the TIOBE Index from www.tiobe.com as the top programming language.
Introducing the C Toolchain
A C toolchain is the set of programs like compilers used to build a C application. We will look at two toolchains. The GNU reference toolchain and the Android ARM toolchain for the ARM family of CPU’s.
One of the issues with C programming is portability. Working at the low level that C allows code can be dependent on a specific architecture such as ARM, and there associated toolchains and native libraries.
The two C toolchains GNU and Android’s arm-eabi-gcc with Bionic that we will investigate will show how useful Docker containers can be to quickly create multiple build environments with different sets of tool chains and support libraries for a range of operating systems. Using Docker this way can greatly enhance a systems programmer workflow when developing system level code.
Docker overview Docker Installation
Linux installation Linux
Windows installation Windows
The official Ubuntu repository has an older but stable version of Docker
The examples are tested on the latest version of Docker running on Ubuntu 14.04 64 bit
The following commands will set up the equivalent Docker installation on Ubuntu 14.04
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9 sudo sh -c "echo deb docker main\ > /etc/apt/sources.list.d/docker.list" sudo apt-get install lxc-docker cat /etc/group sudo gpasswd -a <your user name> docker sudo service docker restart
The source code for examples can be found here.
Learn By Example
We will work through the provided source code and develop a number of programs to illustrate the concepts involved, hopefully many of the details will become concepts in the context of a working example.
We will start with the canonical tool chain and build tool for C development. The GNU toolchain, specifically make and by default binutils. GNU is known for stability and standard compliance. By using Docker we can easily create a development environment using the latest GNU binaries in a debian container.
Using the provided source code navigate to the /gnucomplete directory and in a terminal issue the following command
docker build -t="eduonix/ctoolsgnu:1.0.0" .
Docker will pull the gnu base image from the public repository DockerHub. The first time you build will take a while to pull the image from docker hub however subsequent builds will only take a few seconds.
Once the image has built we create what is know as a Docker container, a custom Linux Container. We now have a root filesystem with the complete and latest GNU toolchain to develop C code. Any source code in the src directory is copied to the same directory in the container and we land in a terminal with the bash.
You can mount additional source folders at runtime like this
docker run -t -i eduonix/ctoolsgnu:1.0.0
Docker is an emerging technology that has many applications. Here we are using it to rapidly set up build environments. Docker increases our productivity at very little cost.
You can mount additional source folders at runtime like this
docker run -t -i -v <path to src directory>:/mnt/share eduonix/ctoolsgnu:1.0.0
WARNING make sure you unmount the attached directory as it will delete the /* if you delete the container with still mounted
Apart from Docker the first tool we will look at is the gcc compiler. The key concept we want to impart here is the basic workflow to create a C executable or native library to be called by higher level code.
This is the basic pattern.
- C source code process.c -> object file
- C main application entry point source code main.c
- Link the object file to the main application and create an executable or library
This is the minimal set of commands to illustrate this
gcc process.c -c
gcc process.o main.c -o myApp
an important concept to impart here is the gcc command runs a binary the acts like a driver, invoking binutils to do the work to create the machine code and executable various other optimisations depending on the options we input to gcc from the command line.
Introducing the C Makefile
You can easily abbreviate the above steps however the goal is to point out the compilation and linking flow. This will quickly get out of hand for any realistic application. To automate this process we have the Makefile tool. In the previous article we saw how Java build tools have a task schema, the makefile follows this pattern. Below is a makefile task definition that will build the executable. It defines the task simplemake to build the code. By issuing the command make simplemake it will build the executable.
simplemake: process.c main.c gcc -o myApp main.c process.c
Run the Docker container
docker run -t -i eduonix/ctoolsgnu:1.0.0
Build the simple application with make simplemake, and run with ./myApp
With this very simple example we illustrated several fundamental concepts. With the Makefile and the GNU toolchain we built a collection of source and object files with command line arguments and generated as output a fully linked executable object file that can be loaded and run. The input files can consist of various code sections so the application can grow in complexity and broken down into units for development, then statically linked as part of the build process into a executable or library. it is hoped this first example by its simplicity will break down the flow if you are new to C and the GNU toolchain.
Creating a Native C library.
Shared C libraries can be very useful particularly if you are programming in a high level language unsuited for system programming such as Java or working in a resource limited environment like Android. We will illustrate with a very simple example how to create a shared C library we can call for a Java source code. we will focus on the build steps not the Java code. There are many tutorials on Java Native Interface JNI out there.
Using the source here we build a simple Docker Debian container.
docker build -t="eduonix/ctoolslibdev:1.0.0"
then we run the container
docker run -t -i eduonix/ctoolslibdev:1.0.0
the container opens a shell in the package base directory == /src build the Java code by at the prompt typing
generate the C header file by at the prompt typing
javah -d include com.eduonix.NDKDevPhaseTwo
change to the jni directory and compile the library source main.c into an object file
gcc -I/usr/lib/jvm/java-7-openjdk-amd64/include -I/src/include -c -fpic main.c
create the shared library
gcc -shared -o libmain.so main.o
copy the library to the package base directory
cd .. cp jni/*.so ./libmain.so
run the code that calls the library
java -Djava.library.path=`pwd` com/eduonix/NDKDevPhaseTwo
the screen shot below shows the expected output for all the commands
here is a link to the makefile to build the library based on the previous commands.
In this and the previous example by utilizing Docker to create the development environments we have tried to illustrate with simple examples how we can utilize the GNU toolchain to build native libraries and applications. The GNU toolchain is both complex and incredibly powerful. It can build the following types of applications
- C (gcc)
- C++ (g++)
- Java (gcj)
- Ada (GNAT)
- Objective-C (gobjc)
- Objective-C++ (gobjc++)
- Fortran (gfortran)
When we are working in a resource limited environment and cross compiling to a Mobile or Embedded system we often don’t want or need all the power of GNU. To optimese a build environment for a resource starved target system we need an optimised toolchain for the target system. A good example of this kind of application development is the Android Open Source Project or ASOP and its Bionic libraries and Android make build flow.
Building a Native C Library for Android
Learn Cross Compilation and Toolchain by Example
What follows is a quick example of how we can build a toolchain and use it to cross compile a native library for a target operating system that is different to the host operating system we compile on. the basic flow here breaks down into two parts. The host toolchain will be used to build a toolchain that builds programs/libraries that will wind up on the target.
Learn by Example
In the advanced module of the source code here a Android skeleton project using the android tool was built
android create project --target 1 -name eduonix_ndk --path ./eduonix \ --activity NDKDevPhaseOne --package eduonix
Then we can build a build a container and install the Android SDK and NDK and use them to build a development environment with using latest NDK toolchain.
docker build -t="eduonix/ctoolsndk:1.0.0"
We set up a toolchain to target a particular CPU architecture, For example arm-linux-androideabi-4.6 that targets ARM-based Android devices or x86-4.6 that targets x86-based Android devices. We can do this by running the ‘make-standalone-toolchain.sh’ script in the /build/tools/ directory. To build for the ARM architecture we would run the script with the following options
make-standalone-toolchain.sh --verbose --platform=android-21 --install-dir=toolchain-arm --arch=arm --toolchain=arm-linux-androideabi-4.9 --system=linux-x86_64
We will need to configure the toolchain for host operating system to run the build from the makefile. To do this we must set the paths for each tool in the toolchain.
For example when we set the environment variables
ENV CROSS_COMPILE arm-linux-androideabi ENV CC arm-linux-androideabi-gcc ENV AS arm-linux-androideabi-as ENV LD arm-linux-androideabi-ld
We set the target architecture, compiler, linker, assembler for the cross-compile toolchain, the official NDK documentation can be found here docs. With this environment setup we can now compile are native library for the Android operating system.
docker run -t -i eduonix/ctoolsndk:1.0.0
In our NDK home directory for our setup /Android/android-ndk-r10c we can see the toolchains available to build
When we ran the make-standalone-toolchain.sh script with the options
--verbose --platform=android-21 --install-dir=toolchain-arm --arch=arm --toolchain=arm-linux-androideabi-4.9 --system=linux-x86
we built the toolchain to allow us to cross compile for the target operating system Android from our host operating system Debian, we can see the components of our toolchain build in
To build for the target operating system we need to configure the makefile at a minimum you will need to set the CC flag
You can find more information for setting up the cross compile in flags and environment from the document below found in the ndk docs directory.
Running the makefile produces the following output
The very simple examples we have worked through are there only to highlight the use and workflow of C build tools, that is the Makefile and the toolchain. We have seen the flow needed to statically link source files to build an application or native library in the simplest possible way then we moved on and looked at building a tool chain for a cross compile workflow. Then the critical and often difficult step of setting up the environment for the cross comple flow, this is difficult as there are so many environment variables and paths to set. With a correct and functioning environment setup we ensure that the toolchain for are target operating system is in our build path importantly this will ensure environmental variables are set so the makefile does not erroneously configure itself for the host (i.e., Linux or Mac OS X) instead of the target.
Working with C and its build tools is complex but rewarding. If you are new to System programming this article will help you get started on the development path needed for these System programming related skills.
There is an alternative that is emerging in the System programming space. That is the Go programming language. In the next article we look at this new language that is both a fun and productive language.