Written: 2023-12-28
Revised: 2024-01-08
A small, easy-to-deal-with build system for C
This is not an ad. It's more of an in-progress experiment.
You see, I've changed build systems many times for my personal projects. In the beginning, I used the suckless strategy of just using a Makefile. If you don't need to support Windows, and don't have any dependencies, and support PREFIX and DESTDIR correctly, and don't need to compile other programs in the middle of compilation, this works fine.
But at some point, I wanted to use SDL, and POSIX make doesn't support shelling out to pkg-config, nor build options. Also, the support for automatic header dependencies is not very good, and usually gmake-specific. Since I don't want to depend on gmake, I started considering other options.
I had used some systems from a user's (or packager's) point of view:
- plain Makefiles are the best, if well done. They are hard to do well when you have more than a few source files, options, or dependencies. Anything more complicated quickly gets tricky (and potentially fragile).
- autotools probably have the best interface: a shell script with standard options. When they work, at least. Configure scripts tend to be very fragile due to their complexity, and porting to a new system usually involves editing arcane, poorly-designed scripts.
- hand-written configure scripts, as the musl libc, cproc and some other projects do, are nice if all you need to configure is a cross-compiler, and possibly some paths or compile options. They also work fine for dependencies; shell out to pkg-config and put the result in a variable in config.mk. When well-done, pretty great from a user's perspective, even on weird environments. When poorly done, can generally be easily edited.
- CMake is very bad. When it works, you need to awkwardly run it from inside another directory, and when it doesn't, fixing whatever stupid tricks the (usually Windows-first) build scripts are doing[a] usually involves changing the scripts. CMake users also have a tendency to make things over-complicated[b], which makes it trickier to replace the build system wholesale, which might otherwise have been a solution on the extreme case where nothing works.
- Meson is better than CMake. Its dependency on Python used to be a bigger problem, but when you don't have Python, you can now use muon instead. It doesn't usually cause problems at least.
[a] Generally, vendoring dependencies with no override.
[b] Usually for the sake of working around the build system.
I also had the pleasure (or, in some cases, displeasure) of using some of these from a developer's point of view:
- plain Makefiles have some issues with dependencies. They aren't usually a big deal. Out-of-source builds are really nice to have, though, and so is not having to write the same template every time... Mostly, I miss convenience features when using Makefiles.
- autotools are really hard to use. They like to break between versions (fragility, again!) and are generally unpleasant to work with. They also inherit most of the problems of plain Makefiles.
- hand-written configure scripts are okay. They have no disadvantages over plain Makefiles, except having to write them, and that's usually really easy, though somewhat time-consuming, especially at the start of a project.
- CMake is very bad. It is designed to generate build files for IDEs, so it's strange to use from the command-line, thus the awkward out-of-source builds. It's hard to figure out what is the "right" way to do anything, and other CMake scripts are full of hacks that will break on certain specific configurations. You need to know where stuff is installed, or use a FindThing.cmake script, or depend on PkgConfig.cmake. These are mutually exclusive. Or you can manually add an option for the user to choose! Fun.
- Meson is better than CMake. It's not well-documented enough for certain advanced uses, and the other implementation (muon) doesn't implement many of the fancier features, so if you want to be fully portable, you have even less features, with even worse documentation. It's probably usable enough, but I wouldn't know. Also does the weird out-of-source builds.
- Premake has command-line use basically as an afterthought. It's from another culture, and I, a Unix junkie, don't like using it.
You can see I really don't like CMake.
Hand-written configure scripts are probably the nicest solution overall, but writing them is a bit of an art, and somewhat inconvenient at the start of a project. One variation I tried was generating a Ninja script instead of a Makefile, and it's unquestionably better: faster builds by default, automatic dependency resolution, easy out-of-source builds. I learned a lot by doing that.
Probably the biggest lesson I've learned from building things from source is that everything should be user-overridable, preferably without editing anything. Dependencies, compilers and paths should be provided by the user, not "found" by the scripts. If the build system wants to have reasonable defaults, that's okay, as long as they're well-documented and easily overridable.
Another thing is that the best way to do out-of-source builds is not to put the build system's files inside the build directory. That forces you to specify it on every build command, or switch to that directory first, which is inconvenient. It's much nicer to have the build script on the project directory, and have it know where the build directory is. This also allows final binaries to be put in the current directory, which makes them more easily accessible.
Windows support is nice, but on that platform, nothing is standard. What most other meta-build-systems seem to do is to either vendor dependencies or expect them to be installed. Or use vcpkg. vcpkg is designed for Windows, but on most build scripts it's usually set up to be used on all OSes. Don't do that. It breaks badly on Alpine, and probably other systems as well.
Ideally, I'd like to specify some dependencies, add some targets made from source files, and have the build system handle the rest. You know, like cargo and go, but without forcing any particular source directory structure.
So, I made a proof-of-concept. It's currently an AWK script (cconf) that generates a shell script (configure) that generates a Ninja build file (build.ninja). It's not perfect, or complete for that matter, but it can be improved over time.
The basic usage is to write a build.config file:
default release
bin foo
foo deps sdl2 SDL2_image
foo cc foo.c sub/bar.c sub/baz.c
And then run:
cconf build.config > configure
chmod +x configure
To get a configure script that can be used with the almost-standard:
./configure
ninja
A few interesting features:
- all the steps run very fast.
- there is no dependency on cconf by the resulting configure script, so it can be distributed with the source, and an end-user/packager doesn't need cconf to build.
- the generated configure script is actually quite readable.
- the resulting ninja build script puts intermediate files in a build/ directory, and the final results of compilation in the current directory. This keeps the source directory clean from object files, but the final binaries easy to find.
- using Ninja instead of Makefiles as backend means automatic header dependency handling, which is really nice during development. For platforms where the C++ version of Ninja isn't available, samurai works as well.
- the generated build.ninja supports debug and release builds at once. You don't need to reconfigure to go from a debug build to a release build, or vice-versa.
Simplifications/anti-features:
- Windows builds are supported through either cross-compilation or MSYS2 (and, potentially, w64devkit). No attempt is made to support MSVC or any IDEs, as that would require a polyglot configure script generating multiple config files, which would be a massive pain to support.
- C and C++ compilers are found by finding a gcc or clang binary on the PATH. No further tests are made. If this is the wrong compiler, or it's not found, you can specify it manually by setting the CC or CXX environment variables.
- dependencies come from pkg-config exclusively, no attempt is made to install or vendor them automatically.
Limitations of the current prototype:
- it's written in AWK, so the build.config syntax is very limited. There is no way to support spaces in filenames inside the source directory, for example. This may or may not be worth changing; AWK is surprisingly appropriate for this approach.
- there's no way to override dependency paths. This shouldn't be hard to implement, I just haven't needed it yet.
- only C and C++ are supported, and there's currently no way to extend that. Not even preprocessors.
- there is no way to build libraries.
- there is no way to vendor dependencies, though this is mostly useful on Windows, which has no standard package manager. It's also useful for custom versions of libraries, so it's a good idea to have it anyway.
- flags handling isn't very good. It would be better to not specify flags, but warnings/language version and debug/release settings separately.
- cconf is not re-run automatically when build.config is edited.