Porting EiffelStudio to OpenBSD (again)

Years ago, I made an effort to get EiffelStudio running on OpenBSD 4.8. After I managed to get it working, Manu from Eiffel software was very helpful by integrating OpenBSD as an automatic build target into the Eiffel software build farm. This ensured that new versions of EiffelStudio would always compile on OpenBSD. At least in the 64 bit version.

Unfortunately, what was not done was upgrading OpenBSD itself. While EiffelStudio was still regularly built on (IIRC) 4.8, the development on OpenBSD continued.
security might be what is mentioned in the media, it is actually the operating system of software quality. Quality and correctness in OpenBSD are paramount, even to the level of release schedule and documentation. Security sometimes seems to be just a byproduct of this approach.

One result of this is that OpenBSD is rather flexible on maintaining backwards compatibility and old infrastructure. This has been very nicely summarised in a recent blog post by Ted Unangst (http://www.tedunangst.com/flak/post/out-with-the-old-in-with-the-less).

Old binaries are not supported to work on newer versions of the operating system. Some backwards compatibility is provided, assuming that you still have the old version of libc lying around. But between 5.4 and 5.5, OpenBSD had a major kernel ABI change (moving from 32 to 64 bit time everywhere, to be ready for year 2038 - https://en.wikipedia.org/wiki/Year_2038_problem). Old binaries do not work between these two versions, which made upgrading 'interesting'.

PorterPackage considered useless

Now, missing binary compatibility would not be a big issue for getting EiffelStudio to work: Eiffel software provides a great little thing called PorterPackages. These are tar-files that contain everything you need to build EiffelStudio on a target machine. And as an Eiffel compiler is not available to bootstrap, all Eiffel programs are already available as pre-compiled C code. A bunch of shell scripts is provided that will compile and assemble a nice little EiffelStudio deliverable.

BUT, just between 5.4 and 5.5, the OpenBSD people removed 'libcompat from the operating system. This library used to provide support for some old, deprecated APIs. One of these is 'ftime', a function that has been removed from POSIX since 2008. Instead 'time' and 'gettimeofday' are replacements for all platforms.

Well, except one: Windows. Because of this, my change request to move to the new API which would have made the PorterPackage work again on current version of OpenBSD was rejected (https://github.com/EiffelSoftware/EiffelStudio/pull/2).

So, there is currently no way that PorterPackages can work without libcompat, thus with a version > 5.4. Time for plan B.

It has to start somewhere

Fortunately, version 5.4 (from 2013) of OpenBSD is still available on FTP servers. I fire up a VirtualBox and install OpenBSD 5.4. Then I download the PorterPackage, GTK and other required libraries, and run the build script. Soon, I have EiffelStudio 15.01 running.

Next, I grab the actual EiffelStudio 15.01 source code. The goal is to get a version compiled on 5.4 that has all the source code changes for later version of OpenBSD. This has two parts: first, apply my patch from above to the checked-out tree. Next, remove all references to 'libcompat' from the platform specific configuration files. These are the 'openbsd-x86-64' and 'openbsd-x86' files in the 'C/CONFIGS folder. Also '/usr/lib/crtbeginS.o /usr/lib/crtendS.o' needs to be added when linking shared libraries.

Once this is done, and after making sure I have a current version of GOBO in my path (which I have compiled from source), I can start building a new delivery that should be the foundation of my future EiffelStudio package.

The EiffelStudio tree has a nice set of build.eant files that supports you in doing all the work. 'geant check_setup' makes sure you have all tools and paths configured. 'geant make_delivery' builds a complete delivery tree.

Once done, I have an EiffelStudio tree (EiffelXX) containing all the changes, it is just for the wrong version of OpenBSD.

Migrating to current OpenBSD

As a next step, I need to identify the parts of EiffelStudio that will require a replacement. The 'file' utility is very helpful to identify these files. What we are looking for are ELF binaries. So, the command line looks like this:

    find . -type f | xargs file | grep ELF | cut -d: -f1

The resulting list happens to be pretty short:

    ./studio/spec/openbsd-x86-64/bin/x2c
    ./studio/spec/openbsd-x86-64/bin/ecdbgd
    ./studio/spec/openbsd-x86-64/bin/ec
    ./studio/spec/openbsd-x86-64/bin/ecb
    ./studio/spec/openbsd-x86-64/bin/estudio
    ./studio/spec/openbsd-x86-64/bin/quick_finalize
    ./studio/spec/openbsd-x86-64/lib/libmtfinalized.so
    ./studio/spec/openbsd-x86-64/lib/libmtwkbench.so
    ./studio/spec/openbsd-x86-64/lib/libfinalized.so
    ./studio/spec/openbsd-x86-64/lib/libwkbench.so
    ./esbuilder/spec/openbsd-x86-64/bin/esbuilder
    ./library/gobo/spec/openbsd-x86-64/bin/geant
    ./library/gobo/spec/openbsd-x86-64/bin/gec
    ./library/gobo/spec/openbsd-x86-64/bin/gelex
    ./library/gobo/spec/openbsd-x86-64/bin/geyacc
    ./library/gobo/spec/openbsd-x86-64/bin/gelint
    ./library/gobo/spec/openbsd-x86-64/bin/gepp
    ./library/gobo/spec/openbsd-x86-64/bin/getest
    ./library/gobo/spec/openbsd-x86-64/bin/gexace
    ./library/gobo/spec/openbsd-x86-64/bin/gexslt
    ./library/net/spec/openbsd-x86-64/lib/libnet.so
    ./library/net/spec/openbsd-x86-64/lib/libmtnet.so
    ./library/vision2/spec/openbsd-x86-64/lib/load_pixmap.o
    ./library/vision2/spec/openbsd-x86-64/lib/MTload_pixmap.o
    ./library/vision2/spec/openbsd-x86-64/lib/gtk_eiffel.o

All the GOBO stuff can be easily replaced by binaries that are compiled using GEC, the GOBO Eiffel compiler. This is portable and works out-of-the-box with newer versions of OpenBSD.

The run-time system libraries (lib[mt]finalized.so and lib[mt]wkbench.so) can be compiled by using the build.eant script in the C directory. This is also true for the other .so and .o files, where the source code is available in Clib directories. Just make sure that you have 'finish_finalize' (a shell script) in your path. For each dynamic library (.so), there is also a static library (.a) which you should also copy over, too.

'x2c' and 'ecdbgd' are both written in C and built together with the run-time.

Now the only binaries left are: ec, quick_finalize, ecb, estudio and esbuilder. I did not actually bother with esbuilder, because I am not using it.

For all others, I have to build the source code, without C compiling it. This needs to be done on OpenBSD 5.4, before you upgrade to a higher version. For example, to create the quick_finalize source code, run:

    ec -config tools/quick_finalize/quick_finalize.ecf -finalize

I do this with all four binaries and then copy everything from my virtual machine (on Linux) to my actual OpenBSD box. Once there, I build the non-Eiffel binaries above and copy them over into the target tree.

Finally, I go into the EIFGENs directory and just run a 'finish_freezing' (perhaps after a 'make clobber') for each binary and copy the result as well. All done, and I have EiffelStudio for my current version of OpenBSD.

EiffelStudio on OpenBSD

The way forward

Ok - I have EiffelStudio working again, but it took a lot of fiddling. I have to develop a version of C_DATE that is compatible with Windows and OpenBSD. Once that is in the main tree and Manu continues supplying PorterPackages, things will be much easier.