Timeline

In July 2003, RTsoft finally decided to release the source code of the Dink engine, which was awaited for years.

Here's a what happened next, from a developer point of view -- DRAFT!

First projects

The announcement was made on the RTsoft forum ([1] [2] [3]) and after some discussion a permissive, free software license was used.

In the years before, D-Mods authors actively researched on how to take advantage of every possible feature and even bug in the Dink engine. With this release, there was lots of interest in adding features to the engine, fixing known bugs, and porting it to emerging platforms.

The code was considered pretty ugly ;) especially because it was very easy to break things, in particular compatibility with existing D-Mods, and because the code was distributed over only a few, huge files (> 10000 lines of code).

Christoph Reichenback (of FreeSCI fame) quickly put it on a public code forge (which offers services to free software developpers) to foster collaborative development. He also set up a classic directory layout, began splitting the big files in smaller ones, but quickly left the project in other new project members' hands - including yours truly :) At the time the code still only worked under Woe. The project was simply called The Dink Project (TDP).

Things got moving faster in September when Shawn Betts (of ratpoison fame) coded intensively to provide the first GNU/Linux port using the Simple DirectMedia Layer library (SDL). According to Shawn this was a quick, basically working port so that people could start actually hacking the engine. The technique used was to replace the subset of DX functions the engine used by a very similar interface written with SDL - which provides minimal changes in the engine, though while not improving the code quality and readability. This is essentially the opposite of what the rest of the team had done: seeking to clean-up the code first to better understand it and then improve it. Some helper files from MS were proprietary (you could not include them in a free software project), and there were either replaced by SDL equivalents, or rewritten from scratch (in the case of FastFile support). That way we got a basically working and portable version in a matter of weeks.

Interestingly, nobody dared to actually modify the engine or add real improvements. The emphasis was put on coding solid fundations instead. However, Dan Walma introduced user-level differences, for example a proper load_palette() function, and new math operations - which he would reuse later on.

Unfortunately, the code was fragile and some new and hard-to-debug errors were introduced. Also, since the engine was pushed to its limits, keeping compatibility was difficult, especially when it came to understand palette bugs in DX. Spotting everything that differs, for example the fact that sounds were not always played at the appropriate framerate, was usually only noticed by long-experienced (i.e. addicted) Dinkers. More generally, the job was not to port the Dink Smallwood game, but to port a versatile game engine used by hundreds of add-ons, which is quite harder to achieve. Lastly, there was little documentation on how the port was done, or what was left to do, and when Shawn got a new job and progressively left, we under-skilled programmers were left with a code base that we didn't understand much more than the original one - albeit this one was portable.

The development still continued. One of the problems I tackled was the filename case sensitiveness: under Woe, both A.txt and a.txt can be used to refer to a.txt, while under Unix-like systems only a.txt can be used (it's case-sensitive, i.e. it treats caps and non-caps as different). So a wrapper was introduced around file access functions to simulate case-insensitiveness with case-sensitive file systems. This allowed us to avoid user-unfriendly work-arounds, such as storing the game on a FAT partition, or renaming all the data files as lowercase. The code was also tested under GNU/Linux and FreeBSD.

We all much appreciated to get control over the Dink game engine. We chose to license our improvements under the GNU General Public License, as it ensures the new engine will remain free (as in freedom) for all users, forever (see copyleft for more information).

FreeDink

My involvement ended in November 2003, due to a bossy attitude from the project admin. I then posted a message to the mailing list stating I would work on the code separately ([2]).

That was the birth of FreeDink. The project was registered at savannah.gnu.org - another code forge, different in that the code running it (Savane) is freely available; with the code, there's no lock-in: you can move out of Savannah and host your project yourself anytime you want.

Since Savannah is part of the GNU project (the origin of the free software movement and quality pieces of software such as GCC, Emacs, Gimp and Gnome), the approval process offers the ability to apply for the package to become an official GNU project. FreeDink was submitted for GNU Evaluation along the way, and the project was dubbed GNU after a one-month review.

FreeDink started with an emphasis on code clean-up, in particular a port from loose C++ to ANSI C. It was about stability, compatibility and foundations for extensibility.

FreeDink was a one-man project. I wasn't sure where it would go, nor how fast, so it was kept quiet: no announcement, no big news, still the project remained in a public CVS repository, so that the code would stay available even if the project stayed unfinished.

Which it remained.

Slow-down

Shortly after, GNU Savannah was compromised. In addition the previous volunteers team left for various reasons. In February I started spending time helping with Savannah administration, which was pretty different since it's a sysadmin job rather than a programmer job, and FreeDink was put aside for a while (I'm still a Savannah admin :)).

Meanwhile, The Dink Project came to the conclusion that they could not work well on the code, for all the reasons stated above. They had all the code history removed, and started a new project from scratch, intended it to be an unofficial, next-generation engine for D-Mods authors with a similar 2D engine to Dink. They called it Windemere, after the name of a town in the Dink Smallwood game. A lot of code was needed for the new fundations of the engine - something that was already available with the original Dink source - so nothing actually usable came out in time. This project progressively slowed down and was eventually deemed dead.

New code bases

Nearly 2 years after the initial source release in June 2005, no project was actively developing it further anymore. At that point, Seth A. Robinson "asked [Dan Walma] to organize a v1.08 release of Dink" with the help of the Dink community ([3]).

Independently, later that year FreeDink got a second life. As stated, it was difficult to know what was ported and working (or not) in the latest TDP/FreeDink code. As a D-Mods wannabe author, yours truly decided he knew the original engine more than the current port, and restarted the port from the original sources, with an emphasis on documenting every step: the problems, the solutions, and the reasons behind choices.

Emphasis was yet again put on getting solid basis. This started with the compiler: the only gratis way to compile the code was to rely on an old free version of VC++6 from a magazine bonus CD. Moreover, the executables it produced were not legally redistributable. FreeDink was hence first ported to the free MinGW (Minimal GNU for Windows) compiler, which includes GCC, and quickly combined both DX and SDL at once, progressively switching from the former to the later, with careful testing between each steps. VC++6 was invaluable at that point in desambiguating particularly weird C constructs (such as comparing a char with a string) that were used in the game, and would not compile with MinGW. The game still only worked under Woe.

The SDL port started with the most independent part of the game: the sound subsystem. This required some research on how to play sounds at variable sound rate (more on that later), which SDL_mixer does not support. The world of digital sound processing (DSP) is pretty interesting and game programmers will probably find it worth a glimpse. The code from Shawn Betts came in handy, being a good example on how to use SDL. The documentation describing game libraries grew larger, including details that showed which ones could be used to (re)implement each Dink feature.

However, FreeDink had remained a one-man job, and still quiet. It's not surprising it hibernated once more for the next two years.

In March 2006, Dan Walma put together the final release of v1.08, which became the canonical version. The new version was actually ready and usable (unlike FreeDink). It was not portable, but it brought immediate improvements. Its official recognition from Seth completely changed the question of compatibility, among others. One could only wait to see how this would shape the future of the Dink community.

FreeDink resurfaces

We're now in August 2007.

Now was time to think seriously about v1.08. What interest would there be to work on FreeDink?

1.08 was still working only under Woe, and did no progress toward portability or code clean-up. So FreeDink still had its place. Now should it be a continuation of the 1.07-based work from 2005 that would integrate the 1.08 changes in a second step, or restart (again) from the original source code, only the next version of it?

A point was crucial. 1.08's design decision with regard to compatibility was surprisingly innovative: since keeping backward-compatibility while improving the game engine was very difficult, why not simply break the compatibility and fix the existing D-Mods themselves? There weren't that many D-Mods, and they were all available from a central location, so that was sounded reasonable. While backward-compatibily was one of FreeDink's primary objectives from day one, this solution was never envisioned.

Now, a year after the new release, the situation was mitigated: on the one hand, that permitted to make a release faster, and made possible to get rid of bugs that were used as features by some D-Mods. On the other hand, there was this "D-Mod Compatibility" page in the forum indicating progress in fixing existing D-Mods, and lost of them remaining unfixed, or worse, untested.

Hopefully this situation will improve, but right here, right now, there's a need to play existing D-Mods under GNU/Linux, there was a need to make a version of Dink that would be portable and backward-compatible. So FreeDink continued from its 2005 state.

Porting to SDL

Thanks to the project documentation, development quickly took on, starting with compilation environment, continuing documentation, and porting the tiles and sprites subsystems to SDL.

Understanding how palettes worked in DX was a very long process. What Shawn had done was simply forcing black as index 0 and white as index 255 (which is the opposite of the Dink reference palette) in the DinkC fillscreen(), but I didn't want to unknowingly break features (which I would have, in particular in the fade-to-black effect). I even wanted to reproduce the palette change work-around that could be tricked with the DinkC copy_bmp() - and I did. To the least, this allowed greater understanding of the engine, and good deal of documentation about it. Wherever FreeDink would end, this would still be useful.

An effort was made to separate code into independent subsystems - that is, get better encapsulation. The more globals you have, the more likely they can be used in a buggy way. It's safer to make the variables private (static), and only accessible through a small number of functions - that's especially useful for testing. Object-oriented programming is based on this concept - group code and the data it manipulates. It's not that easy to achieve when the code is already using the variables in all parts of the engine, so this requires some clean-up.

Next was the input system. It was disappointing to find there was no way to get a consistent keyboard layout across platforms (e.g., the "1" in a US layout would be "&" in a French layout. Also, letters codes may refer to the US or local layout depending on the current platforms). Not that it was the case in the original engine; it was particularly messy in Dinkedit, where non-US users would have to try and test every key to find where the [ and ] keys were mapped - but users did get used to that mess. Sadly there was no way to reproduce it, so the best we can offer right now if suggest users to switch to a US keyboard layout when using Dinkedit. In the game, keycodes were also used to attach script to given keys (say, "C" for cheat). That could be reproduced using Unicode keycodes, which are consistent for letters. This won't work if a D-Mod attached a script to a non-letter key, but this probably rarely happened. Last, all numerical key numbers were replaced by the matching SDLK_, which was somewhat tedious ;)

This kind of limitations raises the question of to which extend one should seek compatibility. In particular, FreeDink won't try to achieve compatibility for engine behavior that was never exploited in a D-Mod (or in the original game). For example, the original engine inefficiently loads the reference palette several time from different files on startup, which gives a change to make it load different successive palettes and achieve particularly weird/buggy effects. This was never used to instead of reproducing this, the code was cleaned-up in FreeDink. Any D-Mod exploiting this after the release of FreeDink would simply be deemed non-portable. It'll always be possible to make non-portable D-Mods even with a portable engine, so we are to expect a few good practices from fellow authors once a portable engine actually is available.

Woe was progressively wearing me off, so with graphics and input in a workable state, I hasted some more toward code that would work under GNU/Linux - I kept better understing how Shawn worked, and copy/pasted some of his code for font processing without as much as precise checking I done until then. Soon I could compile under GNU/Linux with few portability issues (much of the portability lies in SDL), and I re-implemented the case-insensitiveness code I had done 4 years before (time goes by...).

At this point the engine was basically working and could be tested with the original game and even D-Mods. A couple people actually saw the recent changes on FreeDink, gave it try and gave useful feedback. Notably, v1.07 ties game logic and frame rate and suffers from variable game rate, which means the game speed would vary greatly between computers, and became unplayably fast 10 years after it's initial release. FreeDink followed the official v1.08 in setting a maximum limit to the game frame rate. This highlighted the fact the v1.08 framerate is still somewhat variable due to implementation issues, so in FreeDink an average value for chose, and properly fixed using the nice framerate control functions from SDL_gfx.

People wondered about a release. But the engine wasn't this much complete yet. It might be prematurate to announce an unfinished engine, with some probability it would stay that way.

Incidentally, the game is still meant to be portable, but instead of maintaining per-platforms development environment, I looked into cross-compilation. MinGW, being a version of GCC, can cross-compile Windows executables from GNU/Linux, faster than natively compiling, and SDL has long supported it too, so quickly I could build Linux binaries as well as .exe without rebooting :)

This was also the opportunity to setup state-of-art autotools build system in a project. Things can get scary sometimes, but releases, cross-compilation and dependencies tracking then become so easy that it's really worth the effort.

Still in para-development topics, FreeDink moved from the CVS version control system to the newer Git, which allows decentralized development - and is very fast.

I then came back to the game itself, bought a gamepad and implemented joystick support. I also needed a trick to implement the game mouse mode were the mouse is blocked at the corners of the window, while still using relative positioning in the "Load Game" dialog. With those, I started implementing features as separate stand-alone tests that can be easily stress-tested, and integrate them in the engine once they are stable enough.

Now was the time to deal with the nasty remaining details.

Search path

In Unix, executables are usually installed in /usr/bin, while data is stored under /usr/share. Unix also clearly separate privileges (admin and non-admins): it would make sense to install Dink read-only for all system users, while allowing them to install D-Mods in their personal directory (they wouldn't have the privileges to install D-Mods in the system-wide directory owned by the administrator). By contrast, Dink installs all executables in the same directory, and only look for D-Mods in a single directory. Adding more search paths for D-Mods (and by extension, to other data such as saved games) hence had benefits, to the least under Unix.

At the same time, it wouldn't be good to sacrify portability for integration, so the new search paths needed to be consistent across platforms. Coming up with a simple and consistent solution was not really easy. Complexity was mainly introduced by the fact FreeDink needed additional data that were usually provided by the system (the Arial font) or embedded as PE resources (icon and the two Dinkedit sounds). I eventually solved the issue by implementing embedded resources independently of the platform, using a technique similar to self-extrating zips (SFX).

Fonts

It was pretty frustrating so have a different size for the fonts, especially for the editor help, as it usually was too big to fit in the screen. Fonts are surprisingly complex, and research was done to better understand how they worked. Eventually, the solution was simple to implement, but required much reading and testing. So two different fonts in used: Arial.ttf (dialogs) and vgasys.fon (editor, debug). A metric-compatible font called LiberationSans from Red Hat is available as a substitue for Arial, although contrary to what it may sound, it's looking pretty different. Wine has a free replacement for vgasys.fon. Font size is not based on the same property under Woe and in FreeType, looking at the Wine source code was much enlightening. Font rendering differ greatly in FreeType depending on which algorithms (patented or not) are in use. The patented hinting is necessary to achieve nearly-exact graphics compatibility with the engine - it's pretty impressive to see how far the FreeType people went in reverse-engineering, and it's too bad that companies such as Apple and Microsoft use software patents in an aggresive (not defensive) way. Because of such threats, the faithful patented hinting is disabled by default, and while some distros enable it in their binary packages (Debian), a fair numbers of them do (Fedora, Gentoo pre-built).

Variable sound rate

When you pick an item in the game, it makes a quick "chop!" sound. But the sound is actually sword2.wav, only played about 3 times faster, producing a high-speech effect. Such effect can also be used to slow down the sound playing, resulting in a grave sound (e.g. the DDC growl in Lyna's story before entering the Shadow World for the first time, which is the girl scream played slowly).

Numerous engines provide it (OpenAL, audiere...), but no engine provides this and MIDI support. As mixing sound engines didn't sound like a good idea, especially for a portable engine, variable sound rate support was eventually implemented on top of SDL_mixer, feeding the mixer with fake sounds of the right final length that got overwritten by channel effects. The naive implementation (playing one frame out of n) distorted the sound somewhat. It appears this can be dealt with using numerous algorithms, as implemented by Secret Rabbit Code (a.k.a. libsamplerate). I chose linear interpolation, which is simple and fast, and implemented in the Allegro game library, so I had some existing game code to look at. Gladly this now works pretty well.

Future

[This section needs an update, basically v1.08 mode and translation support are finished.]

Next step is adding the v1.07->v1.08 changes, and possibly others changes (another scripting engine and translation support comes to mind). This will probably be a separate version, where compatibility won't be so important. We could also work on making other programs portable (WinDinkedit comes to mind).

Maybe we'd have to maintain 2 versions ("classic" and "new"). That depends on how easy we can upgrade the existing D-Mods.

Can the engine be extended to make any 2D game? Well, just imagining mixing free-scrolling and screen-based 2D games sounds nightmarish enough already. Maybe it could be a collection of loosely related engines. FreeDink will probably rather aim at improving the engine rather than changing its fundations.

But remember: FreeDink is free software, and you're welcome to change and redistribute it under the conditions of the GNU GPL - please experiment, and possibly your improvements will be integrated in the next version of FreeDink!

Thanks to Paul Jewsbury for proofreading this text.