Adding an event driven architecture to ADOM...

Over Christmas (Merry Christmas to you all - hopefully you had one... I was too busy to post anything ;-) )... ok, I am starting again: Over Christmas I finally had a vision for a complex architectural question that bothered me with ADOM for a very long time. A question so important to me that it originally caused me to start writing ADOM II (JADE). Read on to hear a bit about fascinating new venues that are opening... and have been revitalized by my thoughts about what we are going to do with ADOM in 2016 - team play being one very important thing very dear to me right now... but I'm digressing again ;-)

Regarding the complex technical questions mentioned above: Distributed games are characterized by the fact that separately running instances start to interact with each other. Sometimes in simple ways (e.g. displaying a chat message from another player), sometimes in more complex ways (e.g. actual multi-player gaming where people share a map and the map context).

ADOM always was designed to be strictly single-player (and no, this probably is not going to change... at least not towards multi-player). While pondering the major tasks for 2016 (another blog post will elaborate on this) one topic became more and more important to me - especially due to our new presence on Steam and the new communal spirit (and technical possibilities) arising from that: I e.g. had been thinking quite a bit about team play aspects for ADOM (and I think I have some very good ideas there).

The biggest technical hurdles contained in ADOM to such an endeavor are twofold:

1. Internally ADOM uses a so-called REPL architecture (which is quite common in strictly single-player games): Basically this means that ADOM reads some input, evaluates it in the given context and prints the results - all in a loop. If this sounds too theoretical: Just imagine that you press the "move left" key, ADOM reads that, evaluates the pressed key combination to the instruction of moving the player one position to the left (which effectively might cause him to activate a trap or attack a monster entailing all kinds of complex interactions) and finally prints the result by updating the screen, the map, etc.
2. ADOM has been built for the past 20 years with this REPL architecture in mind. Effectively there are many code dependencies, assumptions, special cases and what not in the program code (more than 220,000 lines code  of it these days) that assume REPL. Just take the following example: Initially ADOM didn't have the concept of speed. So the player moved once, all monsters moved once. Then speed was introduced and suddenly movements (and resulting actions became less ordered). Still the ADOM code maintained the strict REPL architecture and you don't really want to imagine the pains that were taken to allow for a more complex interaction sequence. And that's just one example or the internal challenges presented by being REPL for so long.

So for a long time my firm conviction was that it would be impossible to change the REPL architecture. As many things become very hard to implement due to the REPL nature the game I at some point started designing and implementing ADOM II, which from the very beginning used an event driven architecture with timed events and thus already in its current primitive state allows tons of stuff that is very hard or impossible to do in ADOM (e.g. flowing and expanding lava, complex animations, mana waves activating items, etc.).

But things have changed in regard to ADOM. Many puzzle pieces now are coming together to a very interesting juncture in the lifecycle of ADOM:

• When we integrated NotEye (mostly done by Zeno and Jochen) we among other things added an event system to exchange information between two rather disjoined pieces of software. While that event system is very classic and simple (e.g. standard fare in most UI frameworks) it nonetheless was something new for ADOM as we started communicating somewhat asynchronously and had to synchronize at various points.
• While adding the event system Jochen implemented a very neat JSON infrastructure in ADOM which by now also is used in various other places. This finally gave us a system independent and very flexible data infrastructure that is far more flexible than the very low-level byte oriented structures used before.
• While extending ADOM after our successful crowdfunding in 2012 we refactored many parts of ADOM. And while there are still tons of ugly and dark places, the overall state of the code is slowly improving.
• ADOM is written in C - a non-object-oriented language. While this is not a problem per se, it makes implementing many things much harder or far less elegant than they are in languages like Java. Thus we always have been hesitant to adding complex additions to ADOM. But we have started being bolder with the C code we have been writing and it has become more modern, even OO like in some places (with all the added complexities of trying to write OO code in a non-OO language). Nonetheless ADOM is slowly becoming something that resembles something less than a brutal force of nature and more like a designed and planned engine.
• I (and also Jochen) have gathered an extra 10+ years of experience as a senior developer/architect for very complex software solutions. So we have become wiser, braver and much more experienced and nowadays dare to tackle challenges that were far beyond my/our ken 10-15 years ago. Also our knowledge of the ADOM code itself has once more improved drastically over the past four years (lots of stuff was forgotten or lost to time during the period from 2002 to 2011).

The roadmap resulting from the crowdfunding campaign also mostly covered extending the current nature of ADOM rather than introducing truly new innovations. This changed for me personally when we finally managed to launch on Steam on November, 16th, 2015. Suddenly we are part of a community with extensive multi-player offerings, chat support, achievements, network infrastructure and so much more. Since that day I have been pondering true innovations for ADOM. The current - IMHO still pretty lame - crowd mode is one such shade of a good concept. While I still believe that something like multi-player probably never will be possible for ADOM I have become enticed with the idea of a team oriented play mode and I have very specific ideas for that. But those ideas require a different architecture than what ADOM currently has with its REPL system.

And then I suddenly had a technical vision for an event driven architecture for ADOM during the Christmas holidays. Specifically an idea that wouldn't require rewriting tens of thousands of lines of code but rather would allow for a much more gradual introduction of an event driven architecture into ADOM. An architecture that could evolve over time.

I started prototyping it three days ago and a couple of thousand lines of code and a few dozen experiments and refactorings later I believe that my current prototype for an event based architecture is going to work and will be the foundation for future versions of ADOM. 

The current state is:

• My local prototype no longer has a REPL based architecture. Instead it has various event queues with timed events which work somewhat similarly to the ones implemented in ADOM II (see e.g. this ancient blog post for things they enable; I even did a conference talk about it).
• I have an event system with event handlers, event data structures and synchronization functionality that enables asynchronous communication between independently running ADOM instances.
• I have a time based event system that will allow things that have been outlined and implemented for ADOM II (like e.g. rolling boulders, energy waves, flowing liquids, complex timed story events and so much more).

And what's missing? Quite a bit but nothing that is beyond being solvable (as far as I can see right now - keep your fingers crossed):

• Currently none of the event queue infrastructure is stored in save files (which it should be although it even would not be necessary in its current state as there are no long running in game events and similar stuff so far - only code to allow the player and the monsters to act as well as time handling stuff). This is a matter of hours and should be done pretty soon.
• The existing ADOM code still is tied to its REPL legacy - meaning that many things that could be implemented in small, modular and easily maintainable code modules/events currently are hidden in complex, convoluted and extremely hard to understand legacy code swamps. But this is something that can be changed gradually over time - e.g. whenever some legacy code piece needs to be rewritten it as well can be extracted, refactored and structured according to the new architecture. This probably will take a couple of years (if it can be completed ever) but it's not much of a problem either. Quite to the contrary - now there is a structured roadmap for things to gradually become better instead of degenerating further.
• We need more networking code - basically to broadcast the new events and send them point to point. Broadcasting mostly will be useful for things like an internal chat system, point to point communication will be helpful for lots of new in-game innovations (e.g. just imagine a team mode where players can exchange items or maybe even trade mana or hit points and similar stuff - maybe at a price ;-) ). Here we can build on the existing ghost exchange system that we created for the Steam release and that already is running very well causing players to face vile spirits created in other players games.

What benefits will this entail? Basically you should search many of my older ramblings when I started working on ADOM II (JADE). The new system allows for complex timed interdepencies, generally the code becomes more modular and easier to maintain and features that have been very hard to do (if at all possible in the past) now become rather easy to add. Think of in-game features like:

• recharging items
• maturing monsters
• hatching eggs
• flowing lava or water
• geological events (I have certain ideas for the volcano quest :-) )

And think of team play features (no more spoilers for that - I'd love to surprise you all at some point in 2016).

As you can see I am currently quite excited and will dive back into working on the new event based architecture immediately. It beckons of a future full of real innovations I never dared to dream about for ADOM :-)

Read ya soon!

P.S.: I have no idea what this means for ADOM II. For now I'll work on backporting the good architectural ideas from ADOM II to ADOM... and then we'll see what the future holds next.