Several weeks ago, a redditor started sharing some of the work that he has done regarding a potential port of S.T.A.L.K.E.R. (Let’s call it Stalker from now on) for Linux, following a leak of the source code last year. In case you are not familiar with Stalker, it’s a survival-shooter/RPG game set in the outskirts of Chernobyl following a second (fictitious) radioactive incident.

It featured a non-linear storyline, and additional gameplay elements such as clans and factions making the experience unique about every time you play the game. Just like many other Windows games of the time, it used DirectX as a graphics API and there was never any real Linux port it.

The latest update is looking very, very good already. It’s not perfect yet, but the rendering is now much more complete than in the previous shared video.

https://vimeo.com/144270704

The most recent update to the OpenGL backend now includes the following features:

• Steep Parallax [ wikipedia ]
• Shadow Mapping
• View Distance Fog
• (…and many bug fixes)

The current build also partially implements:

• Dynamic Lighting
• High Dynamic Range
• Particles
• Decals

and there is still more work ahead, with rain, sun rays effects, anti-aliasing, anisotropic filtering… but as you can see for yourself, things are wrapping up very quickly.

The conversion from DirectX to OpenGL is currently the work of a single man, Jules, 25 years old, from the Netherlands, who graduated following a specialization in Computer Vision and Computer Graphics. What brought our friend to follow the path he is currently on, has actually a lot to do with the emulation scene.

Jules: During my graduation I was looking for ways to get more experience in graphics, so I joined the Dolphin Emulator project. In this project I spent my time debugging graphical glitches in the emulation and adding more graphical features. The experience I gained there has been extremely valuable to the project I’m undertaking now.

Jules used to be a Linux user, while he has switched to other systems in the past few years, for very practical reasons.

Jules: On the Linux front I have used it as a main OS in the high days of Ubuntu (I refused to switch to Vista), but I eventually switched to Windows 7 for the games and hardware-accelerated video decoding. I still often use Linux as the main OS on my servers, but I haven’t used it as my desktop OS in a long time. I spend more time programming than I spend on gaming, so what I’m mainly looking forward to in Linux is the operating system itself. I just want to make sure I can take my favourite games with me.

But we have not lost him forever - there is a good chance that he considered a come-back to the Linux platform:

Jules: However now that gaming and hardware-accelerated video decoding have come to Linux I finally decided to switch back, probably to the Elementary OS distro. I’m not only a Windows user, I am also a little bit of a Mac user. With ElementaryOS basically being Linux’ answer to OS X got me interested in using it.

The fact that Jules is a gamer certainly did not hurt to help him find motivation on this project. He clocked a lot of time on Stalker as well.

Jules: I like atmospheric action RPGs, first person or third person. Stalker and Dark Souls are my favourites. I’ve put more than 250 hours of playtime in Stalker and I still come back to it for another play-through. I like the atmosphere, the sense of isolation and survival. There’s also the mystery that surrounds the game, even the bugs in the games seem to contribute to that sense that anything could happen.

For example one oversight in Clear Sky is when I found the developers left a gate open that was supposed to be closed. If you go through the gate and start exploring around you’ll eventually come across a parking lot where stepping on certain parts will teleport you to a seemingly random place in the zone and in the story. It’s like the engine created a space anomaly the developers didn’t even account for.

So here you go: combine a fresh out of University graphics programmer with the right skills, a strong interest in open-world action RPGs, and a readily available source code for Stalker… all elements were in place for a spark of some sort.

Jules: When I came across the leaked Stalker Clear Sky source code I started thinking if there was anything fun that I could do with it. Working for the Dolphin Emulator project gave me lots of insight into how these graphical APIs work and how to debug them. However, I wanted some more experience building something from the ground up. So I decided the writing an OpenGL backend was the perfect opportunity to do something fun and challenging while getting more experience in computer graphics.

There are several ways to go about converting DirectX code to OpenGL - either you create tools to help you automate most of the conversion and then fix potential bugs as you go, or you go through the painful work by hand. Jules weighted both options based on what was already done out there, including Valve’s work.

Jules: It was a difficult choice between doing things by hand or automating them. Valve has done a lot of talks about how they converted their Source engine to Linux, so I like to take those as a reference. There are basically two major components that need to be converted: the DirectX function calls and the HLSL shaders.

If the engine doesn’t offer some abstraction between DirectX and its core the only way to realistically port the engine is to offer a compatibility layer where you re-implement DirectX functions with OpenGL functions. Valve has used this approach for their Source Engine with their toGL project, it has the advantage that you don’t need to do a lot of conversions by hand, the downside is that the compatibility layer is very complex and it becomes difficult to add modern DirectX 10 features.

Next you’ll need to port the shaders written in HLSL to GLSL. The two languages are very similar, so writing shaders that are compatible with both languages is just a matter of using a set of macros and holding yourself to some rules that ensure your code is portable between the two languages. This is the approach Valve has used for Source Engine 2, they opted that this was the easiest way to do cross-compatible shaders.

However if the shaders were already written in HLSL with not account for portability it becomes very tempting to automate the process using translators or cross-compilers. The advantage is that you immediately get GLSL shaders that nearly perfectly function like their HLSL versions. However at the same time it becomes extremely difficult to debug them, so you have to weigh in the time spent on debugging issues before you decide to use a translator or cross-compiler. This approach is being used by engines such as Unity and the Unreal Engine. The Unreal Engine has the best cross-compiler by far and while the source code is available the Unreal License does put a lot of restrictions on the usage of it.

In the end, the design of the original engine behind Stalker helped Jules make a proper decision to move forward.

Jules: For this project I decided to convert the function calls by hand. While the X-Ray engine used in Stalker is not the prettiest code repository, it does offer a good abstraction between the DirectX functions and the engine core. This is a legacy from the fact that they had to support many different versions of DirectX throughout the development of their engine. The X-Ray engine has used backends ranging from DirectX 8 through DrectX 11! So doing the API conversion by hand was mainly a matter of making a copy of the DX10 backend and rewriting it for OpenGL, so there was no need for a compatibility layer.

Next came the shaders, this was a difficult choice. The shaders in the X-Ray engine have all been written in HLSL, with no account for portability. So it would make sense to use a translator or cross-compiler here. However since I don’t have any experience porting engines yet I found it more important that I could easily debug my mistakes, even if that meant I had to spend some time converting the shaders by hand and introducing more bugs due to oversights in the conversion.

He was able to progress faster than he had originally anticipated - since he expected more nasty bugs to hinder his work. So far, so good.

Jules: I anticipated that I would hit a brick wall along the way that would take me many months to debug. And while I was met with a nasty display driver crash that took a few weeks, generally progress has gone smoothly. I also overestimated how long it would take for me to port the shaders by hand, turns out of the initial 200 I expected to port there were only around 50 actual shaders the others were just variations of those 50.

Debugging this kind of work can be very tricky, because in the end it really depends what tools the graphics card manufacturer brings you to help you find what’s wrong with your implemented functions and shaders. Jules chose to work on NVidia since they have very decent tools for that purpose.

Jules: Nvidia NSight is a very powerful graphics debugger, these kinds of debuggers are the reason that things such as native engine ports can even be done by a one-man team. It is second only in easy-of-use to the Visual Studio Graphics Debugger which is DirectX-only. With it I can look through every facet of the graphics engine even if all you see is a black screen. For example, I can check whether models were correctly loaded on to the GPU by looking at the buffers as a point cloud:

Here is an example of how one may go about debugging an issue with Nsight.

Jules: A nasty bug I fixed a few weeks ago was with the viewmodels. For some reason the back of the player’s hand was pointed toward the camera instead of away from the camera, the same would be true for any gun he’s holding which is a bit disconcerting. Clearly the values on the z-axis were being inverted. This looked very much like the age-old problem between the coordinate systems of DirectX and OpenGL. DirectX defaults to a left-handed coordinate system, while OpenGL defaults to a right-handed coordinate system:

However this shouldn’t be a problem since OpenGL allows you to use a left-handed coordinate system. So I spent a lot of time going over all the project matrices trying to spot where things went wrong. Finally, while taking a closer look with NSight, I noticed that compared to DirectX, the depth range was inverted. The depth range decides what the nearest and the furthest depth values are. Normally the near value of this range is always smaller than the far value but in OpenGL you can also turn them around, this is something that’s not allowed with DirectX.

So it turns out I made a very small typo:

glDepthRangef(0.99999f, 0.02f)

It should have been:

glDepthRangef(0.02f, 0.99999f)

It had nothing to do with coordinate systems and was just a matter of accidentally mixing up two numbers

When Jules needs help, he knows where to seek it. After all, he was a contributor to the Dolphin Emulator project, where many other graphics programmers tend to converge to perfect the state of the emulation and further improve its overall performance.

Jules: I do often ask for advice from the developers at the Dolphin Emulator. Especially degasus, he knows the OpenGL API like the back of his hand. Whenever I feel like I’ve tried everything, after I explain it to him he’ll be able to pinpoint the problem without even looking at the code.

On top of debugging, another relevant concern for the port is related to performance. By now Stalker is an old game and should run just fine on most modern Linux systems no matter what, but if you compare how it runs on Linux versus Windows, you may find some loss.

Jules: There are some optimizations that I’ve skipped for the sake of simplicity. However this doesn’t mean that it will slow down to a crawl. Modern graphics cards can already run the testing builds at full speed, it just means that initially it may not be as fast as DirectX in a benchmark.

A bigger problem is the OpenGL requirements, currently the backend requires at least OpenGL 4.5 which is available on modern DirectX 11 cards with updated drivers. But eventually with the use of extensions I can get the requirement down to OpenGL 3.3 which should be supported by most DirectX 10 compatible graphics cards.

As hinted, compatibility may be a problem for manufacturers which do not support well the latest implementations of OpenGL… sadly that may include AMD, but it’s too early to say since he has not tested any AMD hardware at this stage.

Jules: Currently I’m developing on an Nvidia card, so that’s where the first builds will run best. I’m not using anything explicitly Nvidia-only, so if there are any bugs on AMD or Intel cards I should be able to fix them. I’m not too familiar with the state of their drivers on Linux, so we will have to wait and see if they correctly implemented all the necessary extensions.

While the videos and screenshots released not too long ago are already very promising, you should not expect the OpenGL backend to be ready too soon… many issues can still occur and delay the project at this stage.

Jules: I’m going to set a very optimistic date and I will likely miss it, but if all goes well a playable version of the OpenGL backend on Windows will be done before the end of the year. I can’t set a date for the Linux version, since I haven’t started on that yet so you’ll have to make do with “when it’s done”. I’ll start releasing builds as soon as I have something that is sufficiently playable. These build will initially be Windows-only until the engine has been ported to Linux. Obviously the build will be released without the actual game content, so you will still need to buy the actual game.

While there is still lots of work on the graphics side, the port still won’t be ready even when that is done. Hopefully some folks will jump in to help Jules when the times come, for the non-graphics related code.

Jules: Unlike the graphics API there was little reason to abstract away the operating system API. While things like the filesystem has been abstracted to deal with their data file format I still expect there will be some challenges in getting rid of all the dependencies on the Win32 API.

I’m currently completely focusing on the video backend, but luckily a contributor has already started on making the CMake build files so we can start compiling code on Linux. I’m hoping more contributors will join, not just to get more work done, but because you can learn so much more when working with others than you can do alone.

Since an OpenGL backed and a Linux port may end up making the game very portable, a Mac version of the port is not to be excluded. GSC may be more interesting by the perspective of having a larger market base if they ever sanction the porting efforts.

Jules: I’m a multi-platform user, so with me also being a Mac user I am definitely interested in porting it to that platform. But it’s not my priority at the moment, I first need to see if we’ll actually succeed in porting it to Linux.

If you are wondering about the legality of this whole process, well… GSC has not sanctioned anything yet. Technically it’s just an engine port so no build will even be released with the data that’s required to play the game, unless they agree to use Jules’ work to make it an official Linux version. It’s too early to say.

Jules: I’ve sent them an e-mail, so far they haven’t responded yet. I’m not sure if they will get back to me at all, but I do hope they’ll respond so that they have a chance to decide whether they would endorse a project such as this. In the end I’m using their source code without their explicit permission which often deters contributors worried about whether the project would be taken down.

As Jules is a multiplatform user, he is no stranger to Valve’s Steam Machines initiative. He remains a little sceptical at this stage as to what level of support their plan will garner.

Jules: A definitive boon for Linux gaming and a smart move for Valve, but they will really need to start putting their weight behind it if they want it to succeed. In the end it will all depend on how many games will have good ports. However, I’m much more worried about VR. The headsets that will hit the market soon will all be optimized for Windows. Linux is about to be left out of another major gaming market.

Many thanks to Jules for his work on Stalker and for his availability to share some thoughts on the port. Note that this project is fully open to contributors, so if you are interested to help/support Jules in his task, please join the IRC channel #openxray on Freenode.

We will be following very closely how the OpenGL backend shapes up by the end of the year - Stay tuned on BoilingSteam for more details and reports to come.