Developer Blog - Air Drone Model, Interactivity, Multithreading in GZDoom and more.

Two weeks have passed and you know what that means? A new blog post! Let's get started right away.

Viral Tweet!

Hooray, we managed to land a new viral twitter post with almost 87.5k views. The video in question is a little “bug” we had in the form of a toilet. Here is the video: https://twitter.com/Nexxtic/status/1549537674712809473

You know, every shooter needs interactive toilets, right? We’re not going to miss out on that action, it would be embarrassing. Us being Selaco, we wanted to make it even cooler. You can not only flush it, you can also flush objects down through the toilet. Too many objects and it blocks, and throwing too much detergent into it will cause it to bubble up and explode. This brought the unfortunate side effect that even our player character, Dawn, was able to get flushed down the toilet drain. Was this a bug? Yes, it sort of was. What happened here is that I was simply prototyping a mechanic. While recording footage, I was throwing a bottle of Toilet Cleaner into the toilet. While flushing, I came to the realization that, given how there is no actor validation when flushing objects down the toilet, the player character should be flushable as well. I did what every sane developer would do on such an occasion and tried to see if it actually worked.. And I was right, it did! Here is the full uncut version:

(Toilet flush effect is WIP)

This lead to a fantastic tweet and a fantastic piece of fan-art

(Artwork by Mach1nka [https://twitter.com/m4ch1nka/status/1551911220831293440]

Enemy Model: Air Drone

We’ve shown concept art in a previous blog post. The model has been done, the sprites have been animated (although some touch-ups are still in order) and their basic AI has been implemented!

High detail render:

Sprite:

(WIP) Plasma Rifle Alt Fire: Gravity Manipulation

(Apologies that enemies are hard to see. Our sprite illumination system is broken in the current build of the game)

We’ve discussed earlier that the Plasma Rifle has the ability to ‘overcharge’ environmental hazards like Explosive Barrel. This will greatly amplify their damage output and damage radius.

The Alt-Fire for the Plasma Rifle is going to synergize with this mechanic. Pressing alt-fire while aiming at an object will allow the Plasma Rifle to manipulate gravity of this object, essentially ‘grabbing’ it. Pressing Primary Fire while holding onto an object will throw it at very high speeds, capable of dishing out high amounts of damage or, in the case of an environmental hazard, trigger detonation upon impact.

The effects in the video are work in progress. Also yes, the Muzzle Flash has a new effect compared to 2 weeks ago!

DMR Update

Nothing new to show yet. Right after the Air Drone was finished, the DMR 3D model went into production! Expect to see the fully completed model very soon.

Railgun Update:

Design is finished! After the DMR is done, that weapon will enter its modeling stage. We’re not going to show this weapon off, though. You will just have to take my word for it: It’s awesome!

Destructibility pass

We’ve further improved our destructive effects, which includes brand new particle effects, separated ‘voxel chunks’ of certain objects, and a brand new sound pass for ricochets, impacts and devices breaking apart - putting a further emphasis on the power of each bullet. This change will be included in our upcoming Demo update.

Throwing Knifes

We’ve added a bunch of kitchen knife voxels to Selaco. Originally intended to be used for nothing but decoration, but after making them interactive (picking them up and placing them elsewhere), I noticed that these are, in fact, sharp pointy objects that can be quite damaging in real life. I decided to allow them to be used as a makeshift weapon. Pick them up and toss them against an enemy! Juggernauts will deflect them like the badasses they are, but lower tier enemies will take significant damage when hit by a throwing knife. Thrown knives can be reused after an unsuccessful throw. However, when an enemy gets hit by the knife, the knife is lost.

The throwing knife is one of the rare few weapons which allow for full-stealth kills. An unaware enemy will die instantly, no death scream, no alerts.

Microwave

This is a microwave. It pings when the meal is finished. What else is there to say?

Enemy Lines

We’ve added ACE Security Landmines a while ago, that the player can deploy, but the Aliens are getting their own type of landmine! A thicker, less clever one. While the ACE Security Landmine detonates whenever the sensor detects a target that the ACE Database considers ‘hostile’, the Alien Landmine doesn’t have such a luxurious feature; it blows up whenever a medium-sized object comes into contact with the mine. The Landmine has a thick armored outer shell, none of your weapons can penetrate it unless you want to waste a precious explosive. What you can do, however, is look for an object to throw on top of it. Hey, I told you they are not as smart! We have finally turned office chairs and whatnot into a game mechanic.

You know what is smart? Multithreading. Which Cockatrice has been spending some time on.

Multithreading support

Background

After the public demo we received a number of performance complaints, which is to be expected because not everybody has the best computer setup in the world but it started to bother us when it appeared that some of the complaints were from people with top-of-the-line PC setups with brand new GPUs like RTX3080s.  It turns out that the RTX drivers are partly responsible for these performance problems but one of the common reports is that the game had a lot of brief freezes or unstable frame rates or "hitches", and then would go back to normal.

This is something I experienced a lot during development because personally my PC seems to be slower than normal at loading files even with an SSD, and we often play the game without pre-loading any of the graphics because it boots the game faster for testing. So I knew one of the main culprits for unstable frame rates was the fact that GZDoom loads sprites and models on the fly during runtime but it does so in a single thread.

Any time a sprite or model has not yet been loaded but needs to be rendered for the scene it is fetched from disk and the whole game engine has to wait for this before it can proceed. Not only does it have to fetch the data from disk (which is a very slow operation, slower still if the player does not have an SSD) but the image or model data has to be uploaded to a buffer on the GPU as well.  That makes the disk access a big bottleneck for performance. Any time a number of sprites have to be loaded from disk the framerate is going to take a nosedive, and in a very busy scene it can freeze up completely for a noticeable moment in time.

We can prevent these freezes by pre-loading the data before the game starts, but doing this with the amount of sprites we have creates an uncomfortably long load time at the start of the game and start of levels. Many people would just close the game because it took too long to load, most likely thinking it had crashed. This solution is not acceptable.

But what does this have to do with audio? Well it turns out that loading textures and models is only part of the problem. A lot of textures in Selaco are very very small. It's also quite complicated to load textures in a background thread because we need to write a different implementation for OpenGL and for Vulkan. (Which is something I am currently looking into). After doing some performance benchmarking on disk access it seemed to me that a lot of time during those little freezes in gameplay was actually being spent on loading audio! Of course, this made so much sense because Selaco is *extremely* heavy on audio, probably more so than any other GZDoom project I've seen, and uses many variations of sounds for the same thing. Individual audio files tend to be much larger than textures and almost NONE of them are loaded before the game starts!

Just like textures and model data, audio has to be loaded from disk the first time it is used which is very expensive, and loading audio is easier than textures because it uses only one API (OpenAL) which supports multithreading and is very straightforward, so this seemed like a perfect place to start to try to improve the engine.

(As a side note, one of places where performance complaints frequently came from is during explosions. As it turns out, an individual explosion when played the first time would often load between 8 and 20 sounds at the same time! No wonder there were hitches.)

Audio Threads

Our solution is very straight forward in design, but implementation was a bit of a nightmare. A number of background threads are created (user specified for now, but in the future we will adjust how many loader threads are available based on how many cores/cpu threads are available). These threads can be queued up with a list of audio files that need to be loaded and they keep running until the list is clear.

During gameplay, whenever a new sound is encountered that needs to be played it is queued up in one of the audio loader threads.

On the main thread the queue is checked every frame to see if any audio has finished loading. If an audio file has finished loading, any instances of those sounds waiting to play are played. Most of the time the delay caused by having to load the sound for the first time is barely noticeable.

There are some weird caveats to this approach. For instance, it would be possible for the game to tell the sound engine to play a sound, and then try to stop it before it has even managed to load in the background thread, resulting in the sound playing when it shouldn't be, or worse a looping sound playing forever! This actually comes up quite a bit with weapon loops and doors/lifts. The solution was to check the audio playback queue whenever a sound is changed or stopped, and change or stop the item in the queue.

While this approach is fairly straightforward, it took a while to actually get it working. This was partly due to my unfamiliarity with how GZDoom actually loads files, getting re-familiarized with C++ threading, and the fact that GZDoom file loading is a giant spaghetti mess of abstractions.  Getting individual file loads to work was fairly easy, as the default file loader was reasonably thread safe and only took a few modifications to work. The bulk of the work was understanding how the PK3/Zip loader worked and finding a way to make that thread safe.

Results

Loading audio in the background has ironed out a LOT of the hitches and slowdowns during gameplay. Especially during intense fights with lots of explosions, even my weaker than normal system runs the game mostly buttery smooth. The solution still comes with a few drawbacks, but the worst one being that if you don't have an SSD, some sounds are going to be noticeably delayed the first time they are supposed to play. But as long as that doesn't kill the framerate I think it's something we can live with.

We ran a brief test on this change with a small sample of players and for many frame rate stability has improved or was always optimal. I think we are on the right track.

Future

I am hard at work universalizing the background load process so we can have an easier time of loading other resources like sprites and models. The goal is to prevent any unnecessary performance penalties from disk access, and in the end we expect Selaco to be a completely smooth experience for every player, without needing the most modern hardware available while we still push the limits of what GZDoom can do.

The first phase of multi-threading has been pushed to the Beta branch of the Selaco Demo, so if you are a Patron, you can give it a go using the code EPICTESTERS2255. 

Conclusion

Another very productive couple of weeks where things got more refined and finished up. We’re currently working hard on some experimental exterior zones to get a proper workflow going, as well as finishing up a new level.

Thank you, once again, for sticking with us and funding our little development team. Selaco is shaping up to be better and better and we are so fucking pumped to get this into your hands soon!

Wesley de Waart