Busy work from different directions

Another Post another unexpected event.

What happened:

So since the last Post I was announcing that Minecraft was updating so I was not able to really work on my game engine because some of my mods had priority.
So it turned out really quickly. Nope Forge and Minecraft aren't ready for porting just yet.
There are some essential features missing or the version is not even declared stable yet.
I technically could sit down and port the mods yet but some of them would just gain a massive performance loss or stability loss, and that's just not worth it yet.

So during that time I was researching on other things to because I had decided ok while I wait on that I could be at least productive for my game engine since I announced no big projects.
What I looked into is explained later. Anyways a couple days later CurseForge decided to update its entire Page look and made it in some aspects better and in other aspects worse.
Overall thought its faster in loading, but worse in exploring.

But that was not the issue. The issue is some of my software uses CurseForges website to run or just to give me data that CF is not willing to give easily. 

So all of these tools (Namely 2) broke and I had a lot of pressure to fix all of them in a Rather quick time. That alone took me 5-6 days to get it properly working at all cases and getting the software that I made stable again.
I lost a bit of data (4 days worth of it) but it is not as bad as it sounds.
Anyways half of the time I had for budget was now gone. Minecraft took another 2 days, so 5 days left.

New Additions:

Between the time I was fixing my tools and finding out that mc was not ready yet I started to research what way I could use to implement Entities. The best looking way was to use ECS (EntityComponentSystem) which was suggested by Overmind multiple times, but after looking deep into it and trying to make a custom version of it I noticed that It would take to much time to even implement and I would have stood there empty handed. Taking the existing Library was not an Option for me because I wanted to remove legacy features they had and on top of that I had no way of testing visually if it was actually working.

So I decided to drop it for now and to go with the slowest but easiest implement and to design with possible. A Classbased Entity System. Though I optimized that to run smoother and not be to wasteful.

This gave me the way to actually have something to test with and since time was getting scarce that sounded like a way for me to have at least something.

The Library I want to actually use will be developed in the background and once it is ready it will just replace the existing System, I designed it to be easily replaceable.

Here is a large clip on how these entities farm the Trees and go back to their home. (Just a simple lerp + cooldowns)

New Optimizations:

After the Entity System was working and I had a test case to work with, (12 hours before the post was written) I decided to do some optimizations that needed to be done.

I wasn't sure if they would get done in time, but since I had the entities to show that was fine by me.

One of the larger issue that this game was going to have would be lag because of the TileSystem.
Yes it is GPU wise pretty optimized and thanks to batching it would render quite fast for a large area. The issue was with the CPU side of the Batching.

On the largest Map (4096x4096) with every position having a Tile the lag of the Batch Controller was 0.6ms to control what is shown and what not. For 16Million Tiles (6710 batches) that sounds like that's not much.
Yes that was somewhat right, but the thing is with the current system 1 batch was accounting only for 1 Tile type. But there is 2500 spots and if you had 2500 different Tile types the lag would grow exponentially. Since I had to check over all of the batches to validate their visibility.

That was a design flaw. So my solution? Add More Batches! (Insert TFS "That is something what goku would say" joke here)
But yeah the solution was shrink down the batches from 50x50 to 16x16 that will turn 6710 batches into 65k (Per Tile Type).
This allowed me to align them with chunks. And also remove the overhead of a "Batch" being only partly visible.

The Chunk has in this case a very optimized visibility manager. The most lag you always will get out of it will be 0.6ms of lag. That will not increase no matter how many chunks exist. Unless you expand the view range further then the current maximum. (around 640 tiles radius) So merging the batches with the Chunks half's the lag in worst cases.

Memory Optimizations:

Extra to that I optimized how batch change requests are made.
Before it was a Object that had the Position (2 shorts), the Tile ID (1 24bit Integer) and the action in it (1 boolean).
So my thought process was: "Since I move the Batches to Chunks and the Changes are fitting into a 64 bit Number I could also make the Chunks handle the changes" that would allow me to control Memory Allocation. This tiny change reduced the allocation in a 16Million Tile spawn so much that massive GC Cleanups turned into 1 Medium sized one (freezed for 0.5-1 second).

The Memory footprint was increased a bit though. That was negated by the second improvement.
My BatchIndexer (DataManager) which manages GLData was using also a very optimized Object but at 16 Million Tiles it was using 0.5GB of Ram. Thanks to changing them up to a integer instead of a wrapper I was able to ruffly shave off 0.2GB of the 0.5GB of ram usage.
Which brings us in the end with a 1 Tile Per Position in a 4096x4096 map to 0.9GB of ram.
This is not the lowest case but a good average to start at a case that will never be reached by default.
Note that this last improvement I was aware of already by a long time and attempted already multiple times (4 times over the last couple months), but only this time I was actually able to do it.

So yeah now I can go tired to sleep because the summer is draining me like crazy.

Thanks for reading and for following my Hobby Game Project.