Planning the Slime Server - Part One

This week will be a little weird and slow, I think. But I need to have a solid plan or at least a good scaffold in place for the Slime Server before I start on implementation.

When considering how the slime data will be handled I have to think about scale, if there are tens or hundreds of thousands of slimes being saved it could cause the server to lock up and no data will get through. I have a few ideas to solve this but I have to spend some more time considering each.

One idea is to have a sub-process for saving. I would need to open another connection to the DB in the sub-process though and would have to implement some IPC with data that could change at any moment. This could be a massive headache and disaster prone.

Another idea is to have a sub-server which I send over a counter of currently active slimes and an array of their IDs, then whenever the save timer timeout occurs send the data over and check it against the counter and ID array. This could work but also has some drawbacks and would have a few holes that could result in lost data if I don't handle it correctly.

Then there's just having a timer on each slime and having that fire the save every 10 mins or on logout. I'm not sure how timers are handled internally in Godot, though, so this could be very expensive on the server.

Saving the slime every time something changes is just asking for trouble, once we get to the point where the slimes are adventuring and taking damage, getting experience, using skills, etc... things will clog up real quick, I would think.

I would expect the timers in Godot use frame delta and track themselves every frame. If there's 10,000 timers how much would that effect the server's frame rate? It also seems like there could be a lot of slimes reading and writing to the db all at once and that feels like it'd backfire on me.

So far I am leaning more toward the sub-server model since it seems like the easier to implement overall with the least potential drawbacks. It does mean writing another network interface and does mean this server will take that much more work and time, but it's better to set this up with scalability in mind from the start, I would wager. The game may never see more than a handful of players concurrently but I should plan for it to see as many as possible because who knows really? Rather not become a victim of my own success, having to completely rewrite large chunks of the servers in a panic to allow more players.

So the sub-server idea feels like the best to me right now. I think I will just spend this week working on design and planning, give my ideas some breathing room before solidifying them into code.

The next thing I need to figure out is how players and their slimes are tracked. I think it will be fine to use the same method I do on the Apartment server with a node representing the player's connection ID that is setup when the player logs in, loads the slimes as individual child nodes under the Player. Whenever the save tick fires it can emit a signal to all the slimes to send their data over to the save server.

When a player logs out instead of immediately destroying their node like on the login server we should fire a logout() function that will cause all the child nodes to fire their save() functions, then queue_free() the player node.

I will keep all the slime data in a hash-table so it can easily be sent over to the save server without having to deal with extra steps of packaging the data. I can write functions to work on the data, essentially getters and setters that should make it easier to interact with the table.

I could, instead of using an array and counter in the save server communication to verify data... just store all the slime data in one big hash-table on the slime server and send that over to the save server. It's all text so it shouldn't be too large for it to handle... at least on paper.

It would be faster to lookup and erase from when a player logs out, since you would have to iterate over the array every time a slime is saved and removed, where a hash-table should have fast look-ups to do the job faster. Also means we wont have to have every single slime send it's data over and will only have to run one function and far FAR less operations.

I could use the child-nodes of the Player as proxies, with some functions to interact with the date under the ID in the large in memory slime bank. Essentially all the node would need to know is the ID of the slime and then all the functions can handle the rest.

Then when the save timer fires just send that data over to the save server as one big chunk, have it iterate over the slimes it was sent and construct the queries for each, then send the data to Postgres to handle the rest.

Once the save is complete we should send back an RPC to the Slime server saying the Save server is ready for the next batch, start the save timer.

I could write the save server as an API and send the data over as a payload as well. There's so many ways to handle this.

No matter how I implement the save server I will make sure it exists inside a virtual network that can not touch the internet for obvious reasons. Fairly easy to do with docker containers, luckily. I can just set up a VPN for the servers to talk to each-other. I already have one in place between the Login and Apartment server.

Now the question is do I want to implement this as an API or use Godot's Multiplayer networking code and implement it in Godot itself? I can get a response either way telling me everything is okay or there was an error.

An API would likely require me to format the data before sending it though and that could hang the server as well. Probably best to just keep it all in Godot for now, if I need to I can use GDNative to speed things up.

So that would be the first thing I had to figure out, I'm sure over the course of the week I will change it at least somewhat.

I can already picture how all the care system interactions work(server side), I will need to implement some things on the apartment server, mainly inventories, to get started on some of them though.

I do need to redesign my Geru Base for the networking and probably will want to redesign the baby sprite base as well because it's currently over 50 frames and I think that will get overwhelming fast.

I should decide if things like bathing should happen in a separate scene. It would take a bit more work but is probably preferable to spawning a little bath-tub in the tank with them. Maybe? Not sure.

After 40 minutes with my guitar I realize I am not well today, suspected as much earlier. Part of the reason I decided this week was better spent planning had to do with the way I've been feeling. But it will likely work out to my benefit and allow me to much more easily implement this server than the last one... at least I hope so.

The best laid plans of mice and men often go awry

Anyway, I've got a good idea what it'll take to SAVE the slimes and UPDATE to the DB, next I need to spend some more time looking at the models and how they interact.

I'll need a way of logging all care actions a player does with a slime, the things they ate and drank, etc. This is in order to handle their evolutions through life stages primarily.

I'll work that out tomorrow. I got the save server whiteboarded and I feel like I am going to crash any moment so I will go make some lunch and rest for a while, maybe I will be back later to plan some more. I have my bookstack that I can write to from bed if I have any ideas, not to mention the mattermost for anything I need to quickly get down. (It's like discord but I control it.)

Anyway, yeah, lunch and rest. Brain not doing so good.