The Client Data Management System

So on the client I was using JSON files, loading them into memory and holding them there as Dictionaries. The problem with this is all these databases can become quite large, increasing the RAM requirement of the game quite significantly over time. While there might be ways to parse these files more quickly by scanning through and finding what we are looking for, then loading that into memory it's not very efficient and would increase loading times. I could use XML which is better for that sort of thing but then I would have to build a parser, among other things.

The solution for me is just use SQL, it's what I know. Luckily SQLite exists and doesn't have any requirements of the user's system. At least I never encountered such a problem across all the environments I have used it on. Plus there's an add-on already set up and ready to go for this purpose. Not to mention we get all the relational database goodies that come with it and don't have as much overhead to deal with.

While I wouldn't use SQLite for a server-side database with potentially thousands of transactions happening over the course of a few seconds it is fine for this application. I will have to remember to write a function to optimize the DBs once in a while of course, but that shouldn't be too hard.

First thing to do is get the documentation open, start a new singleton for the DB manager, then get my db files set up.

It's definitely going to be a half day today, I have a lot to do this afternoon to prep for tomorrow. Tomorrow I will be finding out if I am actually finally moving out of here or not. At any rate, I suspect the DB manager for the client to take several days at least, I have to define tables and handle shifting the data from JSON to SQL. I also have to change my serializers on the update endpoint of the API to sanitize some of the superfluous data out.

Anyway, back to it.

Starting to get a little overwhelming now, geez.

Each of these needs a table and all CRUD functions. Going to be a lot of code here, might end up breaking this script into multiple sub-nodes at some point.

Yeah, I was right about this being a big bunch of code, I haven't even finished all the functions for handling the Slime Versions table and we're up over 100 lines.

So judging by the time now, I might just get the slime version DB functions in today. This is fine.

I set up a test scene for the DB Manager, didn't expect to get this;

Of course the above code wasn't complete, I also forgot to call open_db()

Alright, now I can create and read from the DB.

I should only run the setup_slime_version_table() when there's no such table already in the DB.

I will have the version get updated in each table update function, assuming the update is successful.

It also occurs to me that I may want to use call_deferred() every time I instance a child node in the game server to be sure all data loads and there's no racing or blocking. Creating an issue for that...

From the look of it we're going to have a few thousand lines in this file if I don't break it up into several smaller files.

It's noon now, so lunch time. I will be doing a small amount of work on this once lunch is over, then I have a lot of stuff to take care of. If tomorrow goes well I will probably be taking a week off at the end of the month to get packed and ready. Though honestly I will probably be pulling half-days because there's not much I can do in the morning before everyone's awake.

Realistically it's probably all half days from here to November unless something goes terribly wrong and I get stuck here again.

Anyway, better make some lunch for my dog and I, take her for a walk and then see how much more I can get done today.

Can use this is players are having problems with updates to figure out why.

It will also help me troubleshoot database issues during dev, but a player will be able to copy this text and send it to me in a bug report.

With that, I am going to have to call it for the day. Tomorrow I will be going down to the apartments and see what's going on so I may not be able to do anything depending on how all that goes.

I will be back Friday either way, I think. Either doing the last full day until November or... well, yeah. Let's not try to think too negatively, gotta be positive.

Anyway, clocking on out. Be back tomorrow or Friday. I'll try to get at least something done tomorrow either way but I am not sure if I will post.