Persistent object streaming

[Bambooza]

Finally got around to watching last week's scl and they had an interesting tidbit. There is two databases. Personal(global) and server database. So items in your personal storage be ship, char or hanger will persistent across servers. Items will persistent on the server until clean up.


Crash out. With in ten minutes if you log back in you'll go back to the server.

Things that are stowed in your player inventory or in your ship will persistent across servers. Everything else will stay on that server until its cleaned up base on the cleanup rules.


Ships not stowed before logging out or in the ships bed will exist on that server. So insurance recovery gets you a new ship not the ship you were using. The only exception is they are attempting to introduce a auto stow on server crash

 

[CosmicTrader]

Yes, this is how it will work in 3.18.
After Server meshing (4.0) there will be multiple servers and 100% persistence.

 

[Lorddarthvik]

So, funny thing is they already do all this. You ship and it's inventory, and even shit thrown on it's floor already persists through server changes if the ship is stored, so does everything in your personal inventory.
The rejoin server function was there for a long time, even if it doesn't work 98% of the time. Last couple of session (3.17.1) when the server crashed I was put back into my ship where I left it in orbit around a planet. Sometimes ships get stowed already, and I got back my ship with all the loot placed inside as well.
Ships and bodies already stay around for a while, although that timer is totally random and broken like everything else.

So what exactly will be new about this in 3.18?

 

[Richard Bong]

SSOCS. was supposed to move the database off the servers. That way it wasn't server dependent, like your inventory, and was supposed to be the key to full persistence. As soon as everything was moved to SSOCS we were there. (And almost everything was supposed to be there more than a year ago.

PES is per local server, which seems a step backwards, but is now the key to persistence.

Sorry, but, to me, this seems a step backwards. Consider me unimpressed.

 

[Vavrik]

Having two databases is not an odd thing, and it's not a bad thing. That is done all the time in software, to separate concerns. In the case of a game, the concerns are session data, vs long term persistence data. They actually require two different database architectures to work effectively. Session data needs to be real time capable, and long term storage can work like a normal SQL database. It's a design issue if you try to mix the two.

What I don't understand is why they need to do a database wipe every time they change the data structure. There are tools built into every database worthy of being used to aid with the modification of a database table structure without any data loss.

 

[Richard Bong]

Having two databases is not an odd thing, and it's not a bad thing. That is done all the time in software, to separate concerns. In the case of a game, the concerns are session data, vs long term persistence data. They actually require two different database architectures to work effectively. Session data needs to be real time capable, and long term storage can work like a normal SQL database. It's a design issue if you try to mix the two.

What I don't understand is why they need to do a database wipe every time they change the data structure. There are tools built into every database worthy of being used to aid with the modification of a database table structure without any data loss.

Oh, I'm not saying have only one database, though in this case, it appears to me, that the personal inventory and inventory of what gets strewn around and left behind, seems like the same thing.

I'm saying they made a big deal of this being the job of SSOCS and how SSOCS was separated from the server so it worked across the servers but PES is going back to being a local database per server and is the new way to do what SSOCS was promised to do.

 

[Bambooza]

Oh, I'm not saying have only one database, though in this case, it appears to me, that the personal inventory and inventory of what gets strewn around and left behind, seems like the same thing.

I'm saying they made a big deal of this being the job of SSOCS and how SSOCS was separated from the server so it worked across the servers but PES is going back to being a local database per server and is the new way to do what SSOCS was promised to do.

Two different things. SSOCS (server side object container streaming) allows objects and everything that is a child of that object to be loaded and unloaded into the server memory. Ie into and out of the simulated game world. This works great for loading into the server default objects like spawning in new ships but needs a hard drive file or db to load in game world changes like to coffee cup on the floor. As long as the server was up the coffee cup would remain (there is a crude cleanup subprocess that removed objects from the game world) but if the server crashed then everything resented that was not in the player db back to the initial state on server game start. Ships that the player stowed or logged out in bed were and will continue to be saved to the player db and this db is global in that it doesn't matter which server you log into they all read from this one point of truth.

This new tech allows every server to continuously back up the game world object state to a db and in its first implementation will allow the server on reboot to continue where it was after a restart instead of resetting to the clean slate state. That means the coffee cup on the floor will still be there after a server restart. (unless it's cleaned up). Servers for now will continue to be their own unique worlds so the state in one will be different then the state of another. So the Golden rifle dropped on server one will persistent across server restarts and if you happen to log back into the same server you'll be able to collect it again. But will not be there on other servers that you have a higher probability of logging into.

But mostly this feature has limited impact on the player experience and is just a stepping stone to server mesh. First step was object container streaming now the have the hooks to point the streaming to a db to retrieve and store the data. The next step is to allow the game world to be broken into chunks and finally allow those chunks to auto size.

One thing to keep in mind is client crashing logging in to the same server and seeing everything as it was is because the server didn't crash and is the sole truth of the game world.

For me the biggest impact will be the auto stow on server crash if it works right will hopefully mean no more lost items and cargo.

 

[Richard Bong]

That is not what they told us it was about. It was about inventory, and about persistence, and would even allow for multiple systems and unlimited people per server. It was the "Jesus patch" that would allow everything to work and let additional systems to be added 3 years ago with patch 4.0.

 

[Bambooza]

I don't believe they have ever said unlimited people per server. The idea is we all play in the same verse even if we all can't play in the same server.
So when the feature is complete with meshing a global party at area 18 should be theoretically possible all the SC plays will not be on the same server as it would be cloned as many times as needed to handle the number of concurrent players. Ie 150 players per server is all you would see and be able to interact with even if there was 20,000 logged in players in area 18. This also helps with latency issues with players all over the world this also doesn't prevent @NaffNaffBobFace, @Vavrik and myself from getting into trouble on @NaffNaffBobFace ship in vanduul space. It's all about how the load balancing of players in a shared world simulation is handled.


As for persistent that has always been there. A running server has the full game world in memory and as long as the game doesn't crash the world persistent across days into months and years. The hard part is attempting to make large worlds with many players playing in it. But even here you can have simulated worlds like battlefield with 64 players all doing their thing and it persistenting for the match time. Even there you have player data being stored and retrieve from a global source of truth like stats and unlocks.

The hard part and why it's taking so long is being able to share parts of a game world in real time. It's taking that ship @NaffNaffBobFace is flying and copy it from one server to another in real time so that @NaffNaffBobFace doesn't need a zone and/or loading screen. And it's not just the char ID armor set in shade yellow but the whole ship and each mutable variable like light state doors open. Damage decal placement and location of all 300 medical gowns. This is the magic of containers as all the code needs to do is pass the root location and with it all the trunks branches and leaves are shared.

So now we have a way to load oblects trees and save object trees and now they are adding the ability to save and load the whole game world state (everything in the simulation) the next step is to be able to share the game world and object trees. That way your ship and everything in it can seemlessly move from one server to the next with out a loading screen. Just like client streaming let's you go from the ground to space to a station with out loading screens.