Bolstering the Capacity of NEStalgia Servers

Wouldn’t it be great if you could play NEStalgia on much larger servers? If population surges didn’t mean lots of new servers and inevitable server merges?

This week’s update probably isn’t a super fun read, but it heralds a major change that should significantly improve the NEStalgia experience. Hopefully when the Key of the Exiles expansion goes live, instead of worrying about which official server you and your friends are playing on, you’ll instead be able to focus on saving up gold to purchase some sweet ship upgrades.

Before I get started, I’ll just quickly note that NEStalgia is currently available at a huge discount as part of the Steam Summer Sale. Check it out!

The Indie MMO Problem

The core of NEStalgia’s server system is the common “island server” design where each named server is basically a separate instance of the game running off of its own savefile database. That’s why your characters tied to the Malice server can’t be used on Midgard unless someone manually transfers your savefile. It’s also why each server has separate guild registries and market place listings. This would all be fine and good if the game didn’t have so many players; when I was originally developing NEStalgia I didn’t expect that the game would find such a large audience. It’s a good problem to have, but a problem nonetheless.

Right now a single NEStalgia server can accommodate 50 or so players online at once before lag starts to become an issue; usually each server caps at about 70 players online. This active player limit isn’t a glaring deficit in and of itself, and is actually pretty high compared to most indie games. The difference is that most other multiplayer indie games aren’t MORPGs, and thus they aren’t as reliant on maintaining balanced population equilibrium between servers, nor do they have to be 24/7 persistent worlds. For example, you usually play games like Terraria on small private servers with friends. In games like DayZ you play on larger public servers, but because each play session of the game is generally self contained, switching to a different server and starting over isn’t all that painful.

How We’ve Been Coping

Whenever NEStalgia has received a large influx of new or returning players, we’ve opened up new official servers to accommodate the population load, and then merged those servers into each other as the populations recedes. It’s hard to time these merges so that every single server maintains a satisfactory active population level, and it can also lead to players being isolated from their friends on a more popular server.

All of those factors are why server management has easily been the biggest source of frustration both for players and the dev team. As a player it’s not fun to see your server go into decline after a population surge, and as a developer it’s a huge time suck trying to stay on top of server expansions, merges and transfers. On several occasions we’ve had to open 16+ official servers at once just to accommodate demand… but even the smaller population surges tend to tangle us up in server and savefile management nightmares that bring actual NEStalgia development to a grinding halt.

From here I could  probably write a few more paragraphs about the challenges involved in finding a solution that is a good fit for NEStalgia. However, I’m fairly certain that most of you don’t care about the drawbacks of client side saving or headaches with online shared savefile databases, so let’s skip right to what I’ve actually come up with…

The New Solution

The v1.70.0 update should keep NEStalgia’s core “island server” design intact while still allowing us to accommodate many more players on what is technically a single server. Instead of having every single instance of a server store its own savefile database, official servers will now operate in clusters of server instances that share savefiles. Each of the servers in these server clusters will operate under the same server name, but can have additional instances opened to accommodate demand. Ex: There might be a Midgard Cluster, with open server instances Midgard (1), Midgard (2), Midgard (3), and so on.

Like the current named servers, each server instance is its own separate world. You won’t see or directly interact with players logged in to other server instances, though you can come and go as you please between instances. For example, if the first server instance the Midgard Cluster – Midgard (1) – fills up, you can simply log off and join the second instance Midgard (2) to pick up right where you left off. The same things goes if your friends are playing on Midgard (3) and you’re currently logged into Midgard (1) – you just need to join Midgard (3) in order to play with them.

The different server instances in each cluster share the same guild registry and marketplace database. That means that if  you make a change to your guild cape or recruit a new member on Midgard (1), the change will automatically be registered on Midgard (2) and Midgard (3). The marketplace communicates between server instances as well – every instance of the server sees the same market listings, and purchasing an item on one instance will instantly take it off the market on all other server instances.

Don’t be concerned if all this talk of “clusters” and “instances” has your head spinning. All that you need to know is that you’re going to be able to play the game the same way as before, minus concerns about over/underpopulated servers. In practice all of this should be fairly intuitive even to first time players.

Making the Transition

If all goes well, I hope to get v1.70.0 and the server clusters up and running in the next week or so. The biggest concern is dealing with players who still have savefiles on multiple servers, mostly in regards to merging files and/or dealing with excess characters that can’t be merged. There’s no perfect solution to that issue and it’s probably going to eat up a ton of time, so bear with me as I get it sorted out.

Between all four of the current named servers we’re talking about 30,000 savefiles – and that’s after previously setting aside 50,000+ “legacy” savefiles back in December. Because of the sheer number of savefiles we’ll probably end up with two different server clusters, but I can’t say for sure yet. If that’s the case then we’ll only need one server instance per cluster at the moment, so you probably won’t need to even think about instancing for awhile.

I’m sure that some of you have questions, so feel free to ask away. Thanks for reading!