How do we create a game server that permits unlimited freedom of movement and game play on an expansive map with no apparent boundaries?
We are developing the server for a massively multiplayer online game with a client-server architecture. The game design seeks to create an immersive play experience by enabling thousands of players to interact with each other in a single huge virtual world.
- The game design depends on having many players online at the same time.
- The game design requires the virtual world to simulate a single, large and unbounded open space that permits total freedom of movement and play.
- The game design targets player population ranges for specific regions of the world based on features, player character traffic patterns, and game play activities expected to occur in those regions.
- The game server uses a distributed deployment architecture.
- The game server uses the Distributed Network Connections pattern to manage game client connections.
- The product plan calls for scaling to handle growing player population through creating additional game shards, or duplicate instances of a complete server cluster and the game world it contains.
- The business plan accepts that operating and support costs will increase in proportion to player population.
Implement a variant of the Map-Centric Game Server pattern that subdivides the world map into smaller region maps distributed across a uniform grid.
As with the Map-Centric Game Server pattern, create two server types, area server and world server. Area servers manage region maps and transfer player characters between them in a way that masks the existence of server boundaries, creating the illusion of a seamless world. The world server tracks player character location and orchestrates high level cross-region activity.
The server cluster consists of many area server processes connected by a single central world server process. Use the Distributed Network Connections pattern to manage client network connections.
Area servers manage region maps and control all game behavior that occurs on them. Each region map occupies a single cell of the large uniform grid that comprises the entire virtual world map.
The simplest application of this pattern assigns a single region map to a single area server, but this may be inefficient.
Players usually congregate around points of interest (POIs) like towns, vendors, spawn points, and along common travel routes. For the sake of interest and aesthetics, POI distribution may not be uniform across regions. As a result, some region maps will experience higher game play load than others.
The Map-Centric Game Server pattern implements a straightforward way to address this. That is, an area server process may host multiple region map instances. Simply group several lightly loaded region maps together in the same area server process. More heavily loaded regions would call for a dedicated area server.
The world server serves as a central controller for area servers. It assigns region maps to area servers and coordinates movement of player characters between region maps and their area servers.
The world server is the central authority on a player character’s location within both the virtual world and the area server process nodes that make up the game server cluster. When a player character moves between area servers, the world server updates the routing data used by the connection servers so that client messages for a given player character reach the correct area server.
Moving Between Maps
The Map-Centric Game Server pattern specifies that when a player’s character moves between maps in the virtual world, the character’s state moves from the old map instance to the new one. That pattern implements the more general use case, where characters travel between two maps that are not adjacent region maps.
More complex use cases arise when player characters cross a region boundary or interact with game objects on the other side of a region boundary. These use cases are the key motivators for this pattern. This pattern’s goal is to simulate a vast contiguous world, hiding the detail that the world is really just a composite of smaller maps.
To do this, the game must make movement and interactions across region map boundaries as fluid as possible. Ideally, this means, in order of importance:
- Players experience no lag or hitching when moving their characters across region boundaries.
- Players must be able to see other player characters, NPCs and game objects on the other side of a boundary.
- Players must be able to interact with other player characters, NPCs and game objects on the other side of a boundary. Here, interact means a subset of game play that could normally happen between characters and game objects on the same map.
Moving Across Boundaries
To support moving player characters across region map boundaries, you’ll need to:
- Decide whether you can just load the character data from the persistent store (or cache) as you would with a normal map transfer. If you can do it fast enough, great. If not, then…
- Implement support for serializing the player character and all relevant game state, sending it over the wire to the remote area server, and deserializing it in the target map instance.
- Relevant game state usually includes equipped items or wearables, persistent game effects like buffs/debuffs, and the like.
- As a rule of thumb, include anything that must be immediately available to the player character or other game objects during game play.
- This must happen nearly instantly, within a few tens of milliseconds, to minimize lag.
You’ll also need to overcome several challenges:
- If you do send character state over the wire, when it arrives it must agree with that which would normally be loaded from the persistent store. You must decide which copy is authoritative, and whether to load the persisted data at all or simply save the transferred copy. Timing matters, and you must reconcile any state changes that happen after the transfer and save them as soon as possible. Getting this wrong has been the cause of some famous “item duping” exploits in some well-known games.
- You’ll have to handle movement edge cases and race conditions. These are complex and possibly unbounded. Examples include players rapidly moving back and forth across the same boundary, attempting to straddle a boundary while moving along it, and attempting to move between multiple maps at a junction. The most obvious solutions involve restricting movement and imposing transactional barriers to prevent these cases, but they add latency, defeating the purpose of this pattern.
- You’ll have to decide whether to allow NPCs and other AI-controlled characters to cross map boundaries. Supporting this would probably cause an explosion of complexity. Not supporting it will limit game play significantly. In some games this has caused anomalies where AI-controlled characters line up along a server boundary because they can “see” target characters on across a boundary, but not follow them across.
Seeing Across Boundaries
To support the ability for players to see game objects across region map boundaries:
- Enhance the server visibility system to include game objects on a map adjacent to the player character’s current map that should be visible to that character. This is the subsystem that determines which game objects are visible to each other.
- This means that the area server for each region map must send position, orientation, and state update messages to the area servers of adjacent maps when game objects are within some visibility threshold of a region boundary.
- In turn, these adjacent area servers must track these remote game objects in its visibility graph as if they were local, but located beyond the normal map extents.
Game Play Across Boundaries
To allow players to interact with game objects across region map boundaries:
- Implement support for sending game events initiated by a player character (or other game object) on one region map to game objects on an adjacent region map’s area server.
- This really should be built on top of an existing foundation that uses async message passing for all game events. If that exists, implementing this will be much easier. If not, you might want to reconsider doing it. Depending on your definition of interacting, the scope of work here could range from ranged combat to item trading
We have an MMO game server that simulates a vast, contiguous, virtual world that allows unrestricted movement and game play. This provides a deeply immersive play experience, at the cost of greater implementation complexity.
This pattern shares the scalability challenges of the Map-Centric Game Server pattern. In addition, this pattern exacerbates the scaling problem by limiting region maps to a single instance each. This leaves few options for handling increasing player load:
- If possible, increase the number of areaserver processes (and possibly hosts) so that each process handles fewer region maps.
- Add new points of interest such as towns, vendors, and spawn points to less populated regions to improve game play distribution across region maps.
- Increase the size of the virtual world by adding new region maps at the edges of the existing grid.
The allure of a vast seamless virtual world with open-ended game play is compelling. For some players, it’s the epitome of what an MMO represents. Game designs that seek to create this illusion must build a seemingly unbounded landscape from smaller parts, and develop a way to move between them that appears seamless.
This pattern describes a way to do this, but with significant technical investment and risk. In some ways, this pattern is really an antipattern. The technical challenges and risks that it brings may outweigh its benefits in most cases. Also, it inherits a key flaw from the Map-Centric Game Server pattern. Namely, it falls short with respect to the separation of concerns.
As in the Map-Centric Game Server pattern, the areaserver assumes responsibility for all game play, character movement, physics, and visibility. This leads directly to the biggest technical issues associated with movement, visibility and game play across region boundaries.
One of the biggest architecture constraints of the pattern is the need to distribute the world map geometry across multiple processes. By having areaservers implement both game play and geometry-centric operations, we automatically tie game state and physics state together. This, in turn, forces the areaserver implementation to transfer all game state between processes when players move, view, or interact across region map boundaries.
This pattern may be appropriate when:
- The Map-Centric Game Server pattern is inherently appropriate for the project.
- The illusion of a vast, seamless world is essential to the game design, and to player immersion and enjoyment.
An approach might be to separate game play from the geometry-oriented behavior, as is done in the Responsibility-Oriented Game Server pattern. This would simplify movement across boundaries by limiting the affected state to just that required by the movement and physics systems. Likewise, it would simplify visibility for the same reasons. Finally, it would completely eliminate the need to move game play state, because the pattern inherently supports game play interactions between server processes.
- Distributed Network Connections
- Client Side Load Balancing
- Map-Centric Game Server
- Responsibility-Oriented Game Server