Launching large-scale multi-player online games requires considerable investment and effort to purchase and manage the computing and network infrastructure necessary to deliver a consistently satisfying gaming experience. The traditional approach taken by most publishers and game providers is to install a dedicated infrastructure for each game, resulting in high risk and expense, and potentially poor use of resources. As the number of online games grows, it is increasingly difficult to predict the success of a title at launch, or its popularity as it ages. This problem strongly motivates new models for deploying computing resources to support the plethora of online titles slated for release over the next few years. The games industry can benefit from the ability to leverage infrastructure across game properties in an on demand fashion to better support the current and future online games environment.
In this project, we develop and implement a virtualized, on demand service platform for games, consisting of automatic game infrastructure provisioning, bandwidth-efficient distribution of game content, and policy-based software deployment services for game publishers and developers. The platform is built on Intel-based Linux servers and a combination of off-the-shelf commercial, open-source, and custom software components. As a part of this project, we have also collected and analyzed a large amount of data from different game servers in order to better understand user behavior, server and network load, and their impact on the on demand hosting platform.
One of our aims in developing a game hosting service platform is to investigate the opportunity for multiplexing infrastructure across business applications and games, for example to better utilize servers and bandwidth during idle periods. We have compared the workload characteristics of a variety of interactive business and educational Web applications with several games, and found a clear offset in peak load periods, particularly on weekends (see Figure 1). We are currently investigating non-interactive business applications which may provide an even greater opportunity for sharing with online games.
Figure 1. One week comparison of interactive business site and popular action game workloads. The challenge in developing an on demand infrastructure provisioning for games is to create a solution that meets the unique real-time requirements of online games. In contrast to traditional Web-based applications, many games demand very low latency and high responsiveness from the server and network infrastructure. Moreover, measuring system-level metrics such as CPU load, memory usage, and link bandwidth utilization, is not necessarily sufficient to characterize game performance. Various games and game types require specialized application-layer metrics, such as the number of players, spatial distribution of the player population, or game state computation delay, to correctly model their performance. One of the important aspects of our work is identifying the metrics that correctly reflect the performance of gaming applications. We have instrumented gaming applications and developed techniques for collecting and analyzing load information for use in resource allocation decisions.
Another challenge in on demand provisioning of gaming application is that the provisioning system itself is required to be highly responsive when load increases. Our studies have shown that players are impatient when trying to connect to game servers, with less than 20 percent willing to return after a single timeout. Our solution to this problem is predictive provisioning. Our analysis of workload data collected over more than a year for different games shows that the player population exhibits stable temporal patterns, particularly for mature games. We can exploit these patterns to develop predictive models that enable the provisioning system to anticipate peak periods and make sure that sufficient resources are available. We are currently exploring predictive models that can use game workload patterns to deploy game servers slightly in advance to further improve the perceived provisioning delay.
Figure 2. Weekly half-life player population over a four-week period.
Figure 3. Game provisioning platform diagram and management console. In addition to infrastructure provisioning, we have developed additional reusable components that are useful in a game hosting service offering. In our discussions with online game publishers and producers, we found that network bandwidth is a major cost factor in running a game, both in traffic as well as content and patches that have extraordinarily high demand over a short time period as soon as they are released. To address this problem, we have developed a content distribution service that uses game clients and servers in the data centers, to establish an ad-hoc delivery network to distribute large game patches and content. The service allows simultaneous download from multiple nodes, servers as well as peer game clients. The set of download nodes are chosen according to their network proximity and connection speed with respect to the client.
Finally, the platform also includes a software deployment service to enable game publishers and developers to deploy their games without understanding the complexities of the provisioning system. The publisher can express system requirements (e.g., disk space, memory requirement, system libraries, network bandwidth, etc.) and also policies on the minimal number of servers that should run the game, or the maximum number of servers that the publisher is willing to pay to run the game. Our implementation of the software deployment service is in the form of a Web-based portal.
We have demonstrated the feasibility of the platform by provisioning multiple instances of Quake II, a popular action game available as open source. In order to develop and test the platform, we also developed client-side “bots”, synthetic players that emulate real players in the game to allow us to experiment with various player workloads.
The project has been an important vehicle to demonstrate IBM’s on demand technology in the context of gaming applications to a variety of customers in the industry. The on demand platform for games was also featured at the IBM Game Industry Press Briefing held in Hawthorne in August 2004, and covered by the Wall Street Journal, New York Times, Financial Times and Spin magazine among others. |
|