Monthly Archive for January, 2009

[This is a cross-post from Princeton S* Network Systems (SNS)'s Dirty Slate Design Blog]

Introduction

I’m currently working on a project called Sirikata, an open, programmable, scalable, secure, and extensible virtual world.

One aspect of this virtual environment is application-level messaging. Users can create objects in our world, and these objects can communicate with each other. To get an idea of what application-level messaging looks like, we wanted to take a look at one of the world’s biggest virtual worlds: World of Warcraft (WoW).

In WoW, users can create an “Addon”: a user-interface modification component that can add enhancements to the game. To communicate, these addons (written in Lua) have access to a function called SendAddonMessage.

WoW SendAddonMessage Function

Sends a message to the hidden addon channel.

SendAddonMessage("prefix", "text", "type", "target")

• prefix: String – Message prefix, can be used as your addon identifier.
• text: String – Text to send.
• type: String – AddOn channel to send to. Valid types are “PARTY”, “RAID”, “GUILD”, “BATTLEGROUND”, “WHISPER”.
• target: String – Used only for “WHISPER” communication – the player to whisper to.

Example:

SendAddonMessage( "CTRA", "User Data: XYZ", "RAID" );

Notes:

• Calling this function results in the event CHAT_MSG_ADDON being invoked on:
  • <target>’s client if <type> is “WHISPER” or
  • all clients in the <type> chat channel, otherwise
• The combined length of message and prefix can be at most 254 characters (255th is the tab character used to delimit the two)
• Length above 254 will disconnect you.

Example Addons

EPGP

EPGP is an Effort/Gear reward system and addon for World of Warcraft.

It is a DKP system, described below:

Dragon kill points or DKP are a semi-formal score-keeping system (loot system) used by guilds in massively multiplayer online games. Players in these games are faced with large scale challenges, or raids, which may only be surmounted through the concerted effort of dozens of players at a time. While many players may be involved in defeating a boss, the boss will reward the group with only a small number of items desired by the players. Faced with this scarcity, some system of fairly distributing the items must be established. Used originally in the massively multiplayer online role-playing game Everquest, dragon kill points are points that are awarded to players for defeating bosses and redeemed for items that those bosses would ‘drop’. At the time most of the bosses faced by the players were dragons, hence the name.

EPGP uses the messaging system to send information between users of the addon about the values of loot. For example, if an item drops for the first time, the “loot master” might assign a value to the item. This value is then shared with other people using the epgp addon.

Properties

• small message size
• very infrequently sent
• must be delivered

Gatherer

Gatherer is an addon for herbalists, miners and treasure hunters in World of Warcraft. It’s main purpose is to track the closest plants, deposits and treasure locations on your minimap.

In WoW, players can gather resources. The resources are spread across the world and spawn in what seems to be a randomized fashion. In fact, though, the spawn locations are actually preset, so this addon tracks the location where users find resources to make it easier to find later.

Gatherer uses the messaging system to share the location of found resources between players. For example, when you find a resource, you can have it automatically share the location with your guild members. This way, if many people in your guild use the addon, you will get a nice compiled database of resource locations.

Properties

• small message size
• medium send frequency
• not important if messages are delayed
• not important if some messages are completely lost

Recount

Recount is a graphical damage meter.

In WoW, players fight bad guys. To do this, they have to deal damage. A metric for evaluating players is to look at the amount of damage they deal per second (DPS).

Every action that players within your party take is logged, which includes actions that result in damage being dealt. However, you only get log information for players that are within a short range of you. Because of this, in some encounters, some players might be too far away from you for their actions to be logged.

To make the meter more accurate, Recount uses the messaging system to aggregate logging data about damage being done.

Properties

• Medium size messages
• Heavy traffic load during encounters (about 10 seconds to 6 minutes)
• Latency effects instantaneous accuracy during an encounter
• Syncing algorithm still works if some messages are lost

Data Collection

To study WoW, I wrote an AddOn called AppMsgLogger. This AddOn collects three pieces of information:

1. A timestamp every time the user logs in or out
2. For every addon message received (the CHAT_MSG_ADDON event), a timestamp, the zone the user is in, the channel it was received on (PARTY, GUILD, RAID, etc), and the length of the message in bytes
3. A list of the AddOns the user is running

I had several players run this AddOn for me and send me the data they collected. As a result, I have data consisting of 49 hours of time played across eight players.

Results

Out of the 173,456 messages received, here is the breakdown of the channel they were sent on:

• Battleground: 262 (0.15%)
• Whisper: 1,240 (0.71%)
• Party: 3,185 (1.84%)
• Guild: 21,439 (12.36%)
• Raid: 147,330 (84.94%)

So, only less than 1% of the traffic is unicast (Whisper) while the majority is broadcast traffic related to combat (Raid) and social organization (Guild).

Next, I wanted to look at the breakdown of message size. Here’s a histogram plotting message size frequency:

You can see in this histogram that the vast majority of messages are small (< 50 bytes), with a small cluster of messages near the maximum size of 254 bytes. This is probably because most addons use a library called AceComm which splits messages longer than 254 bytes into multiple messages.

Next, I wanted to see the distribution of messages across the different addons sending them. I matched each prefix string to its corresponding addon and plotted their message frequency:

Next, I wanted to look at how bursty the message traffic was. To get an idea, I took a single user’s 24-hour play time (over multiple sessions), strung it together, and calculated the average byte rate of each 1-minute interval (click for full size).

What you can see in this graph is that there is considerable variability: some periods show a high rate (up to 14 Kbit/s) while others show a rate of almost zero.

To put these numbers in context, I added a background color that alternates every time the user switched zones. For zones that were longer than 10 minutes, I also added the name of the zone in the center of its colored region. As expected, high-rate periods are seen during raids (Icecrown) while the user was engaged in combat. Low-rate periods are seen when the user was presumably idling in capital cities (Orgrimmar, Dalaran).

The last thing I wanted to see was what percentage of messages were wasteful. Since messages are broadcast to their corresponding channels (GUILD, RAID, etc), users in the channel will receive all messages sent to the channel regardless of whether they have an addon actually listening to a given prefix string. Since I had a mapping of prefixes to the sending addon, I calculated the percentage of messages that were received by each user with a prefix string for an addon that the user wasn’t currently running. The percentages for the users that I collected data from are as follows: 58.9%, 28.2%, 15.9%, 12.6%, 9.5%, 86.1%, 83.9%, 17.0%. Since the sample size is low and variability is high, I’d be hesitant to draw any conclusions from those numbers, but it leads me to believe that a significant amount of traffic that is broadcast is wasteful.

Conclusion

I hope you’ve found this analysis useful. We’ll be using some of this information as we continue to design and implement Sirikata.

Sirikata