Apples and Oranges
“Shirkying” – a singular word describing the style of Valleywag‘s Clay Shirky when he attacks Second Life with a mix of bad statistics and biased opinions — seems to be spreading and becoming more popular. In effect, when talking about Second Life®, being able to successfully predict the immediate future becomes a pastime of many, not the least of yours truly, but so many others engage in this ever popular exchange of opinions, that you have no option but to adopt a specific style to push your views ahead.
Recent trends published by the New World Notes, GigaOm, or the Second Life Insider — and even BusinessWeek! — all seem to point in the same direction: corporate presence in Second Life is declining somehow. They come, see the hype, never gather enough attention, and go. What is to blame? Depending on who you read, the opinions differ.
Some, like Tateru Nino and Hamlet Au, imply that the issue is simply dealt with statistics — corporate virtual presences do not have any impact on the metaverse culture, thus they don’t attract visitors, and the interest in staying around in SL lessens — and they eventually go, sooner or later. Others suggest poor marketing skills that are unable to tap into the potential of SL, and until this is better understood, corporations simply wait and stay outside SL, waiting for further enlightenment. And, naturally, the lack of security (against griefers) and control (of the server grid) are also pointed out as reasons for leaving SL, or at least for not investing as strongly as before.
It’s worth considering these reports, news, and claims for a bit. But first I’d like to thank Moon Adamant for the following metaphor, which I’ll leave for you to meditate upon:
A soccer club and a bank both invest on a huge TV marketing campaign. On the following Sunday, the soccer stadium is full with perhaps 80,000 soccer fans. At the end of the week, the bank attracts a thousand more mortgages. When looking at the poor performance of the bank in failing to attract more customers — compared to the soccer team — should the bank simply give up on advertising? Look at their respective websites. The soccer team has information, news, and forums. A couple million users post there every day; the site is popular and attracts lots of ad money from Google AdSense or similar sites. The soccer team invests on more technology: video (crossposted to YouTube), merchandising being sold on their e-shop. Hundreds of thousands watch the videos; a few hundreds buy the T-shirts. In the mean time, the bank also invests on web technology. Their site now offers homebanking, FOREX exchange, tickers, and fresh news from Wall Street. 500 new customers sign up and a few even do some online trading. Was it worth to expand their corporate presence on the Web?
What are we measuring?
As mentioned on my previous article, one of the issues currently very heavilly discussed is metrics. And the point is easily understood even to a non-expert: anyone launching a business in SL — or at least doing advertising on a business – needs to have an idea on how successful their virtual presence is. But metrics go beyond that: for teachers and educators, it also means being able to properly analyse what their students are, in fact, learning. For sociology researchers, it means trying to figure out how this whole environment behaves, how it affects us socially, how we establish connections and trust. And for Linden Lab’s developers it means understanding how people are using SL, how stable it is, how the performance is measured, how the impact of a new feature or bug will spread like wildfire across the grid (or have no impact at all).
Clearly, the word “metrics” has quite a large application base, and it definitely means different things to different people. It’s another one of those cases that trying to squeeze a simple definition into a vast array of possible meanings and applications. And obviously we’re getting into all sorts of problems because of that.
We can’t escape the simple fact that SL is a computer-based technology that runs on top of the Internet; people used to those two environments (computers and networks) will naturally pull data out of SL conforming to those two areas (which today are neatly blended together). So when we talk about “SL performance”, we’re used to look at the frame rate per second; if we talk about “sim performance”, we look at time dilation and script instructions per second; if we want to have an idea on how bad our connection to SL’s grid is, we look at packet loss. These are both very concrete metrics (like FPS) and abstract ones (such as time dilation) which are directly tied to hardware, operating system, and network performance, so it’s “natural” for a computer/Internet guru to rely on them.
They don’t show the complete story, as any content creator knows. A sim can “lag” for several possible reasons — one of which being, for instance, the polygon count that is rendered on a specific view, and that depends, as everyone knows, on how many avatars are in the field of vision — as well as on how many “tortured prims” are around. But it also depends on several other conditions — like how big the textures are, or how many of them are visible. And it depends on the network stability, which can often be misleading when our client reports 0% packet loss. Worse than that: two sims can display exactly the same set of statistics, but have completely different behaviours, which can even be time-dependent (the same sim, with 40k+ users online at the same time, will behave much worse than with just 20k+). Almost all SL residents are aware of this abstract interpretation of the numbers, and after enough experience in being in-world in SL, they can figure out, based on what the numbers tell them, what might be wrong (and, in some cases, fix it).
A simple analogy can be seen on the weather prediction. If you see something like 27º Celsius and no clouds, you expect to dress on your summer clothes, because it’ll be warm. But it might not be the case; a chilly wind might blow and the humidity might be very, very low; in that case, it’ll feel much colder than it actually is. As humans born into a world with constant weather reports, however, we’re pretty good at figuring out how the weather will really feel like. 27º C in January will feel like a blast of heat from the furnaces of Hell (specially if it has snowed the week before!) and be very, very uncommon on the temperate regions; in August, with some nice breeze, it might be even a balmy, comfortable day. The psychological aspect of interpreting numbers related to the weather is often more important than “scientific fact” — a day with 27º C in January is precisely “as hot” as one with 27º C in August, but we certainly will experience it quite differently!
There is obviously something we need to measure to be able to make reasonable, factual statements about what we experience, and, if we’re scientific about it, providing others with the same set of data will allow them to reason independently and come to the same conclusion. In the business world, marketeers need to have an idea on how big their audience was after a campaign was launched. The marketing world has devised measurements and statistics for about every area where you can place an ad: from outdoors to newsletters, from junk mail (in your physical mailbox) to TV ads. And, naturally, on websites.
Internet “oldies” remembering a webless Internet (how strange that might sound for people that always used the Web as their entry point into the Internet!) will also remember the hefty discussions around “metrics” on web servers. The most basic numbers came from how many pages a specific website has served; obviously, that number was pretty worthless except for system administrators needing to tweak the performance of the webserver, since the same person might click on the page several times, or a page might have several components (images, videos, etc.) and hit the server multiple times. Over several years, these metrics were fine-tuned to better reflect what people are actually experiencing. They would also apply to different servers, under different conditions (ie. a slow server could be effectively compared to a fast one). Ultimately, not even that was enough: in order to plan for good website navigation, better metrics had to be devised, to allow marketeers and webdesigners to understand what people have been visiting on a website. Tools like Google Analytics allow very detailed analysis of a website and how successful a marketing campaign is to bring visitors to a specific area inside the site. While these tools are not perfect (people can click on links randomly and thus offset the statistics) they are the best that technology and creativity has brought us in the past 15 years of the World-Wide Web.
Second Life, however, has two issues that aren’t addressed by number-crunching. First of all, we only have, by default, traffic — which is calculated using an obscure formula that is very hard to grasp. It was designed to work on a single parcel and difficult to “game” (which obviously is still “gamed” every day, thanks to camping chairs and dance pads). However, it’s not a reliable metric to establish — from a marketeer’s point of view — how “popular” a venue is. It’s also not directly comparable: a small, popular club in SL with lots of dance pads can have more traffic than a huge, multiple-island virtual presence stretching from horizon to horizon. Although we have similar examples on the real Web — having a lot of content on the Web doesn’t automatically mean that there are a lot of visitors! — on Second Life the values are so relative that they can’t really be compared. If a website has twice the visitors (independently of the number of content) you can reliable assume that twice the number of people have visited it in the same amount of time. If one parcel in Second Life has twice the “traffic” than the one next to it, you have no idea if it has been visited by twice the number of people, the same number of people but staying there twice the amount of time, or just one person logged in there 24h/7 (and most likely an alt or a ‘bot that has no human actually viewing the content).
Traffic is nice for ordering the Search results, but pretty much useless for anything else. Still, it is a metric, and in a sense, a parcel with 6,000 of “traffic” is probably “more visited” than one with just 60. Since it’s parcel-based, it also gives no clue as to what people are actually viewing on that parcel — they can simply be on a dance pad and doing nothing — or how effective the content displayed there is, to attract visitors to either view information and/or buy content. Finally, it gives no clue on how people have found that parcel — unlike the Web, where we can figure out pretty much the success of a marketing campaign seeing where all those people come from. And, naturally enough, there is no reliable mechanism to profile your users — not even separating them according to simple things like country of residence, for instance.
No wonder, thus, that professional marketing companies in SL are relying on different models of establishing metrics. With a combination of sensors and phantom’ed collision-activated prims, they can measure pretty accurately who has visited, who is a returning visitor, how much time they spend on a parcel, and how they have navigated across the content. By tracking down what items an avatar has interacted with (eg. what clothes they’ve bought; what notecards they’ve retrieved; what movies they’ve played with information on products), a further layer of information can be conveye, number-crunched, and presented as a report to the marketing group. SL content creators with their shops have long ago relied on simplified versions of those statistics; corporations are adding a further layer of more complexity in their attempt to estimate how successful their virtual presence has been.
Interpreting the numbers
We move than to the next level of abstraction in measuring the success of a virtual presence. Armed with the data suggested above, content can now be tweaked — if an item is never “found”, it might need to be displayed in a different way. If areas of a sim go totally unexplored, they might be removed or changed. If people read the information but never buy anything, something is wrong either with the type of offerings or the price. If they log in, stay a few minutes and go — a typical “browser” — the virtual presence is not being effective in capturing people’s attention.
The numbers don’t provide a “magic” answer. They can, however, aid people to reformulate their assumptions. Someone might have their clothes shop near to a beach, where a social space has been built, and they can then estimate what happens: do the avatars buy a few clothes and then go and try them out on the beach? Or do they go straight to the beach and chat with others, and basically ignore the shop altogether? Or do they bypass the beach completely and focus on the shopping? Depending on the paths they take across the parcel(s), the data will be different — and a marketeer or business owner, frowning over the numbers, can design a better model. Perhaps people like so much the beach that the shop is basically worthless — the beach should be the focus of attention, with lots of social events there, and clothes could be displayed around that area.
Still, profiling the customer is very hard in SL. It’s not just the anonimity; it’s the total lack of connection between someone’s interests and what they do in SL. In effect, a similar thing happens on the Web. Unless you have a very complex e-commerce site (eBay, Amazon), it’s hard to profile your customers and understand why they come to your site. You can see from where they’re coming — eg. randomly, through a search on a search engine, or from links from other sites — but that’s about it. The rest is just wild speculation — unless you are able to build feedback mechanisms somewhere that tie to the effectiveness of a campaign. One example is spamming people and tracking its effectiveness by placing a link with a “hidden” key: that way, spammers know not only that the email was delivered and read, but who went to the site by following the link. Yes, it’s a bad example — from someone who has been condoning spamming actively and publicly since 1996 — but, sadly, spammers are pretty good at these kinds of profiling mechanisms. A legitimate use, of course, is asking the potential visitor for more personal data and give them access to restricted areas of a website. The sample will be much smaller, of course, but richer in information; if you can get a few million users registered that way, you’ll be able to figure out much better who your visitors are, where they come from, and what needs/tastes they’ve got, and thus better target the website to them. Thus, you might not need millions of users hitting your site to give you an idea on what they want to do there when visiting it; instead, having perhaps several thousands of fully-profiled users will give you a far more realistic picture. This is “basic webmarketing” for you.
How does this apply to Second Life? Consider the initial example, and let’s have both a soccer club and a bank inside SL. The soccer club can create a link on their webpage — visited by millions of users — and give them a code that can be presented to a device in SL to give them a freebie T-shirt, branded with your soccer club’s logo. That way they can effectively measure the impact of a campaign to attract visitors into their virtual presence. They already know that most people that are going to get a T-shirt come from a prepared web-campaign; they can thus leverage on that information and see how many T-shirts were really given away, and compile the information.
In terms of SL, this soccer club would certainly have “a lot of traffic”; but they could be much more precise than that. They could see that the code was redeemed by, say, 100,000 users, and of those, 70,000 would have come to SL and retrieve their T-shirt. Obviously, they would know that they have “millions of users” visiting their websites; but only 100,000 would have shown an interest in logging to SL. 100,000, however, would be a huge number — when compared to a similar campaign, say, on the radio, where people would be able to make a phone call to redeem a scarf (these usually get thousands of calls at least). A marketeer working with the numbers would thus be able to conclude that a campaign tying into SL would give a much higher return in mere brand awareness than a radio campaign giving out scarves. Now they would be in a position to estimate costs and return on investment: how much does it cost to produce a few thousand scarves and set up the organisation for delivering them to the people that call that phone number? And how much does it cost to create a virtual presence that gives away virtual T-shirts? How many people, in each case, will be exposed to the soccer club’s brand? These are the kind of metrics that people would be able to measure and calculate to establish “success” of their virtual presence in SL, when compared to other media.
Similarly, a bank can do a campaign on the radio, offering discounts on mortgages if you call up a certain number after listening to the radio add. Or they can use the same system inside SL: after visiting their virtual presence, you can touch an interactive object that will give you a code for a promotion that you can redeem on a bank’s agency. These and similar marketing ideas have been actually tried out inside SL, and by measuring how many people actually get those codes and redeem them, a corporation can have an indication on how successful that strategy was.
However, very likely, the soccer club will have many more people redeeming those codes to get a scarf than a bank would have in new mortgages! This does not mean that the bank’s campaign was a “failure”. In fact, it might even have been a broader success in the sense that they have attracted a number of customers that will pay them a recurring monthly fee for 30-40 years — while the soccer club, on the other hand, might have given away 70,000 scarves, but will all these people be next Sunday watching the match? There is no way to know.
What has to be understood here is that soccer marketing and financial marketing are very different fields of business. They might use similar techniques — and definitely the same media — to raise brand awareness and get new customers. However, when comparing their metrics, they’ve got completely different goals. You can’t simply say that “this marketing strategy worked for a soccer club, so it should work for a bank as well” — that’s simply common ignorance: such dramatically different business models require very different strategies to enhance their brand awareness and get new customers, but also different metrics. Even if at the end of the day, both use the same media to do their advertising: newspapers, magazines, billboards, radio, TV… and the Web and Second Life.
Coming to the wrong conclusion is easy
A disturbing number of articles have recently reported about “the lack of interest” of SL residents in corporate virtual presences. Their usual argument is: “nobody wants to visit a corporation in SL, they lack interesting content and appeal to bring in recurring visitors” and the “proof” presented is that these spaces do not have enough traffic to justify their presence.
These are all bold claims based on the assumption that a single metric is useful to compare all corporate marketing strategies, and also assume that corporations in SL are, indeed, interested in competing with existing content! Truthfully, they are not, but we’ll have to fall back to some analogies and actual examples to understand what they’re after.
Sunbelt Software is a software development company doing anti-virus/anti-malware software. They have their own 30,000-large community that regularly keeps in touch using the Internet — through forums, chatrooms, and similar Web 2.0 environments. Not all participate in all platforms/sites, though — some favour forums, some go to chatrooms, some simply lurk, a few are very active. When Sunbelt decided to open their virtual presence in SL, 5% of their existing users actually registered through Sunbelt’s site to get the “Sunbelter” last name (not available elsewhere). Many have also registered in-world to get a free copy of the latest anti-virus software, which was released about the same time their virtual presence in SL has started. Their virtual presence in SL is targeted to the Sunbelter Community of users: after a short orientation area, there are community spaces to hang around, talk about Sunbelt products, even a small sandbox, and obviously a club, or a larger auditorium — these are meant for the community to use. There is also the big “corporate building” but it has resident-run shops at ground floor (as well as a few Sunbelt-branded items). So how much traffic does this private island generate? Is there a ROI?
Both questions are actually not answerable in the usual “SL-business-sense”. Sunbelt Software cares mostly about their own community. They’re not in SL to create huge virtual spaces for communities with 7 million users, and become the traffic leaders either in event hosting, clubbing, or shopping. Their purpose is just to provide their Internet-savvy community with an additional place to meet and talk and exchange tips about Sunbelt’s products, and only very marginally attracting outsiders that might be in search of a new anti-virus solution; “SL-at-large” is too vast, they aren’t expecting to attract all 7 million users to their island, and that was never their intended purpose! On the World-Wide Web, out of the billion users, perhaps only 100,000 (0.01%!) — according to Alexa — visit their website. Sunbelt very likely would be happy to attract the same ratio on SL: just 700 users! But they have already got twice as much registered SL accounts created through their site, and very likely (I haven’t asked!) a few thousands of casual visitors have dropped by their island. In effect, the SL population knows more about Sunbelt than the broad Internet population. This metric — and not any other — is their measure of success and validates their investment.Similarly, as an European, I had never heard about American Apparel or companies like Text100 if it weren’t for their virtual presences in Second Life. What this means is that all this corporations are measuring how successful they are inside Second Life when compared to the Web at large — or compared to any other medium, in fact. They would be expecting about the same impact inside SL than they have on the traditional media; but actually, they’re seeing overwhelmingly more “casual visitors” impacted by their brand inside SL than outside. And this, of course, is what keeps them coming to SL!
On the Web, the corporations aren’t competing with YouTube, MySpace, or FaceBook, either. Just because a company has “only” 0.01% of the traffic on the Web — and not the 10%-40% that the “top ten” websites of the world have — it doesn’t mean they’re not doing a good job of attracting people to their websites, or that they are not successful and considering abandoning their websites. Claiming that would be ridiculous.
However, this is precisely the same thing that is currently being claimed using the traffic metrics as a measure of success of virtual presences in Second Life. We need to be very careful when making those claims. Dozens of millions of companies are on the Web, and they’re not competing with Google, Yahoo, or Microsoft Network for the top position; rather, they’re disseminating their information for the ones looking for it: casual browsers, people interested in their products and services, people searching for information on related aspects of their activities. Many are also creating their own Internet-based communities, through blogs, forums, and lately, chatrooms — their websites are for their users, and for people wanting to know more about their companies.
When these companies started to come to Second Life, they never wished to beat the popular clubs or shopping malls or the casinos. Claiming otherwise is just comparing apples to oranges; while nobody minds some more extra visitors, they are not “resident traps” in the sense that they expect even to come close to “compete” with the popular spots in SL. IBM or Dell are not going to drop a few casinos and camping chairs to drive their traffic up on the search rankings — they have rather other customers in mind, and they measure their success quite differently. Saxo Bank, a Danish bank, on their first day in Second Life, was visited by 5000 people in 24 hours. This might not mean much, but if you take into account that this was 10% of the Danish population in SL at the time, you can imagine how happy the corporate marketeers at Saxo were! Sure, they’re aware that any of the ten top spots in SL get those 5000 people every day. But then again, Yahoo.com is viewed by 40% of the Internet population, and a Danish bank is not competing with Yahoo on the Web, either.
As Second Life becomes mainstream and attracts more corporate presences, it also attracts more professionals on all areas, and marketing is certainly one of them. Second Life, as reported elsewhere, has been seen as being part of the rise and the triumph of the amateur that is able to establish successfully their own content, business, and brand, without any business skills whatsoever — and the popular spots are usually very good examples of it. However, appealing to the masses is one thing — or we wouldn’t have MySpace, YouTube or FaceBook as huge successes — while successfully targetting your marketing efforts towards the audience you wish to capture is something completely different.
Thus, what drives a resident to create the “best club” on Second Life, or Pontiac to increase their brand awareness in SL, is quite different. Their individual success is measured differently. The “best club” will probably measure their success in how much traffic they generate (since it allows them to be ranked higher on SL’s Search engine — at least, until LL removes traffic from the calculation) and if they’re able to make a comfortable living out their SL virtual presence. Pontiac, however, does not want to sell cars through SL — and even if they did, probably by selling just one car per month to one the many visitors of their sims, would pay for all the recurring monthly costs with their presence in SL. They would also make far more money with just a single sale than a crammed-full club on a sim. That’s not what they’re after — they’re more focused on giving their visitors the notion that Pontiac is a ‘friendly’ brand who is not afraid to work with the SL community and give something for them to have fun (Nissan’s approach is very similar). Renault is sponsoring the soon-to-be-announced Second Life Formula One Racing Competition — they’re not expecting millions of avatars to view their ads pasted on the SL F1 replicas, but just making the few racing addicts in SL happy that they can count with a little advertising from such a huge French brand. It follows that “attracting millions of residents” is very, very marginal on most corporation’s minds — they prefer, like they do it on the Web, to attract the thousands that might be directly interested in their products and services. And this they’re doing admirably well — as can be measured with their own metrics: compared to their websites, their virtual presences in SL attract a much higher relative number of visitors.
A couple of years ago, a lot of residents on the forums voiced their “fear” that they would be “pushed out” of SL because corporations would have enough money to “buy the out” and establish a “corporate metaverse” without the need of the talented amateurs that managed to survive in SL pushing their content for a few Linden dollars. I always found those claims perplexing in their ingenuity. It would mean that things like MySpace and YouTube would be doomed once professional designers and video producers would set up their own sites; but as we all know, this never happened. In effect, video producers and TV stations are releasing their content on things like YouTube, but they’re just a bubble in the ocean of amateurish production showed there. But then again — they’re not on YouTube to push the amateurs away and become the “best” and “most famous” youtubbies there. Instead, they want to explore a niche and say: “we’re here, too, if you want to look at our content” (which not everybody among the 150 million YouTube users wants to do).
So, while the fears of “corporations displacing amateur content” were pretty much ungrounded — American Apparel did not sell more clothes than Pixel Dolls! — the claims that “corporate virtual presences are not effective in generating traffic” are also rather hard to validate, without access to the marketeer’s metrics that enable them to measure success and a return on investment. Some of the most proeminent analysts in Second Life are so good at measuring apples that they think they’re able to count oranges as well.
- 02 June 2007 at 10:06pm
- WebMetricsGuru 17 June 2007 at 10:06pm
- Moderne-Unternehmenskommunikation.de 26 August 2007 at 7:08pm
- Gwyn’s Home » Please get your facts straight!
Second Life Metrics - Gwyneth Llewellyn... A friend sent me this link - ...
Und nun… Second Life metrics... Autor: Jörg Hoewner Die ersten ...
[...] writing previously on the subject, I commited an unfortunate mistake — I simply ...