Skip to content

Wrong Answer, Mr M Linden

The fear of different products

Another thing that always made me wonder was the reluctance of Linden Lab to offer differentiated products. It’s a well-known fact that more tailor-made products require more (human) work to deal with different types of clients — and complicate the invoicing — but somehow with computers doing the hard work, the point becomes moot.

Consider SL in 2005, or the OpenSim-based grids today. You’d place an order for a new sim. After billing approves your payment, an order is sent to an available technician, who would run to the co-location facilities, look for an available server, install the software, and, with luck, get it running in a short time, so that you can enjoy your new sim. LL used to deliver sims in “batches” (usually twice a week) since those were the days the operations teams would be physically at the co-location facility screwing servers to racks.

In those days, the quicker you could deliver a sim to the client, the faster you could invoice them again (for more tier), so, the less options the client had, the better. Hardware was standardised; and so was software: one sim has 256×256 m and 15,000 prims (less in 2005, but you get the point). There were no choices.

Now fast-forward to 2008. You have a fully-automated model that charges the customer after they pick a spot on the map, automatically locates the first available sim in the first available server, loads the proper server image (all automatically from a central repository), boots the sim, and delivers to the client. That can ultimately take just a few seconds — as easy as, say, installing WordPress from a hosting provider like Dreamhost. Sims boot very quickly  — if OpenSim servers are a valid comparison (even if they’re totally differently designed), you can boot a dozen sims in a bit less under a minute, and that on old hardware.

When providing a service does not require human intervention (except when something goes wrong!), the first thing that a service provider does is differentiation — basically, allowing you, the customer, more choices. So long as these choices are handled by computers and software, it doesn’t make any difference in the cost of providing a differentiated service.

Now, LL has basically three choices: mainland sim, private island, openspace (and now homestead) sim. Why so little?

It’s not a limitation of the server software. Things like the maximum number of avatars per sim, or the number of prims, are simple variables that are set when the sim boots (some things like the sim’s size, ie. 65536 m2 on a square 256×256, seem to be hard-coded on the client too, so they’re far harder to change).

On the other hand, and assuming that LL’s sim software is multi-threaded and works on multiple CPUs, you can start doing nifty things. Imagine having a whole server just for you! Now you could use those 4 cores on just a single 256×256 m sim — and get 60,000 prims and 400 avatars on it! Or, alternatively, configure it to give you 16 low-traffic sims, and spread the number of total prims among them. In fact, depending on how good LL’s code is, you could even fine-tune things more, ie. buying a sim with 50% of a CPU and get 7,500 prims on it. And the pricing, of course, would be flexible to adapt to all those variations.

OpenSim can do that at a pinch, and, as often repeated, it’s below-alpha software. So why doesn’t LL offer a similar service? Both the high-end users — getting 400 avatars in a densely-packed event would simply be awesome, and you could buy four servers and do a 1,600-avatar event on sim corners! — and the very low-end users would be happy with more variety on pricing. In fact, giving a constant — a Class 5 server can run up to 400 avatars, up to a 512×512 tile on the map, up to 60,000 prims, all on 1 GB Ethernet, for US$1,200 — that gives quite a lot of variety to pick from!

But there is more! For a short while in the history of SL, class 4 sims used to be cheaper in tier than Class 5 sims — you were allowed to pick the server of your choice and pay accordingly. Although I believe that Class 4 sims are only used to run openspace sims these days, LL pretty much gave up on the price differentiation (they’ve grandfathered the tier costs of sims running on Class 4 servers, however). With Class 6 servers on the horizon, people could have a further choice!

In fact, this is what hosting providers do. You have a choice of software, hardware, and features. It’s pretty much standard with every hosting provider — you, the client, pick the hardware you can afford, run the software you’d like to do, and select the features you wish. The combination depends on the performance you need and the money you’re willing to spend on it!

However, LL never went the route of “flexibility”, relying instead on one-size-fits-all, absolutely standard solutions. Why?

While a lot of human intervention was needed, I can understand the problem about setup costs. But now that it’s all fully automated…?

Instead, we get the following speech from M Linden:

“Because we have complexity everywhere else, we’re loath to add a highly complex pricing structure.”

There is still hope, since M Linden goes on and says:

“Nevertheless, it’s clear we have to build a product mix and pricing structure that offers more flexibility.”

… and then announces that for a more limited set of features you’ll have to pay more, which is completely and utterly insane!

So the message seems to be: “if you want more flexibility, you pay more than for a standard solution, and get a worse service”. Well, one can understand the theory that a “personalised” solution ought to be a bit more expensive than a standard one — not that you ought to get worse service! But, again, this makes sense only if there is manual labour involved. If the cost of setting up a sim is the same, no matter what its ultimate characteristics are, why should it be more expensive to provide a more limited feature set?

I’ve tried to think this over and over, and I really cannot understand, from the little input we get from Linden Lab, how they figure out their pricing. From an outsider it looks like they throw a random number out of their pockets and use it as their price, and then see how well it financially works. But that’s hardly what can be the case, given M Linden’s background and expertise. No, Internet hosting providers are mostly cost-based operations, so there is something missing here in the whole equation.

Ultimately, there is a very complex formula that has shown, on M Linden’s spreadsheets, that continuing to provide a “limited set of features” for a fair price will lead Linden Lab to ruin and bankrupcy. Where exactly that happens on the formula is impossible for us mere mortals to understand. But it has to be a very strong point indeed: one that makes LL decide to cut their income in 20% instead of continuing to offer that service. And I’m at a loss to explain such a decision.

Perhaps LL should try to use some OpenSim-based servers on their own grid :) They’d be able to run a hundred sims on a Class 6 server pretty easily, provided they’d have enough memory and bandwidth, and sell these for the “magic” price of US$10-30 per month. They’d be fully integrated with LL’s asset servers, so you’d still have access to those billions of objects. But they’d have lousy performance and crash a lot. The point is, they would be ultra-cheap and compete aggressively.

Being a Premium account resident, and having the choice between a 512 m2 plot on the stable mainland with 117 prims, or a totally unstable and unreliable private island on an underpowered server for the same price, I’d very likely go for the latter! And I’m pretty sure that most of the 90,000 Premium account residents would think the same way. So, instead of losing a million US$ and 13,000 sims from the grid, LL would, with this option, earn the same amount of money and run seven times as many sims (but use the same number of servers!) — while at the same time making everybody very happy!

It would make sense to me, if I lived in a sane world.

Instead, Second Life has become Surreal Life, and I cannot figure it out any more.

Related posts

Readers who viewed this page, also viewed:

  • Ananda Sandgrain

    Oh, I don’t know Gwyneth… I think you did find a reason while you were writing this. Namely, that Linden Lab is more interested in development than in being the biggest hosting provider in the world.

    As you mentioned, after introducing the OS product they basically saw the number of sims (if not servers) they needed to manage double in less than a year. That’s a crazy amount of growth to try and keep up with. I’m betting they’d rather just try to improve (and figure out how to save money on) a more sedately growing world. If the world grows at a slower more manageable pace it buys more time for things like developing that licenseable server product, making grid interoperability happen, consolidate onto more efficient server setups, etc. One way to reign in crazy growth is to raise prices. Just my speculation.

  • Prokofy Neva

    There is a widespread belief that all throughout the mainland, the Linden used, and are using, void sims. They aren’t. They are using full-prim sims. Go and look. They have always used them, with lots of prims left over. In fact, “void sims” i.e. actually in voids and not for sale are all over, and not the low-prim kind, but the full-prim kind. Perhaps they have or will start swapping those out, but I see many still full-prim, as they always have been.

    You’re not watching what the land barons are actually doing. Anshe Chung is frantically hustling 4-packs of OS sims back into islands at the grandfather tier of $195 and selling them, and people are buying them thinking they have that lower price for a year. She is selling them outright, so that people get that tier of $195, not pay her on top of that tier.

    Others are giving or selling their OS sims to those who can pay. And if you brush aside all this agida, you see that a bargain is to be had: instead of a $1000 sim, you get a $375 sim, and you pay $125, instead of $295. 3750 prims, to be sure, doesn’t seem like much, but for a house and furniture and a boat and dock, it’s plenty. Put in some sculpty rocks and a waterfall, you’re done. Far from losing a million, they are just making sure this product goes into the hands of those who can really pay it, and not default on 100 of them at once because they are clueless newb landlords unable to cope.

    Philip long dreamed of flat tier, tier by the square meter, without the jumps. That would kill off rentals quick, and maybe that’s the plan, it sure looks like it, though M’s letter seems to speak of rentals still in benign tones.

    Your theory of the aging out of servers as the single greatest driving factor of the land market is probably true, but you don’t explain why they suddenly could “pass on a big price break” all in one swoop, and then suddenly “have to pass on big usage costs” in another big swoop. Well, which is it?

  • Damanios

    On the subject of offering a service in the ‘sweet spot’ 10-30US range, people tend to forget that LL has been offering services in this range for years. It’s the staggered tier model, which was the basis for land sales in the first place.

    All LL decisions of the last year (except for the openspaces), basically point to reaffirming this sales model again. (Cleaning up adfarmers, hiring a public works team, creating themed environments, excessive marketing of areas like Nautilus etc.)

    The open spaces are basically ruining the tier sales model though. (Why go through the hassle of mainland/tier if you get a full configurable area for 75US.) So the only way to make the model viable again is to get rid of these. Which is exactly what LL did. The fact that areas in Nautilus sold on auction with a huge profit is probably one of the main reasons for LL to take this strategic decision.

    From a strategic business perspective it seems like a reasonable logical step. Consolidate to one sales model, fix the issues, and then roll out enough server space within this new sales model.

    Sadly, for LL, you can’t put back the genie in the bottle… I’m afraid it won’t turn out the way they expect it to turn out…

  • http://www.rezzable.com/metaverse rightasrain

    I am not sure you have the core issue right. From my discussions with various techie types there is no specific coding that would allow for a direct mapping of a region to a core. A CPU in fact does not get arithmetically more powerful as there are more cores–ie a quad core is not 2x as powerful as a dual core. So it is unlikely that a quad-core, dual cpu machine would effectively handle 8 regions under the same code that is running 2 regions on a dual-core, dual-CPU class 5.

    In fact it would seem odd to map a Region to a core when in fact the usage on Regions is not flat, but more bursty depending on load. For example–why leave a core processing unit idle when there is no contention for processing. So if everyone is at a party on part of the CPU, then why reserve processing for empty “habitations”.

    In fact Linden has envisioned a more dynamic virtualization for Regions, but as usual started it only and not carried it along. http://wiki.secondlife.com/wiki/AWG_Virtualization.

    I am pretty sure (but actually don’t know 100%) that the Region code hits the CPU and the core is an internal CPU tool that increases parallelism of processing and pushes more work through the CPU. The whole reason that Intel started the core technology was that it could not increase CPU speed cost-effectively and parallelism was next best thing.

    Also on the LL website they say that a Class 5 is a dual-core, dual-CPU and they run two regions = 1 region per CPU.

    I would also doubt that even LL is dumb enough to lease a server for more than 12 months at a time. I think running 10,000 and more servers already puts them in a good enough negotiating position. I would see it more likley that they are rotating the older machines into grid services rather than hosting even voids.

  • http://firstlife.isfullofcrap.com/ Crap Mariner

    “So, in essence, the Premium account, as a money-making service, is not a success, and no wonder that LL is planning to get rid of it altogether.”

    Based on?

    -ls/cm

  • http://www.art-photograph.com/ Scarp Godenot

    Gwyn: I really appreciate your detailed analysis here. Much to think about. And I think I do now understand why LL did what they did: They had been undermining their other products too dramatically. Simple as that. This had NOTHING to due with resource management and increased usage of the opensims at all. ( …and how could it, as the prims and usage numbers are a zero sum game?)

    On the mainland, openspace sims stopped production of new mainland sims cold. This happened due to the profit model of renting mainland at the same price became far less attractive than opensim rentals. (Mainland land prices as a consequence also dropped as low as 2.4 L per m2. A BIG reflection of no demand…)

    On the estates, openspace sims undermined the previous market model of renting space completely.

    So,Linden Lab’s decision here I think was NOT for the short run, but intended as a means for rescuing the long term model of their future sim sales.

    Let’s take Mainland for example here. A single buyer could buy 16 K of land and 3750 prims at $75 per month tier OR get 4 times the space and the same prims with water, no crappy neighbors and estate priveleges. Which one are you going to choose? No brainer here. Hence the end of Mainland new sim sales. (What we do see here by the way is the intense demand at that $75 price point, especially with an attractive product.)

    Now let’s look at Estate Land. Same prims per price and 4 times the area? Of course this totally undermines estate rental as well. People don’t feel crowded and it is a MUCH more desirable product. I’m sure LL took a hit on its increase in Estate sales.

    This is why they knew something had to be done. They didn’t like the future: the future in which SL would become mostly all opensims! The sheer speed of adoptions showed them this was coming.

    So, once the decision is made, LL’s problem now becomes:how do they “sell” this news to the thousands of people who are about to get screwed? That is what all the confusing nonsensical gobbledy gook about how they had been bad and were getting what they deserved. Total rationalization and excuses for what they KNEW would be a firestorm of protest. Less said the better they are thinking, weather the storm and all will blow over eventually. That is their strategy.

    It is simple to understand where they are coming from here when looked at this way I am thinking.

    The new ‘Homestead’ pricing at $125 makes people have to choose between “prims per dollar value” of the mainland, and the advantages of water and space of the Homesteads. OR They must now choose between the lower prims per dollar and more space of the ‘Homesteads’ versus the more prims per dollar and smaller space of the Estates.

    All in all I think LL is banking on the fact that most opensim SLers are not going to leave SL because the alternatives are not yet up to snuff.

    As an aside, I already have two friends who have bit the bullet and upgraded to full estate sims from openspace sims. I am sure they are not alone.

    Have a good day all!
    Scarp Godenot

  • http://www.metaverserepublic.org Ashcroft Burnham

    The whole Openspace mess is a catastrophic and overwhelmingly obvious planning error that could easily have been avoided, but is now likely to wreak immense and permanent damage on the entire economic infrastructure of SecondLife.

    Linden Lab have, in effect, drastically undercut themselves with what was supposed to have been a low-grade product used to fill in with a little greenery, but with what residents soon discovered was nearly as good as an ordinary sim. Months passed as the number of Openspace sims, almost all of which were used as substitutes for full-prim sims (and blindingly obviously so) burgeoned in unprecedented numbers. Vast numbers of residents spent huge amounts of time and effort setting themselves up to work with the new Opensim economy that Linden Lab had created.

    Eventually realising, well over half a year after the fact, that Openspace sims were not just being used for forests and sailing and wilderness, and that their margins had been drastically compromised by, in effect, selling access to their basic resources at a substantially lower margin than they had previously been doing, they decided to reverse the policy. The consequences of their original decision to unbundle the voids was overwhelmingly obvious at the time, and would have been even more so as the numbers of their sales increased beyond all expectations in the first month or two. If they had not unbundled them in the first place, or if they had realised the position within the first month or two after the unbundling, then they could quite readily have introduced a prospective price increase, whilst grandfathering the existing (relatively few, by October 2008 standards) Openspaces. Some people would have been unhappy, but massive upheaval would have been avoided.

    The position now is different. The margins might already have been too severely compromised to allow grandfathering. Profitability might well be at stake (there is no use in having increased revenue if the costs increase more than the revenue: in that case, it is more profitable to have less revenue and much less cost), even the financial viability of Linden Lab, especially since credit is now expensive to obtain. So, Linden Lab drastically change their pricing structures, and, in so doing, cause disruption on a scale never before known in SecondLife.

    Untold thousands of people had put immense amounts of time and effort over the course of many months into projects which depended for their viability on the Openspace pricing structure. Linden Lab’s abject failure to comprehend the blindingly obvious until far too late will destroy a large proportion of that work. A large number of people will leave never to return because of that destruction, even if they would have been perfectly happy to stay if they had never been lead to believe that their project was viable. People are entitled to expect stability, and expect that a project that appeared viable in the conditions in which it was started to remain viable for the long-term. People who are given to believe that a project is viable, and who, on that basis, invest immeasurable time and effort into it will very likely never trust again the institution that deliberately made the project impossible to complete after it was started, having originally organised itself such as to allow for such projects.

    As a result of that, a very considerable number of people are likely to leave SecondLife, many of whom would have remained in SecondLife quite happily if there had never been such a thing as an Openspace sim in the first place. As Gwyneth correctly identifies, it is inconceivable that the claim on the Linden Lab ‘blog that the issue is either unexpected or the fault of residents could have been made in good faith. Linden Lab have always been manifestly incompetent: they are now fundamentally dishonest. A great many people who have trusted them hitherto will, quite rightly, never do so again. Many people who have not yet had any contact with Linden Lab will never trust them to begin with.

    Ultimately, such disruptive and dishonest behaviour is self-destructive. A virtual world can only expect to be economically viable – for any purpose – if it is populated with large numbers of people willing to engage in whatever counts as economic activity in that world. To focus on corporates at the expense of individuals is suicidal: the corporates are only likely to have any long-term interest in SecondLife as a platform so long as there are a large number of people using it as individuals. If they want private meeting spaces, away from the main grid, then they will have no long-term need to have any involvement with Linden Lab: OpenSim will be viable (and far cheaper) for such an application long before it will be viable to rival SecondLife as an integrated, generalised, social virtual world. If Linden Lab’s business model is in the future going to be selling connexions to the main grid, then it must continue to ensure that the main grid is worth connecting to. Making drastic and sudden changes that have the effect of destroying the hard work of a large proportion of the economically most active population is not consistent with that aim.

  • T C

    Just a couple of tidbits to chew on, don’t know if they help or hurt understanding what LL is up to.

    LL buys servers outright, it doesn’t lease. http://blog.secondlife.com/2006/11/09/concierge-townhall-transcript-2/ – search that page for Martin Mounier.

    There are still full island regions on class 4, though mainland was all bumped up to 5 in the last year or so. Voids have worked pretty much like full islands, with the older ones on 4 and the later on 5. Some older hardware does get repurposed, but not really on Agni.

    Old servers may seem almost “free” on the surface, but they have lower rack density, higher power consumption and potentially higher maintenance needs, no it’s not necessarily a bargain to hold onto them in numbers.

  • Mark C

    *** I am pretty sure (but actually don’t know 100%) that the Region code hits the CPU and the core is an internal CPU tool that increases parallelism of processing and pushes more work through the CPU. ***

    You’re right — you don’t know. Try as I might, I can’t even make sense of your first sentence, in fact.

    Cores are CPU’s. They’re just CPU’s that are on the same chip. They may share part of their cache, but other than that there is no difference between multi-core and multi-cpu.

    *** From my discussions with various techie types there is no specific coding that would allow for a direct mapping of a region to a core. ***

    Then you aren’t talking to the right “tech types.”

Status Updates

 Second Life® View GwynethLlewelyn's profile on slideshare

My images

Smiling at homeWorking towards a national OpenSim gridTesting Beta 3.2.7View of Neufreistadt
Digg! Listed on BlogShares Site Meter

 

Gwyneth Llewelyn
Gwyneth Llewelyn is offline in Second Life.
Creative Commons License http://gwynethllewelyn.net is licensed by Gwyneth Llewelyn under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.