With Google Lively out of the picture, and an uncertain future for “new” virtual worlds to be launched in a world gone insane with the perceived financial crisis, it’s natural to ask over and over again, if there is an alternative to the Second Life® virtual world platform, what it will be.
Since early 2008 I have been very skeptic about “alternatives” to Second Life. In my mind, and specially after talking to several gurus and enlightened residents like Pavig Lok, any “competitor” to Second Life will require those mandatory features to succeed:
- Collaboration and user-generated content. This mostly means being in a shared environment being able to work together. It goes way beyond simply being in the same place and chat together (either in voice or text), but being able to do things together, be it simply playing the same game, driving the same vehicle, using the same object, or, well, building the same building. And there can’t be a restriction on how you work together: it has to be both synchronously in real time (everybody seeing the changes being reflected immediately in the environment) or asynchronously (you might work on something, log off, and your friend comes in and finds things exactly as you left them and can continue working on it). Finally, user-generated content is not simply having an off-world Web site where you can buy clothes and furniture (like on IMVU or Moove): it means dealing with intellectual property rights; it means permissions; it means group ownership; it means an economy based on buying and selling digital content.
- Persistence. This is a much more subtle feature: the notion that you can rez a prim, and it will be still around after five years. You might take this for granted, since in SL this is the way things happen. Many platforms might have user-generated content but rely on peer-to-peer networks to show that content to other users: OpenCroquet is the best example of that model. Under those models, either you save your work “somewhere” or it is lost once computers get disconnected from the P2P network (sure, you can upload them again once you connect).
- Contiguity. Most virtual-worlds out there have a model where worlds are sharded (the same copy of the content is replicated among several servers; a set of users connects to one local copy of the content, and although they can talk to users on other shards, they’re not in the “same” virtual world at the same time. World of Warcraft is a typical example) or roomed. Under this model, each server just holds a part of the world — from a single room, to a single region, to a “game level”. When moving to another room/region/level, you are actually changing your environment and point of view, by connecting to a different virtual world. Granted, tricks like “portals” and the ability to chat across rooms/regions/levels will provide you with the illusion of contiguity. Second Life uses a single world with the “tiled” paradigm (the world is divided into many regions, all side-by-side, but it’s a single world — at least on the mainland you can walk from one point to the other without going through “portals” or “teleports”).
And of course they require a valid business model to be still around after a few decades. Some virtual worlds have all the above requirements, but they are simply venture capital burners until Google/AOL/Yahoo/Microsoft buys them. 3D content hosting works for Linden Lab, and one might imagine that there are other possible models (e.g. advertising or sponsorship), but we have seen few of that happening. Subscription payments to get access to user-created virtual worlds seems, however, to be a dead end — probably the reasoning behind it is that if users create the content anyway, why should they pay a third party to see it?
(Movie available on the next pages, do read on! 😉 — Gwyn)
Second Life’s closest competitor is Second Life itself
Virtual Worlds, it seems, come and go. Over the years, I’ve added to my skepticism a certain amount of cynicism as well. The Silicon Valley hip culture has brought Geekdom the ultimate playground: just implement your crazy idea, and sooner or later someone will pay you for that. A lot of money. Don’t worry about long-term goals: worry instead on applying your skills to develop an awesome product, and you’ll be fine. Sooner or later, it’ll work out. Forget business plans — start with something, develop it to the full extent of your abilities, and you’ll succeed. Trust us. We’re business angels and venture capitalists, and we know how it works: for every ten good ideas, no matter how crazy, one will succeed (and pay off for 8 worthless business ventures and one which is so-so).
Well, the cycle seems to be always the same, even after the dot-com era. Companies are still launching impossible-to-sell virtual worlds (not to mention Web 2.0 websites). The whole messiness of the way these “new products” are launched is appalling. They have no business models. They have no idea what they’re selling. Granted, they often have awesome technology, excellent designers, programming experts, and good evangelists. They have talented teams with experience.
But they have absolutely no idea if they have a good business model or not.
Why the stress on business and not on technology? Mostly because Second Life is not an “awesome technological breakthrough”, although it certainly has quite good ideas. The renderer is by far not the best renderer in the world. The user interface was already obsolete in 2001, years before launch. And as we soon will see, not even communication protocols and server implementation are great. Not when compared to other, more sophisticated solutions.
Linden Lab was actually very lucky. They started with the wrong business model, and, even worse, the wrong market. They tried to sell a subscription-based system to games developers, when clearly Second Life was not at the stage where games could be developed on it (and some claim it will never be). Even worse, they started with all the wrong assumptions. Looking at ActiveWorlds, where almost all content is user-created, they tried to push for a similar model — kick-starting with some content (Linden trees, Welcome areas, roads, bridges, and some decorative elements), they hoped that users would do the rest.
They struck gold. It just barely happened to work. But to make money out of it, Linden Lab had to switch markets (from targeting SL to game developers to digital content creators and now to businesses and education for quite different reasons). They did, however, make some bold decisions: listening to Lawrence Lessig and implementing intellectual property protection on user-submitted content (“the user owns the copyright”), allowing the licensing of that content to others through SL’s interface, is dramatic — not even Facebook allows that!
When you add all that up — a combination of technology; innovation; user-generated content where users retain their copyrights and can sell licenses to it; novel business models (3D content hosting!); and luck — you’ll see that launching a new virtual world is anything but a piece of cake!
As a matter of fact, right now, and unless someone is working in a deep, hidden cave on the Next Best Thing, there is really just a competitor to Second Life that has all its features and characteristics, and adds a few more. Yes, it’s Second Life itself — but not the variety that Linden Lab developed, but, of course, OpenSimulator.
The Dawn of The Open Source Metaverse
Let’s go back a few years — late 2005, in fact. About that time, Second Life was aggressively competing with There.com and IMVU. And Philip Linden made a surprising announcement: that they were working on releasing Second Life as open source software. The hopes were high, since it was clear that expanding the metaverse would require, at some point, a freely available technology that did basically what Second Life was already doing so well. Clearly, an “open source Second Life” would, at some point, be the way to go.
It was not until much later, in 2007 to be more precise, when things really started to become interesting. When Linden Lab released their client as open source code, developers could finally take a look at how things are done in terms of communications. You see, Second Life is not really the hardware, or the server software, or the client software: it’s the software architecture and the communication protocol.
Reverse-engineering the whole protocol was a daunting task, even with a friendly hand from Linden Lab employees, but libopenmetaverse (formerly known as libsecondlife) was born, with all the pitfalls it brought: namely, a very easy way to copy content. But this was the first step towards a far more interesting project. Not making a new SL client — as probably Linden Lab expected — but a new server.
This project started early 2007, and what is surprising is how quickly it progressed. The core team is actually small, as most of these open source projects tend to be. Linden Lab was very encouraging — who knows, probably they could come up with something that LL didn’t ever think before (and my, haven’t they…). Also, it was quite exciting to see a small group of enthusiastic hard-core programmers doing from scratch what Linden Lab was doing in the remote days of late 1999. Eight years later, the wheel was being reinvented from scratch, but it was far more than “just another wheel”. The open source metaverse was born.
One might argue that there are plenty of open source virtual worlds out there. Well, there certainly are. Many of them would probably be usable as the base for an open source metaverse, with enough tweaking. VRML/X3D is still a strong candidate. People still haven’t given up on OpenCroquet. There are plenty of semi-abandoned projects on all sorts of open source repositories around the ‘net.
But there is only a single technology that allows you to use the very same 3D viewer downloaded by 16 million people to connect to a virtual world. Or rather, several virtual worlds. Which can even be interconnected (more on that later).
So, how good is OpenSim?
With so many recent “bad news” presented to SL residents — the change of openspace sims into “homesteads” is just one of the latest, and there will very likely be more — it’s not surprising that many look for alternatives. MMOGs and MMORPGs can provide some relief for anyone too frustrated with LL’s recent attitudes — at least they’ll find friendly and tight-knitted communities there to give them some support, but also a lot of fun and entertainment. Soon, however, the hard decision has to be made: going back to Linden Lab’s fold, or seek an alternative?
Luckily for many, OpenSim is not the crippled and broken technology solution from “last year”. From little more than a cute toy, it became a workable platform. But how good is it really? Well, here’s an example. It starts with the worst-case scenario: slow logging in, multiple sim crossings with confused attachments, 8000+ prims being streamed across two sims (which is what happens on a four-sim-corner), and the SL viewer being thoroughly confused about where it is — at some point, the avatar was in one sim, the camera on another, but the grid thought my avatar was on a third sim!
After that, a longer sequence of OpenSim at its best. Really, you won’t notice it’s OpenSim, except for the lack of my Animation Overrider (I did manage to compile it, but I still haven’t done any animations for it!)
Convinced? Well, perhaps you are 🙂 Or perhaps the logging-in sequence freaked you out. If you’re looking about the user experience with OpenSim grids ran by third parties, you can read all about it on Prokofy Neva’s blog. He’s quite honest with what you should expect when entering OpenSim with the mindset of looking at it as a real alternative to Second Life — and from the perspective of someone that is going to use a third party’s services to connect to a different grid than LL’s, it’s only natural that comparisons are made.
If you’re looking for more information about the technology, do read on 🙂
OpenSim is a reverse-engineered application running on top of .NET/Mono. What this mostly means is that the developers of OpenSim don’t really know how LL’s servers work. All they know is what the SL client expects to receive in terms of communication. Thus, the SL client, to a degree, defines how an OpenSim-based grid ought to work.
This mostly means that some limitations that are built-in in the client will also show up on OpenSim. Typical examples: OpenSim also has prims; there is also a 25-group limit (OpenSim currently has little or no support for groups though); sims are 256×256 m in size and the map is tiled; avatars just have one type of skeleton… and so on.
On the other side of things, of course, some limitations are built-in on LL’s servers — and here is where OpenSim can shine in all its glory. There are no prim limits (default is… 45,000… but you can change that if you wish). There are no avatar limits. Uploading textures costs nothing. And so on — these are really arbitrary things set by LL which can be changed at will from OpenSim’s configuration files.
Then, of course, OpenSim’s technology is “unstuck” from LL’s. We don’t exactly now how LL’s servers really work, except that they use a lot of open source technology. What we know is that it lacks modularity. You remember the issues with Havok, and how it was soo hard to upgrade it? Or how Mono took a year and a half to be deployed? Or how LL’s “bug fixing” tends to create new bugs elsewhere?
Well, these are symptoms of what is generically called monolythical programming. While we cannot be sure of it, of course, at least it seems that at the very beginning of Second Life’s server development, we had a single block of code. Adding or changing it seems to be insanely difficult. Some of you might have noticed that Linden Lab is attempting to introduce the new 1.25 version of the server software, which allegedly has not many dramatic changes (they even call it “Featurettes Batch #2)… for over two months, with several deployment failures. These come apparently from bugs having been introduced by slightly changing the code here and there. Good code ought to introduce features without introducing new code. There are always problems, of course, but it seems that as time goes by, this gets harder and harder to do.
Contrast that with OpenSim’s modular approach. The early versions were quite pragmatic: get to the point where an avatar can log in to a stand-alone server without crashing immediately. Once that’s done, add extra features on external modules.
OpenSim’s all about modules. Physical engine? No problem! By default, you have none enabled. But you have an option of four different ones. Just choose the appropriate one depending on your hardware (Open Dynamics Engine gives the best result but is also the heaviest). Problems with detecting volumes? (important for knowing when your avatar is going under an arch or bumping other avatars) Well, choose simple meshes (just cubes) if you have little CPU available; get complex meshes to get fine-grained volume detection. And of course you can mix and match OpenSim versions and setup different modules on each sim (or at least each server) for all your grid — Linden Lab called the possibility of having different versions of the simulation software in the same grid the “Heterogeneous Grid” and it was a masterful effort taking several years and dozens of developers. OpenSim does that from Day One.
If you’re already drooling, there is more. Although LSL compatibility is around 85-90%, this means that a handful of LSL functions are still unimplemented. But to give you an idea on how fast the implementation of the full set is progressing, here’s an idea. I tried very early on to run Francis Chung’s free and open source Franimation Overrider, used by about a hundred thousands Animation Overrider packages, on OpenSim. The idea is that this is one of the most complex scripts I’ve got. A year ago, it didn’t compile at all, not even with serious tweaking. Half a year ago, omitting some functions, it did compile, but didn’t work. Nowadays, it works flawlessly — so in 8 months or so, OpenSim’s developers went from “crippled” in-world programming to “almost full functional” status. Not bad, considering that in the process they have added a lot of things that LL never ever dreamed about.
One of those “things” is the ability to directly call functionality from the server. You all know how the first-generation ‘bots worked to extract significant data like avatar metrics, land parcels, objects (for external search engines), and the like. LSL has poor support for all that. Things like sensors are painfully slow, consume oodles of resources, and don’t always get all the information. What those ‘bots usually did was to do several “passes” for each sim, and update the information on external databases every time. These days, most of those ‘bots use libSL, because it has access to all information and bypasses LSL totally — it’s fast, efficient, and consumes few sim resources.
OpenSim goes way further than that. It includes those function calls in LSL — they’re extensions to LSL (functions get prefixed with “oo” instead of “ll”), and you can call basically everything you wish to know about the sim you are — directly. This naturally means far more efficient LSL-based “metrics robots”. It is also “dangerous”, so the OpenSim owner can turn those extensions off if they wish, or limit the functions that are available.
Now LL has been doing the same for years, since they also have realised that providing these statistics without needing LSL scripts or libSL-based ‘bots relieves the load on their servers. Well… allegedly, a large part of the web developer team at LL is doing exactly that for over two years now (which eventually became LL’s my.secondlife.com project, status unknown). You have probably seen those nice in-world displays that show you your own profile picture when you come near them — they use LL’s interface to get that data. And possibly you also have seen SignportMarv Martin’s huge effort to extend LL’s own web services and provide semantic extensions on top of that.
OpenSim has all of that already implemented.
Then come the whole plethora of external applications. These are one of the areas where OpenSim shines. It seems so obvious that it’s completely impossible to understand how Linden Lab never did the same.
When you configure your OpenSim for the first time, you can select on the configuration file if you wish it to run on “standalone” mode (just your sim). But with some simple changes, you can run not only your own private simulator, but all other servers as well — user server, grid server, asset server, inventory server, and messaging server (collectively known as the “UGAIM” servers). So it’s the same code for all servers, all you need is to change some parameters on the configuration file. And you have several choices of what database to use. Die-hard MySQL fans will naturally use that as their database of choice (the same that Linden Lab uses, after all), but you have more, simpler options (granted, with less performance) if you don’t wish to use MySQL.
And, of course, you’re not limited to run one sim per server. You can run as many as you wish. One, four (like LL does), sixteen, a hundred, a googolplex… the choice is really yours, it all depends on how many your hardware is able to support.
But that is, obviously, not all. You can run those UGAIM servers on the same machine as your sims, or on different machines — one per UGAIM server, or one for all of them. Database storage can also be on the same machine or on a different one. This means that it’s easy to plan for your own personal mini-grid: start with everything on a single server, if you wish, then separate the servers as the grid grows and you require dedicated hardware for each external server. Your other servers on the grid can just run simulators (and no UGAIM servers) and point back to your UGAIM servers. Or if you wish to “move” your sim between grids, all you need to do is to change the UGAIM servers it points to (you’ll see below how this magic has been expanded to do quite neat intergrid connectivity).
But wait! There is more!
OpenSim developers are, for some reason, obsessed with Instant Messaging, something I never quite understood why, but they share that passion with Linden Lab — in fact, the Open Grid Protocol, after addressing teleporting (without content), is now going to discuss IMs (instead of what matters, i.e. content transfer via your inventory…). True to their love, messaging is a strong feature of OpenSim — not only you can have your own messaging server, but by activating a plugin, you can directly interface with IRC. That way, a sim gets a IRC channel and private instant messages go via IRC’s own private message handling. No more need for complex gateways like ChatBridge to connect across virtual world grids and the rest of the off-world Internet.
There are lots of similar goodies hidden as plug-ins for OpenSim. Remember how long it took to have Linden Lab select Vivox as a voice provider and integrate Vivox’s software in SL? Language Lab had already done that a year ago, but OpenSim goes even further. What about integrating SL-based voice with, say, cellular phones or your corporate telephone network? It’s easy — OpenSim includes gateways to a relatively stable open source technology called Asterisk which allows you to run your own telephony server and use it as a VoIP-to-PSTN gateway.
So what does this mean? Basically, you can be in-world in SL and call your mother on her mobile phone. Or if you wish to go wild, you can do a business teleconference where a third of your workers is in-world using voice chat, another third is at their desks talking on their phones, the last third is using Skype. And if you wish you can even stream that all to different parcels. The mind boggles — because all that on LL’s Second Life is just plain science fiction and not even on their list of “future projects”. OpenSim, however, adds support for all that.
OpenSim also shines on support for professional builders. One typical problem is that you might crash (or lose your Internet connection) and lose your work for some reason. Or, after adding a thousand prims to an existing 5,000-prim-build, you suddenly find out that you’ve rezzed the wrong building and have now to start from scratch. How useful it would be if you could track down versions of your content!… Well, OpenSim has built-in CVS support. Even though it’s on its early stages of development, it means a completely new level of content creation inside the virtual world, where you can add versioning to your assets, restore old copies, integrate content from different people and release just the newer, most up-to-date-version… all things that you have to painfully do manually in LL’s SL, and have multiple folders called “old copy” or “not-so-old copy” or “copy tweaked by MyFriend Avatar in August” or that kind of thing. Truly a pain. Good enough for amateur content creators, but pros need more than that.
Tired of the 8-year-old avatars? Do you wish to experiment with new skeletons and meshes?Tired of the lack of shadows and meshes? Well, you can use realXtend instead, which has far better graphics, much closer to the state-of-the-art (*waves* at Sony Home). OpenSim has now merged its code with realXtend — so what that means is that you can use the same software to serve content to two different virtual world technologies. This means that you can log in to the same sim with the same content with different (and currently totally incompatible) 3D clients — and each user can see and interact with the users from different platforms. Now that is integration as promised over a year ago by LL — but apparently they have done little to actually make it possible.
Interconnection and managing load
Linden Lab launched the Open Grid Public Beta with much advertising on my 4th rezday (talk about coincidences…), which was for many (including yours truly) the beginnings of a dream coming true. The whole OpenSim development could become quickly pointless if LL didn’t give any signs of allowing integration between OpenSim-based grids and LL’s own Second Life Grid®. It’s obvious that our friends from IBM have been steadily pushing for this kind of thing — allegedly, they run a mix of OpenSim and LL’s own servers behind a firewall and wish to fully integrate all of them.
The Open Grid Protocol is LL’s answer to a possible integration between OpenSim grids. At this stage, it’s beyond me to fully explain and understand how it works “under the hood“, you’ll have to rely on hard core programmers like Tao Takashi to fully grasp the technology behind it. The Public Beta, to an extent, has, however, been a slight disappointment for me, and let me briefly explain why.
First it’s important to understand that this is a closed beta, not an open beta. You can’t apply for it any more! Allegedly, one month or so of testing with 100 residents was “enough” to prove that it works at all. But the way it works… is quite limited.
First, Linden Lab has to manually add your avatar to the list of “gridnauts”, avatars specially enabled to be able to jump across OGP-compatible grids. This doesn’t take much time (a few hours), but LL is not able to do that for every avatar. So just a tiny group is allowed to use this amazing technology.
Also, LL hasn’t integrated their SL “main” grid with OGP, but just the Preview Grid. What this means is that you’re not really “teleporting from Second Life to OpenSim-based grids”, but just from one grid run by LL. Still, we know that all content on the Main Grid is available on the Preview Grid, and this first stage of OGP integration was only about authenticating avatars, so that’s enough — your avatar login will work as well on the Preview Grid as on the Main Grid, so, for all purposes it’s a valid test.
You also need a special version of the SL client (dubbed “Open Grid“) to manage the intergrid teleports. While LL definitely released several patches over few months to make that new viewer closely compatible with the standard viewer, it means yet another viewer around your applications folder. The differences are minimal, of course.
Intergrid teleporting is not “painless” but requires a few manual steps. Nothing very troublesome for a SL veteran, but still a bit complex for regular users. But the real disappointment comes when you suddenly realise that you’re just being provided remote authentication and nothing more. What this means is that your avatar on the remote OpenSim grid will have no inventory (and no way to add anything to an always-empty inventory) and be unable to change appearance. You’re just a “ghost” travelling across a foreign landscape. You can talk to other residents, of course, and interact with anything you see. You just can’t exchange inventory.
Granted, things have to be done a step at the time, but it is a bit frustrating to have waited a year and a half or so just to be a cloudie on a “foreign” grid… and no clues on when LL will continue with their work. Allegedly, IMs is the next step (as said before, LL and OpenSim developers are obsessive with IMs for some reason, when all they needed to focus on is on Jabber/XMPP and get that obsession out of their systems…). Inventory transfers? Well, perhaps in 2020, unless something changes (see below).
So that’s the state-of-the-art on LL’s side for now. If they had an automatic way of enabling “gridnauts”, this would at least be minimally interesting, since you could avoid all the trouble of registering to the myriad OpenSim-based grids out there, and just use your LL account as a single sign-on solution for all OpenSim-based grids and LL’s own. That would at least be worth the trouble!
Sadly, LL has no intention of automating the “gridnaut upgrade” in the near future. This just remains a proof-of-concept which was field-tested by a handful of residents and nothing more — for now.
Hypergrid — intergrid teleporting done right
Just six weeks after the Open Grid Public Beta was closed, the OpenSim developers, frustrated with the slow pace of LL’s intergrid development, as well as the unnecessary complexity of the whole Open Grid Protocol, officially launched Hypergrid. According to legend, from concept to implementation it just took three weeks (probably for a handful of developers). And yes, as that page so well explains it, it’s full teleport between any OpenSim-based grid.
It’s devilishly cleverly done. When teleporting to another OpenSim-based grid, your avatar retains the information of what asset/inventory server you use. The “foreign” grid just needs to ask your UGAIM servers about that information (and cache locally) and rez it. That’s all it takes! It’s mind-bogglingly simple, and I can very well believe that it really just took three weeks to code that, since almost everything else was in place — the ability for avatars to “know” the URL of their UGAIM servers is, of course, at the very core of OpenSim’s application classes (contrast that to LL’s solution, where the same information is hardcoded for one grid only, ie. their own — that’s why they can’t have separate UGAIM servers for their Dallas and San Francisco co-location facilities, and had to use VPNs for that, and expend an insane amount of money now on a fibre link between both facilities to get rid of the VPN troubles. All that could have been avoided with just three weeks of proper coding!). The team behind Hypergrid only needed to make a few changes here and there.
Even more fun is the notion of “virtual world hyperlinks”. Basically what you do is to add an “alias to a sim” (similar to a desktop alias, if you wish, or a symlink for you Unix gurus out there) on your own grid. That “sim alias” is actually on the foreign grid, but the Map will show it as if it belongs to “your” grid instead. So your users just need to teleport to that “sim alias” on the map, and without noticing much of a difference, you’ll suddenly be on a different grid. But… with your avatar’s appearance, attachments, and full inventory!
The concept is insanely cool, specially because it doesn’t require any effort. All avatars are automatically Hypergrid-enabled. All current versions of OpenSim support Hypergrid natively. Activating Hypergrid on your grid is just adding a flag and a configuration line and rebooting your OpenSim application (which takes, oh, perhaps a minute or two, depending on how many sims you run on that server and how many prims each sim has). That’s all! The system administrator can then interactively log in to the OpenSim console and start adding “sim aliases”, which the OpenSim team calls “virtual world hyperlinks”.
It’s so easy that even a child can do it. Oh, sure, there are glitches — only to be expected for something that was developed in three weeks! One serious glitch is that the SL client (this has nothing to do with either OpenSim or the Second Life protocol) has some problems teleporting to sims that are more than 4096 sims away. On LL’s main grid, this never happens, since no two sims are further apart than that. On OpenSim-based grids, you can use the whole gridspace, and people started to use it, so this unknown bug popped up (LL might fix it in a year or two, they’re aware of it, and it’s reported on the JIRA). Right now, what this means is that if you wish your own OpenSim-based grid to be both OGP-enabled and Hypergrid-enabled, you’ll have to spread a few empty sims in 4096-sim-intervals, and ask your users to jump between those, a step at a time. But that’s the only limitation!
Even permissions are properly handled. So, yes, you can bring your own content and rez it on the “foreign grids”, and it will still list you as the creator. Of course, remember always that the OpenSim manager will have access to their own databases and be able to copy your content, but, alas, there is nothing we can do about that. You’ll just be careful only to teleport to OpenSim grids you trust.
In any case, this is so good and so well done that I almost had a heart attack the first time I’ve tried it out 🙂 Sadly, the only OpenSim-based grid I managed to teleport to was an Italian one, and my lack of understanding of Italian didn’t allow me to talk to the residents there — and there weren’t many around anyway.
OpenSim’s Hypergrid gave me in 2008 a glimpse of what LL’s OGP will be in, oh, perhaps 2020 or 2030 or so.
Load balancing and dealing with performance
For ages, Morgaine Dinova has been complaining about the lack of proper use of resources in LL’s own grid. She argues very thoroughly that SL is wasting a huge amount of resources to the point that the tecnology is not scalable. The argument is actually quite easy to follow. The top number of simultaneous users in SL, by the end of 2008, is 70,000 or so, for around 30,000 sims. We know that except for OpenSpace sims, LL runs one sim per CPU, so this mans something like 3 avatars per CPU. But the reality is, of course, much worse. Thousands and thousands of regions are usually empty, while a few hundreds have all those residents spread unevenly among them. So very likely dozens of thousands of CPUs are always unused at all, while a few hundreds are frying their motherboards trying desperately to take all the load of the thousands of avatars on a handful of sims.
This happens because SL’s grid uses a tiled model for the map, where each sim has a fixed amount of land (256 x 256 m) to manage. You cannot predict in advance where the avatars are going to be, so most of the CPUs will be wasted as they just run the software for an empty sim. Although disk space might pretty much be “free” (such low-cost as to be irrelevant), CPU and bandwidth are not — and all those overpopulated sims are taking a serious hit, while the rest of the grid is totally idle.
Well, Morgaine concludes that in terms of architecture this is completely mindless, and from a business perspective, you’re wasting your money by running empty sims, and never providing enough CPU + bandwidth where it really matters — on the popular sims.
LL vaguely claimed that their idea of 3D content hosting was to provide “isolated” sims (with a fixed amount of resources, e.g. 65,536 m2 of land and 15,000 prims) that would be independently leased, “similar to web hosting”. In practice, however, web hosting doesn’t really work like that. On a shared host, what you actually rent is disk space and traffic. The host will load more or less instances of the web server depending on how much traffic you’re actually getting on your websites. This happens dynamically. Hosting providers give you next-to-infinite disk space to use, but charge you by traffic. The best example is, of course, Amazon’s services, which basically cost “nothing” if you just leave all the files on their server, but they start charging you for using CPU and bandwidth — the more you use, the more you pay. Thus, you can adequately “share” a single server with several users, and only give them the resources they need and bill you for them.
In Second Life, however, you pay the same for an empty sim (ie. no avatars, no prims rezzed, no scripts running) as for a fully-loaded ones (15,000 prims built and 100 avatars rezzed 24h/day).
Now, OpenSim, to be SL-compatible, uses the same model. In fact, it would be the only way to make it easily compatible with the SL viewer. Things like the size of the sim are hardcoded on the viewer. The best you can do — and this is quite easy under OpenSim — is to move your region sims around between servers so that you have fewer heavy-traffic sims on a single server, and leave the “empty”, mostly unused sims to be launched elsewhere as a group (not unlike what LL attempted to do with their openspaces product — and we all know how that didn’t work out).
So in Morgaine’s view, OpenSim’s architecture, since it’s so closely modelled to LL’s own, is also not scalable, and the opportunity should have been used to implement a better architecture.
Enter load balancing and region splitting. Developed independently by the Japanese company 3Di, Inc., this is cutting-edge backpatching on top of OpenSim that allows a far higher degree of scalability and handling of “overloaded” sims. Two strategies have been employed: first, a dynamic load balancer, that pretty much does what I’ve suggested above, but automatically. A set of applications monitors the current load of a sim (one of a set of several running on the same server), and if it reaches a certain threshold, pushes it to an “empty” server. The way this is done is high-tech magic — not unlike the magic employed to move a running script from one sim to another: you capture the status the sim is currently on, create a clone of that sim on another server, and unpack the captured status, telling all clients that were connected to the sim to use the new server address instead. Amazingly, this all works quite well — I guess that avatars on the sim will experience a short delay like a teleport and then suddenly watch the lag disappear. Once an event is over, the “empty” sim can then be pushed back to the server again.
What this means is that an OpenSim-based 3D content hosting provider will do is to try to gauge the usage of their grid in advance. We can expect a similar degree of usage as on LL’s own grid. So perhaps out of ten servers, one is overloaded. All you need to do is to figure out how many sims you’ve got, divide by ten, and buy that amount of high-performance servers to give your users a good experience. The remaining 9 servers can be on “low performance servers” — until they’re needed.
Region splitting is even more radical. Under this model, it’s assumed that a single server is so overloaded that it can’t handle the load any more, not even when it’s running just a single sim (and uses, say, 16 CPUs and all bandwidth to serve all those avatars). In that case, the 3Di software splits up the region and distributes copies of it among several servers. Each server just needs to update a part of the region, and communicate the changes to the other servers. Avatars are neatly split across all those servers too, and get updates from the server they’re connected to. What this means is that you can have thousands of avatars on the same region. Or perhaps millions, if you split the region enough times. In practice, however, the communications between the many “split” regions to give the overall picture back to each avatar will overload the network and hit a limit at some point. Nevertheless, it definitely means that 1000-avatar-events are possible without lag on OpenSim. And all this is done automatically and dynamically — this means that you don’t need to setup anything in advance for your residents, the automated software will handle it all for you if someone gives a party and invites a thousand friends, and will neatly clean it up after the event is over.
Contrast just these two amazing improvements with LL’s own. Second Life Grid will simply never manage that in the next decade or so…
So, well, lots of promising technological goodies inside OpenSim (oh, yes, there are more). But how well do these actually work in reality? Let’s find out…
Living on the bleeding edge
For the past two years I have been always patiently explaining to everybody who complaind about LL and SL and threatened to move over to OpenSim that OpenSim is not really “production ready” yet. What this mostly means is that you’ll be accessing a SL-compatible grid that is in its pre-alpha stages — pretty much at the stage where LL started to do their own first closed betas, in 2002 or so. That’s not for the faint of heart.
Most people have contact with OpenSim by logging in to one of many OpenSim-based grids. Although many of those residents do have a technical background and understand what’s going on, most have just “abandoned” SL in disgust and are looking for an alternative that looks as similar to SL as possible and is familiar enough to them.
They get quickly disappointed.
First, they have read extraordinary claims about OpenSim, the first being that “everything is free” (including land, of course), and that attracts them quickly. What is often not mentioned is that everything has a cost, and that you pay for what you get. Running a virtual world takes computer resources, but most importantly, it takes a team of system and network administrators to tweak the servers to make them run smoothly. It also requires a lot of bandwidth. The first OpenSim grids just ran from everybody’s home — enough for you to build and have some fun and invite a handful of friends, but not more. And, of course, when your Windows PC crashed, you’d dump all your friends out of your sim, and probably lose all content that way, with corrupted files on your disk.
Secondly, people confuse “many OpenSim grids” with particular grids. “I’ve moved to OpenSim”, they say, but what they have done is a registration with one OpenSim grid, probably one recommended by a friend or on a blog they read. And their judgement of OpenSim will often come from the experience they had with one grid — you can get a very thorough (and critical!) review of OpenLife done by Prokofy Neva (he actually additionally tested several different OpenSim-based grids; you can get the many reviews from his blog).
The issue here is that it’s “not obvious” for a new resident of an OpenSim grid to understand what’s all about. All they know is that they can connect to “a grid not run by LL” by adding a parameter on their SL client, and that’s all. They understand they’ll “start with a new avatar” — that’s all right, they’re used to start with alts from scratch, so this is nothing new. What they don’t understand is that LL’s infrastructure and operations team are really good at what they do and that SL’s software is production-grade, not “a very early alpha”.
They’ll be very happy to learn that whole sims with 45,000 prims will cost them as little as, say, US$50/month on a OpenSim-based grid. But what they don’t understand is that LL doesn’t charge what they do because LL is big, mean, and greedy — but because providing high-end servers and running them in a very complex environment is very expensive. I try patiently to explain that LL’s prices are not that high (although the setup fees are — more on that in a bit) for the service they actually provide.
Let’s do some maths. People usually shop around on the net for server hosting services and get quite cheap prices there. My company’s own mini-grid of eight sims runs from two low-end servers, one of them hosted by aplus.net, which just charges US$/70 or so for a low-end, single-core server. That’s cheap. If I take into account that I run 4 sims on that server, plus all UGAIM servers, and even a few monitoring tools, well that means that each sim costs about US$16 or so, if I’d be willing to sell land on an OpenSim-based grid.
But that’s obviously not the whole story! aplus.net is actually pretty reliable, but if you go to the top tier providers, like Rackspace or Verio, you’ll pay for the same service perhaps US$300/month. Why the huge price difference?
Well, reliability mostly. Rackspace or Verio give you guarantees on their performance. They have multiple redundant high-speed links to the Internet. They have the best engineers, system admins and network admins in the world looking over your servers. They almost give you 100% uptime, or your money back. These are the kinds of providers you use if you want to be absolutely sure that your server never goes down. Aplus.net is reliable enough for a small company with a specific project and purpose in mind — but I certainly wouldn’t be running “hundreds of avatars” and selling land services on OpenSim from a server that only costs US$70/month!
Aplus.net obviously also has far better hardware for higher prices. Something close to what LL uses costs above US$ 300/month, and that’s the kind of hardware that you’d be using for hosting just four sims, if you do the same as LL (ie. on sim per CPU). It’s cheap — if you don’t wish to provide rock-hard reliability to your customers. The same server at Verio or Rackspace, of course, will cost you four times as much — or pretty much what LL actually charges!
But there is a difference. LL is a solid company with ten years of existence and 300 employees. The ops team has been running the SL grid for years and years and deals with 16 million registered users every day. Granted, they’re not perfect, but they are used to provide 3D content hosting.
John Doe who runs his own OpenSim grid from his home, with a handful of friends, and low-end hosting providers, and perhaps a few months of experience in tweaking OpenSim… is a completely different story. So, to recap: you get what you pay for. For US$70/month for an OpenSim, well, you get a service that is far below what LL can provide.
(As a side note, setup charges by LL are terribly inflated, since they used to charge the time it took for a member of the ops team to install the software on a server and give it to you, which took a few days — or several weeks. These days, the whole process is totally automatic, so there is no point in charging any setup fees. I believe that in 2009 LL will probably drop those. By contrast, most OpenSim grid operators still do that work manually. It’s not hard but it’s not exactly “instantaneous” either, specially if the OpenSim grid operator has to lease more servers in a hurry!)
Also remember that most — not all — of those OpenSim grids out there are ran by tiny start-ups or even individuals with little or no business experience. There are obviously a few exceptions (notably, 3Di), and more and more will appear over the next few years, but you have to take into account that most of them just saw this as an opportunity to get some extra income and have “dream jobs” elsewhere (you know, the type where you work from 9-to-5 with an adequate salary and don’t have pretty much anything to do during the whole time 🙂 – giving you plenty of opportunity to run your own 3D content hosting business from your workplace that way). The good news is that many of those independent 3D content hosting providers have had previous experience running their own 2D (i.e. Web) hosting provider businesses in the past, so at least they’re definitely not clueless. But I imagine that most will never present their credentials if asked.
Things will certainly change in the next few years, but the important thing to remember is: OpenSim performance depends a lot on who’s managing the servers. While it’s pretty easy to download the software and run it on your own computer for some fun with your friends, providing a 3D content hosting business professionally is something entirely different.
No matter how good your hardware and networking infrastructure is, there is the other side of the coin: OpenSim software is at the early alpha stage, even though these days it has more features than LL’s own server software. What this means is that development is oriented more to adding new features — matching the feature set as closely as possible to LL’s own — and less on “fixing bugs”. That’s the only way they have managed to reach almost the stage where they’re quite close to LL’s own product.
For instance, groups don’t work (yet), although you can parcel out plots and set them for sale (buying requires a special viewer, since SL’s viewer will always try to retrieve your SL account’s data from LL’s user servers). So this means that group ownership will not work, which leaves out creating a rental business that looks like the ones in SL. Similarly, you cannot set an object to group, allowing people in the same group to collaborate in building it with you. You either give it full permissions or no permissions at all. This is good enough for selling items on OpenSim-based grids (interestingly enough, there is an “economy server plug-in”, where you can track down people’s L$ using an external server to log transactions, give people access to their balance from a website, etc. just like in SL), but it’s not easy to, say, buy textures from someone and use them on your own content to be sold later.
This is the sort of thing that will annoy residents most: finding out that most of the things actually work, but not all. A relatively recent version of OpenSim did not allow directly creating clothing items on Appearance mode, or creating new scripts inside a prim, or edit notecards inside a prim. You had to create the clothing item (or body part) or the script/notecard in your Inventory first, and drag it to your avatar (or a prim) next. Then it worked! (even more recent OpenSim versions allegedly allow the same behaviour as SL). Attaching items to your avatar (as well as HUDs), even scripted ones, works quite well. But until recently, to save the position, you had to adjust the item and not forget to detach it, and re-attach it again to double-check. If you just logged out, OpenSim would not save the last position. Also, high-prim attachments tended to be very hard to correctly position (again, this has improved a lot in the latest versions).
Depending on the server and the bandwidth used, you’ll find terraforming and building an immensely pleasing experience (ie. terraforming might work tens of times faster than in SL, because it uses a different way to store the terraforming data, which apparently is far more efficient than LL’s own) or an absolute nightmare (if your server is underpowered and has little bandwidth, the reverse would be true: laggy terraforming, which is almost impossible to do correctly, and you’d have to rely to an upload of a RAW file with the terrain). With very large buildings that are totally unlinked, you’ll very likely see some prims missing now and then — not unlike building in SL in 2004/5, when the same problem existed. OpenSim’s latest version has a more efficient way to push data about a linked object to your viewer, so the trick is to properly link everything.
Since you can use your own physics engine (out of a choice of several) and different mesh calculations for checking volume boundaries (important for the client to know if your avatar is “bumping” into walls or other objects), depending on the combination, you might get almost the same “movement feeling” as on LL’s grid (namely, if you go with ODE and full meshes). But on a slow server you might use a different combination (simple physics, no mesh calculations but just boundary cubes), and things will feel quite different — and you might not even be able to pass under arches or through doors made as holes in a prim. Again, this will depend on what performance your server has, and what options you’ve selected.
Vehicles don’t work yet — Cory Linden had implemented “vehicle engines” on Second Life Grid which are quite complex — but they will work, sooner or later. The rest pretty much works as advertised, although you can expect some scripting limitations, or some things to be slightly differently implemented and not give the exact results, specially on complex scripts. The good thing is that you can move with scripted objects across sims, just like in SL, and with a small setting, they’ll even cross sim boundaries and keep running. Since people can compile scripts to do malicious things (specially when using the “os” function set that can ‘talk’ directly to the sim and even to the server beneath it…), OpenSim admins can disable that feature for security reasons (you’ll actually see a lot of weird options to deal with security — thus dispelling the myth that OpenSim developers never worry much about security. It’s not true. Some OpenSim grids were already victims of griefer attacks, and have now far more powerful tools to deal with that than LL has on the Estate Tools).
The bad news is that sim crossing is way harder than LL’s (see again the start of the video above… running around sim corners with high-prim attachments is definitely a bad idea), and, again, feels like SL in 2003 or 2004, when LL had far slower servers and little bandwidth.
There is an incredible exception, where OpenSim outshines SL in performance. If you’re in SL and let everything painfully rez, have downloaded all your items in the inventory, and stand still, doing nothing, in an area with a high density of textures, you’ll see from the Statistics Bar that you’re always consuming some bandwidth. Depending on SL’s version and the time of the day tis can go as low as 10-12 Kbps, but is usually at 30-40 Kbps, sometimes a bit more. This is intriguing because technically “SL is doing nothing”.
By contrast, OpenSim takes up so little bandwidth that it feels strange. Yes, if you’ve loaded everything, the bandwidth will go to zero, as expected. Even when turning the camera around. If you move on a fully loaded sim, you’ll see 1 or 2 Kbps of bandwidth (because some positioning data about your avatar has to be sent and acknowledged), but that’s all.
So why does SL always consume a bit of bandwidth even when doing nothing?
There is some speculation about it. First, remember that the Second Life Protocol was reverse engineered (even if with some help from LL’s employees). Some messages were not understood, and dropped from the implementation. So it seems that LL’s simulator software are a bit more “chatty” with messages that we don’t really know what they’re for. Tateru Nino and others believe that every second or so your SL client is checking all those calling cards on your inventory to see, for each one of them, if your friends are online, looping over and over again. This, of course, is an insane amount of resources being consumed (OpenSim definitely does not work that way!) — both CPU (to do constant polling) and bandwidth (to constantly get presence data from your friends). Knowing LL’s obsession about doing Instant Messaging “their own way” and reinventing the wheel, this somehow rings true, even if it’s one of the worst possible ways of implementing things… even if it tends to guarantee that you have always an updated list of online friends.
I also think that the other reason has to do with metrics. LL’s viewer is constantly sending out metrics and information about the status of your SL client. That’s how LL can look at the statistics and claim that, overall, the SL client is improving FPS for all computers and crashing far less. Remember that the reports by angry people complaining about the lack of performance are just a vocal minority — granted, a large minority, and a very vocal one, but a minority nevertheless. LL needs to have access to the real number of people with problems, or without them, which will never comment on how good (or bad) SL runs for them.
Or perhaps there are still legacy communications going on, that are implemented differently, and these have been dropped by the OpenSim team. The end result is that OpenSim consumes orders of magnitude less bandwidth than SL. And, as a side bonus, not having to decode all those messages will free up your CPU to push more data to the graphics card and worry about the viewer and the UI, and less about the communication. In my case, with the same settings, I get twice the FPS, on sims with an equivalent amount of prims and textures. That, of course, for servers that are 16 times slower than LL’s own (my guesstimate), which means that often there is not enough CPU/bandwidth to keep those pipes flowing, and my OpenSim regions need to retransmit a lot of data (which always causes some lag; I can spend up to a minute “floating in the air” when jumping to a new region, since my poor overloaded servers need to push all that data back to my SL client).
The experience will always be subjective, and since there are so many differences in hardware, bandwidth, and geographical location, I’m quite sure that everybody will feel OpenSim to be different than LL’s grid. For our internal testing, we used two servers with 4 regions each. One is physically co-located between LL’s Dallas and San Francisco grids, it is low-performance with just 512 MB of RAM, but unlimited bandwidth. The other server is in Portugal (which means ultra-low latency for me, although ping times will always surprise Americans connecting to it), has 2 GB of RAM, but quite limited bandwidth (just 1o Mbps). There are a lot of differences between them! OpenSim, as a software, will eat up as much RAM as it can, but will not be a dramatic hit in terms of CPU. However, bandwidth is required in an infinite supply, much more than I actually expected. I tend to tell people to forget to run OpenSim from your homes, since the best types of connections will give you 1 or at most 2 Mbps upstreaming bandwidth. That’s far less than you need — you can forget anything less than 10 Mbps to have at least a few avatars happily around. Unlimited bandwidth is, however, the way to go. If you’re operating your servers on a budget, aim for more cores or processors, as much RAM you can afford to buy, get unlimited bandwidth, and worry less about high clock rates for the CPUs.
The amazing thing about OpenSim is not how many limitations it has, or how awkward some things are, or how frustrating it is not being able to have some as-yet-unimplemented features. No, the amazing thing is that it actually works at all, and, even more important, in terms of a roadmap of features (and system architecture design), how amazing the new features are, which LL hasn’t even dreamed about. Hypergrid allows for distributed UGAIM servers allowing a contiguous grid with many different servers; region splitting and load balancing deal with immediate problems of performance on high-traffic areas and allow, theoretically, lag-free events with thousands of avatars. A modular approach based on plugins, external applications, and web services, allows people now to extend OpenSim’s functionality in several ways and avoid all the silly pitfalls of LL’s own monolithic approach with the long waiting intervals (months) between release dates. To give you an idea, if things in 2009 go as badly as they did with the upgrade to simulation server 1.25 — three months for deployment, and it’s not working yet, the next update will be attempted by the end of January — LL, at most, will be able to release three, perhaps four, versions of the simulation software in 2009. By contrast, you can count on daily releases of OpenSim, and most of them these days are about new features developed by the smallish core team — bugs can be fixed, over time, by the less experient OpenSim developers.
What this means to me is that by the end of 2009, Second Life will have a competitor. It’s called OpenSim — but not one OpenSim grid (ie. not one virtual world), but a complex and intricately interconnected Metaverse, made up of several independently run OpenSim grids, all neatly “hypergrided” between them. Users will just pick one place to log in and create their avatar and leave the inventory there, and happily jump around all over the Hypergrid. Once group functionality is implemented, and vehicles become fully functional, there will be little that OpenSim doesn’t do. Granted, the experience will vary wildly. I can imagine a few OpenSim operators to focus on different aspects. Some, for instance, might just get you super-high-performance UGAIM servers but no regions — for a fee, you’ll have your assets on the best possible hardware, and happily jump across OpenSim grids in search for a place to buy land. Some might focus on the low-end market, giving you cheap servers lightly priced. Others might try to push for high reliability grids, where the cost is pretty much what LL charges you, but allow you to use, say, 20 or 30,000 prims.
Hypergrid also brought the promise of intergrid economy — what this means is that you can use the currency you get in a grid on another one, so long as both providers agree between themselves to exchange currency information. This is quite similar to, say, travelling around the globe and knowing you can exchange your US$ or Euros or pounds sterling by any other currency, and that it happens automatically when you use your card on a local ATM.
So, barring any major catastrophes over the next months, at the end of the year, LL will be in a tight spot: they will be the lone player out of the game.
So, what would I do if I were LL? The answer seems pathetically simple: switch over to OpenSim as well, and drop the internal efforts of maintaining “their own product”, but instead focus on adding value on top of it. In the real world of computer software, this means choosing between the Microsoft way of doing things (“we have our own way of making wheels”) or the Apple way (“let’s get FreeBSD as our OS, paste nice graphics on it, and trademark it as ‘Mac OS X'”).
I suppose this would be gradual. Once OpenSim supports at least the same features as LL’s own simulator software, the first step would be to launch a hypergrided new continent running OpenSim — “inside” LL’s own map — and charge lower prices on that. This would be an option to solve the openspace/homestead dilemma. Give users what they like: cheap, underpowered sims — and keep the “high performance” servers running LL’s own software for high-end customers (e.g. large shops/malls, high-traffic event locations, and so on). With easy teleporting between both, this would just mean that LL would be competing with two lines of products — one for the high-end user, one for the low-end user — and if a lesson can be learned by the openspace/homestead drama, is that people are willing to pay less for lower quality products. Instead of losing all that market to the ever-growing (interconnected) OpenSim providers, be a part of that market. Learn the IBM lesson: you can still sell mainframes and low-end, Linux-based servers.
Best of all, launching a product that has no maintenance costs and no licensing fees, thus being able to be deployed much cheaper, does really not hurt the overall business model (again, that’s the hard lesson that IBM, Dell, HP, or Gateway learned when starting to offer servers with open source, license-free software). It just expands sales, and that is a good thing. It’s not as if LL requires more people to support OpenSim as well. No, in fact, a few of the core developers of OpenSim are LL employees — which means that they would be developing for it anyway, in the interest of LL itself, and they’re already commiting (human) resources to the project. Well, it’s payback time! Let LL launch their own OpenSim grid!
If they don’t, the future in 2010 will look dark for LL, because they have really created a monster that might destroy them. Even if LL continues to insist in “their” Open Grid Protocol, developed at such a slow pace that asset transport between grids will take a decade to be implemented, it’s quite clear that by 2010 all OpenSim-based grids will just use Hypergrid that works today. They might also allow OGP just to get some “visitors” from LL’s own grid, which will just notice how cleverly OpenSim-based grids are already working so well, and create a new account there — leaving LL’s own SL grid. LL tends to insist that the “slow pace of development” is mostly due to security and copyright issues — but I find that just a convenient excuse. First and foremost, as time goes by, it’s quite clear that LL is unable (or unwilling) to prevent content copy on their own grid. Why should they be so worried about content transfer to other grids? I mean, it’s not as if it’s not insanely easy to do it right now, in spite of all LL’s “fears”! On the other hand, the Hypergrid protocol fully addresses ownership and permissions. What it doesn’t do is any “magic” to plug the analogue hole. So, for me, the question is really moot. LL’s focus ought to be how to be part of the interconnected metaverse running OpenSim, and how to be part of it now, way before they figure out that, by 2010, the OpenSim Metaverse has far outgrown LL’s own landmass — and has the same features, the same look and feel, the same people, the same things.
Competing with yourself is always the hardest thing to do. So, no, I don’t think there will be any other “competitor” to SL during 2009. Sony Home might go the same way of Google Lively or not (I think it will), and there are new companies popping up all the time with their own idea of what user-generated content ought to look like. However, that’s really just replacing one company-controlled virtual world with another company — and in my experience, none of those companies will be so “nice” as Linden Lab.
The future is interconnection, and LL has been quite good at promoting that idea. They have always envisioned a metaverse of several interconnected grids, with LL at the core. However, it seems that they have underestimated what this actually means: the interconnected grids using LL’s protocols and looking just like SL are already there and have 30 or 40 thousand users. They will grow to the point where LL is running just one of the many SL-compatible grids, and the only one that is not interconnected, which is absolutely ironic.
I estimate that LL has one or two years to catch up with what they wish to be in the future; after 2010, it’ll be too late. I would urge them to start launching their own OpenSim grid now, abandon OGP in favour of Hypergrid and interconnect their OpenSim grid with other OpenSim grids, and allow all SL-registered avatars to use LL’s UGAIM servers on any other grid interconnected via Hypergrid. And, starting in 2010, enhance OpenSim to replace all sims on their own grid…