Philip Linden Announces New Open Source Model for the Second Life Client

Philip Linden headshotWhen M Linden took over the CEOship at Linden Lab, many of us have been dreading what would happen with Linden Lab’s open source projects. Specially when Zero Linden sort of put his own Office Hours on the backburner for four months or so. People worried about the worst, except for the Luddite gang, of course, which was happy to see open source efforts to become a lower priority.

Then after a few months we heard about the standardisation effort at the IETF level to get a single inter-grid protocol — MMOX. So at least we knew that they hadn’t abandoned it.

But then we went through the Kirsten’s Viewer drama and the follow-up with the OpenLife Viewer drama. And it was clear that all these viewers out there have different agendas, different purposes in mind, different ideas on where to go with LL’s source code… and also different interpretations on licensing 🙂

Granted, some of them — Nicholaz, Kirstens’ Viewer — have been hallmarks in pure, raw performance. A few, like Imprudence, focus on features. The “Restricted Life” series addresses intriguing ways of adding “plugins” — in-world HUDs communicating directly with the SL client. And quite a few address interoperability with other, non-LL grids (including LL’s own “Open Grid” viewer). The trouble with all those projects — or most of them anyway — is persistence in time. Although quite a lot of developers submit patches now and then, even though most of these patches take 6-24 months to be incorporated in LL’s own code, keeping up developing a SL viewer for months and months is not an easy task.

You might have seen how most SL viewers are often tied to a single developer  (Imprudence might be one of the exceptions). And what happens is that programmers, well, tire out, and give up supporting their code — it happens all the time. To do a long-lasting open source project, it takes stubbornness, a clear goal, and a team with “critical mass” of developers, that can keep it going even after many of the programmers have dropped out.

So what is the alternative? Currently, all those projects are highly individualistic, even if patches do, indeed, get submitted piecemeal on the Public JIRA and are popularly applied on almost all non-LL viewers.

When M Linden was quoted as allegedly saying that “Linden Lab needed to take better control over the viewer”, this sounded ominous — and some people dreaded the worst: LL’s stopping the continuous release of their open-sourced code. That would mean that all the independent developers would work on just what has been released so far — once GPL’ed, it remains GPL’ed — but future viewers would simply be closed-source. That would indeed be the worst-case scenario.

Instead, as if by miracle, Philip Linden just announced the complete opposite today. Not only is Linden Lab going to intensify their open source initiatives, but they’re doing something quite intriguing: opening up the co-development of the “official” SL viewer to any developer, without the time it takes to get a two-line patch approved.

At first sight, this doesn’t seem much of a difference — in fact, it seems pretty much what Phoenix Linden had in mind on his announcement of the open source client back in early 2007.

What happened in the mean time? Second Life in general and Linden Lab in particular complexified. An average “big” project at LL, according to Tateru Nino, takes about 14 months to complete — Mono, Havok 4, Windlight are typical examples. Probably Shadows, Flexisculpties, and Meshes will also take that long. But even small patches take months and months to get incorporated. This was always true in the past, but now that so many people with so many responsibilities use Second Life, Linden Lab felt (probably very correctly) that the time to “develop quickly, release often” is over. Moving from 1.21 to 1.22 took eight months and a dozen Release Candidates, with painfully long quality assurance tests between each release. The grid simulator software was even worse: it took 12 months to release a new version! Contrast that to the wild days of 2005, when you got a new release every two weeks — and most of them were a nightmare to deal with, but at least you knew that you would only suffer another 15 days for the next release…

These days, however, LL cannot afford to make their 17 million registered users to wait that long. So… change has to be moderate.

It gets even worse. Since pretty much 2007, the focus of development has been mostly on stability, and a bit on performance too — at least, on the grid side of things, less on the client side. Sure, we got a lot of nifty features on the client side as well: Windlight; voice; flexies; sculpties; (partial) HTML support. Still, compare that to, say, June 2004, when during a single month of development you suddenly got streaming audio, better vehicle support, custom gestures, user-generated animations, XML-RPC (to allow web servers to communicate with in-world objects), better estate tools, and a host of smaller features and bug fixes. You can see how huge the difference in development time was back then — and what grid-shattering changes it brought in a single month!

So Philip is planning to reintroduce that feel of “constant innovation” again. The details are not yet clear, but on the blog comments for the announcement, Rob Linden already tries to explain the procedure: Linden Lab will probably have a new branch of their code, the “Bleeding Edge” super-client, which will constantly incorporate all those gazillion of patches and innovative features submitted by independent developers. And on the other hand they will plod along with their “boring old client” as before. As soon as some of the “novelties” are deemed to be “safe” to incorporate on the “mainstream” client, they might be ported over — in due time. After several months. Maybe 🙂

So what is the big difference? After all, people have always been allowed to launch their own “bleeding edge” SL clients (at least since January 2007). Well, there are two major differences: with this change, LL becomes much more closer to a true open source shop, where one of their major products is actually co-developed with the independent developers and branded by LL (ie. they’ll give it their official “stamp of approval”). This naturally also means being able to overcome licensing problems (with the JPEG2000 library; or with the voice client; or even with the fonts!).

But more important than that, it means that Linden Lab will act as an official repository of the source code for all developers — just like any other open source project, really. Anyone will be able to ring over their patches and features and add to the common project — instantly — and a new release will be out during the night. A new official release!

Before you start trembling with fear at what that means when evil programmers start to contribute malicious code… remember, this will be the bleeding edge version, downloadable additionally to LL’s regular SL client. LL will surely warn everybody about the lack of support or guarantees on the “bleeding edge” version. Also — this will be peer-reviewed code, from a community of developers. Just like Wikipedia pages are subject to vandalism, since anyone can edit them, but the Wikipedia editors are always alert and revert the changes, LL’s “bleeding edge” version will also be subject to revision by a lot of people too — beyond LL’s own teams, of course. They’ll be able to change things quickly back. And, ultimately, it will be LL monitoring and supervising the procedure: e.g. instead of taking months to approve a harmless patch, they will allow any patch to be quickly implemented, but also quickly remove malicious or broken patches.

After all, big open source projects like Apache, MySQL, all the Linux/FreeBSD distributions, Mono, and even WebKit (also known as Safari under the Mac) work exactly that way — in some cases with big brands (like Novell or Apple) behind those projects, contributing with resources and validation. And these projects work. SL’s client is a tiny project compared to any of those, and it will not be too hard to keep it properly under control.

And obviously, if you are trembling in fear from the power of open source software, you’ll always be able to safely download that old, boring, painfully slow, bug-ridden, but official, stable version — where Linden developers approve each line of code after months and months of excruciatingly slow testing 🙂

Kudos to Jacek Antonelli on Plurk for the link!

About Gwyneth Llewelyn

I'm just a virtual girl in a virtual world...

  • So you’re confirming my point about opensource — it’s not so open or free.

    When people like Kirsten or whoever go off on their own thing, this is, oh, “drama”. Or violating some “license”. Licenses that are arcane and disputed and not so open.

    And…er…the entire thing needs a “strong leader” then to come and “whip everybody into shape” with “daily reports” and “commitment”.

    OK, like I always said: opensource=closed society.

    Evil programmers are evil, yes. There is little good that has come out of the viewer hitherto but griefing and exploits and drama and bullshit. Philip working on it means it merely has an insider company leadership that apparently all soi-disant “opensource” requires or it becomes unravelled. So, meh, not so open.

    All these projects, uh, work? But…how? They work by exploiting volunteer labour for big companies that then suck up all that exploited labour. Modern-day Marxism.

    So, interesting — does Jacek then use this new interest of Phili’s as a channel to bully and push and impose his Impudence viewer that he constantly bragged was deliberately not listening to “the people” (so much for that, uh, concept of open source being, er, open) becaues he was “progressive” in his views of how to monkey with the viewer? Or does Jacek now get well and truly screwed.

    Yes, hearts may be broken in opensource land tonight, whoever didn’t get on Philip’s friendship card list.

    Uh, no. I don’t “tremble in fear” of open source. But I criticize it as the self-parody that it is. Oh, wait. I don’t even have to anymore. It self-parodies, like it is doing now. Stay tuned, kids. Don’t try this at home!

  • I’m concerned about quality control. The SL Client is already notorious for being a leaky memory hog. I can sit there and pretty reliably watch it go from 250 MB RAM usage to 750 over an hour. If I’m using Winamp or in a space with 30+ avatars, I have to reboot every hour. I fail to see how amateur hour is going to fix that. I am not subscribing to be a beta tester.

  • @Prokofy, I personally never make the claim that “open source” means “absolutely free”, not even that “it always saves costs”. It all depends on the specific situation and a specific type of software. Most frequently, companies shrug off the costs of maintenance and support when using open source software, i.e. “who do I call when I have a problem?”. If they are able to afford to pay someone in-house to keep running the software for them, then sure, go ahead, use it. If not, outsourcing closed software might be a cheaper solution.

    Many of the most popular open software platforms developed by professional companies rely on the business model that most companies out there don’t have the know-how — or cannot afford it — to run their open source software, and sell basically “licenses” for doing the installation, maintenance, and customer support. Typical examples are Alfresco (data management, workflow), Zimbra (a MS Exchange clone), Red Hat (an operating system based on a Linux kernel), MySQL (database management), Mono (a clone of .NET), WebKit (an HTML rendering engine used by Apple and Google), or even Boonex’s Dolphin (a social site building tool) — but there are tens of thousands of others. None of these companies went broke because they released their software open source. In fact, they’re quite well-off, they have millions of happy customers, many of which are quite happy to pay for customer support and maintenance.

    The difference in all those applications is that there are always a large group of people that download the software for free, attempt to install it, validate the code, submit patches and new features, organise a forum or similar group-discussion tool, exchange tips, help new users, and spread the happy news (free promotion!). This means that an obscure, small company can quickly build on an established base of knowledgeable users, which will become their evangelists, and push their product to colleagues, partners, or even clients — for free. It’s that crowdsourcing work (not only in programming, but also marketing and customer support) that is so appealing for companies who are willing to release their source code.

    All those companies with their products also keep a degree of control over their software. Pardon the pun, but it’s not a “free for all”: these companies rely on selling maintenance and customer support agreements for their software and are naturally more than interested to point the software in the direction that their paying customers want. Getting free testing, bug reporting, patches, and contributions is just an added bonus — often a very important one, since mature code mostly means “heavilly reviewed code”, as in: “thousands of people have seen the code and get it to work properly”. Small software houses are not always able to get enough people to review their code. Even large ones (like, well, Sun, Apple, Novell, IBM…) benefit from an extended group of code reviewers, allowing them to focus their developers on new and different projects.

    So your ideological view that “open source is evil” is totally misguided. You’re thinking mostly of the small, one-person development efforts which are here one day, and disappear the next. There are actually very, very few projects that survive that way. Programmers are notorious for their short attention span, and too individualistic to truly share their expertise (specially for free). What they aim is to gather enough “fame” for their project and attract similarly-minded programmers, and — hopefully — keep it going, until, say, an enough number of them are able to create a company (or a foundation) to act as keepers of the code. It takes someone to do that work — and work it is, so it better be paid!

    Oh, sure, of course there are exceptions to the above examples. There are always exceptions — edge cases — which will be brought up whenever someone wants to make a point that some open source software survives even without “companies” behind them. These are notorious exceptions in any case. Long-lasting projects, however, are different 🙂

    @kanomi, just remember three things: first, Linden Lab will not “abandon” their main series of viewers. These will continue to remain available for download. Secondly, a whole generation of contributed patches that actually make the viewer work better and faster will be immediately applied to the “bleeding edge” viewer, meaning that at last we won’t have to wait for LL to apply a 6 or 12-month.old patch to their code to have it work better and faster. And thirdly, on most open source projects that have a company or organisation behind it, the resulting software is usually quite thoroughly reviewed — just because there are way more people doing so. Many forget that before there was a quality assurance team at Linden Lab, LL relied on volunteers to do the testing for them — there are still Wiki pages showing what those tests are (or were). These are quite thorough. When the independent developers start to notice that their testing and patch submission actually goes through, they might feel encouraged to test even more and submit more patches… and thus making the overall code be of even better quality. That’s what happened to almost all open source projects, and it’s particularly interesting to see how things like “clones” of existing software (e.g. Mono’s replica of .NET) actually become way more robust than the original software, in much less time — just because they get so many more people to tinker with the code, test it thoroughly, and make it better.

  • I understand what you’re saying Gwyn but a 3D virtual world client is just not the same thing as say even a web browser like Firefox; and Firefox has probably orders of magnitudes of volunteers more than SL clients will have!

    The hardware involved is infinitely more complex given all the possible combinations of CPU architecture, motherboards, 3D cards, and hard drive formats, to say nothing about driver versions, OS versions, RAID, SATA, etc. etc.

    I think bringing state of art graphic and network performance to the vast sea of possible PC configurations out there is already challenging enough to successful companies like Blizzard. It is already something Linden Lab can barely manage. I don’t think Open Source is the way to go. They should shoot for last year’s hardware spec and delivering a reliably stable experience to as many users as possible, the target audience isn’t bleeding edge Crysis players with water-cooled quad SLI rigs anyway.

  • Last year’s hardware specs definitely include a “vast sea of possible PC configurations” too 😉

    Joking besides, the point is that LL’s developer team is totally unable to test all possible configurations out there, since, as you so well put it, not even Blizzard can do that… nor Microsoft 🙂

    On the other hand, increasing the number of developers through sharing the open sourced code definitely allows more and more people to be able to test it for their configuration and submit patches that will work for them — and, incidentally, to everybody else who happens to have the same configuration.

    As a Mac user I’m part of the lucky class of computer owners where all possible configurations are well known in advance, of course, so your suggestion would benefit Mac users immediately. But, alas, SL ought to run well on non-Mac hardware too… 🙂