I thought to repost this post I did on the forums, regarding the latest developments and announcements by Linden Lab. To be honest, this is not Linden Lab’s officially presented plan, rather the contrary. By pulling information from bits here and there, from half-told stories, from vague allusions to a distant future, and several other sources, mostly informal, one can get a rough, overall picture of where Linden Lab is heading with their Second Life platform.
Some people have seen my post on the forums on this issue, and asked me “who told you that?”. I feel it’s only fair to answer truthfully: nobody. Thus, I might be totally off-track. There might be some confusion between what Linden Lab is actually doing (or what they should be doing) and my own wishful thinking. I might have read their snippets of information incorrectly. And, more important than that, knowing how Linden Lab works for the past 22 months, might not be enough to have the required insight to interpret their actions correctly.
So, take all the following with the required pinch of salt. Caveat lector 🙂 I still hope you find it interesting.
(originally posted at: http://forums.secondlife.com/showthread.php?p=1045872#post1045872)
While I’m no futurist, and despite (some) claims to the contrary, I’m not Philip’s alt 🙂 — but still I’d like to add that I’m also a “naive optimist” and have been accused in the past of having been a visionary myself 😉
It’s fun to see that as SL grows, more and more people notice that the “problem” with LL and SL is not a technological one — but one related to people 🙂 I think that’s a good sign. It means that, hopefully, by recognising that the flaws are directly tied to people, the technological bits can be worked out easier.
LL worked rather well when they had two major advantages:
- direct ties to their customers. When there were only a few thousand of them around, they could get in touch with all of them directly (anyone in SL for over a year which hasn’t been locked behind a virtual door will quickly accumulate near to a thousand acquaintances)
- a slow growth, allowing them to think a bit before committing ideas to paper (or rather, to code).
The first advantage meant that they could work on three focuses:
- giving people the tools/features they needed
- fixing bugs they knew that annoyed most of the people
- developing their own “visionary” new concepts and trying it out in the field
For this sort of company, a flat hierarchy with highly talented individuals was more than adequate. Developers were interchangeable; I guess even Philip would have enough time to do his famous rocket launcher or music device (how is it called again?), while fine-tuning the streaming algorithm, and still have some time to do PowerPoint presentations 🙂
Then growth hit Linden Lab. It was not exponential at first, but gradual — still, for a long while, there were diminishing returns. The first thing that happened was losing a more direct communication with their users. At some time, just the number of new forum posts was so high that nobody at LL could read them all. In-world meetings tended to attract the very few that managed to enter the lagged sim; most didn’t care. The joys of “working together with LL” — in the sense of meeting them face-to-face on the grid and discussing things together — were slowly faded out as almost all Lindens started to handle an increased workload. Discussions and meetings started to be held in “corporatese” instead of “geekese”. These days, most Lindens seem to be working on 12-hour-days, even during the weekends, and forget about holidays. They still love to work there. But they simply can’t encompass the lot of things that are going on.
The voting feature thingy is a good example of something gone awry just because of the sheer number of people using it. I’m one of the many who have pushed the concept to be implemented; I was one of the many to cheer and applaud the decision to let residents help the prioritisation of ideas; I’m now one of the first to say that it simply doesn’t work. It’s an abandoned concept, impossible to follow, and the time required to track it down (namely by simply merging proposals…) is way too high, it would be a full-time job just for that, and even so, I suspect it wouldn’t be enough if we grow to a population of one million. Or ten millions.
On the other hand, I view the SL client as an advanced piece of technology… of 1999 🙂 As a proof-of-concept, it might have served its purposes. After so many years have elapsed, one thing has come out of the whole experience in developing it: SL is not the ultimate game-development tool that LL thought it would be. It’s being used for too many different things, and the codebase simply can’t handle them all. The result: a monolithic nightmare to mantain.
From snippets here and there, I’m aware of several incredibly interesting ideas that seem to have failed to be implemented:
- Havok 2/3/4. The port is “almost done”. It was always “almost done”. But the problem is, you can’t integrate it into the code without breaking everything.
- SpeedTree. Cool concept. Sadly, it can’t be integrated into the code.
- uBrowser (Gecko-in-SL). It was scheduled to be “a six-week-project” after someone developed video streaming inside SL — the concept that you could have a somewhat independent bit of code actually outputting pixels on top of a texture. Nothing new there (OpenGL handles that natively), but it hadn’t been attempted. So, six weeks should be enough for the job of putting HTML on prims. 14 months have elapsed. Only now will we get the 2D in-world HTML browser, which was “95% finished” 10 months ago or so.
- Mono. The whole implementation was done by just one top programmer, who presented his results last year. Sadly, in the mean time, the code grew, and now it can’t be integrated into the current code base — it needs a complete rewrite.
- The new 2.0 renderer — which doesn’t exist anymore. After it was pretty much finished, it would give rise to a completely new (and incompatible) SL. I assume that they devised several approaches to re-import all the data from SL 1.X into SL 2.0, but after seeing that it would break too many things, LL proceeded to break the 2.0 renderer in chunks and integrate it back, a piece at a time, in the current code base.
- New permission system, integrating Creative Commons licensing systems. A lot was written about that; LL promised that they would do a long discussion on those, until it would get implemented. And so they did. We’re still waiting, over a year later 🙂
I could go on and on, but what seems to be the pattern here is always the same. Developers are assigned to projects. They develop them rather quickly; amazing things are done in a few weeks or sometimes a few months, and they have a proof-of-concept operational to show the rest of the staff. Everybody applauds and approves the change. Then they start to integrate it with the current code — and it all breaks apart. So, all the effort done by these very talented guys get freezed. I imagine it might be terribly frustrating for them! (I always imagine Xenon Linden behind his 23″ screen doing utterly lovely Poser 6 avatars, with hyper-reallistic texturing, animating them finger by finger, all with 12 different layers of textures getting baked together, and developers telling him that there would be no way to integrate that in the next ten years…)
To get things under control, LL hired not one, but at least two project managers. They have a whole team of “bug hunters” and a quality assurance team. They have fixed 6,000 bugs in 9 months (not bad!). Still, at the end of the day, what do we have? Lots and lots of unfixed bugs, major performance problems, a model that doesn’t scale too well, almost a year of bug-fixing and dealing with scalability issues, and no performance increases, nor new major features (except now for 1.10….). And on the other hand, hundreds of thousands of lines of code of absolutely fantastic concepts, all of them working at some time on specially-modified SL clients — but which won’t work on the current version.
Well, is there any easy way out of this? I think there is. 🙂
Once I wondered why LL had so many web developers, when their website was nothing spectacular — except for tying in into some grid-based database servers for some cool features, the website is the sort of thing a professional web programmer could set up in less than a week (after all, it’s built upon a Drupal install…). But they still keep a HUGE amount of web developers. What are they doing? 🙂 Nice maps and SLurl? 🙂
Actually, now we know: they’re slowly and painfully porting all the bits of the client-server communications to webservices.
One might argue if this is a wise move or not. The point is, I think they’re addressing their biggest hurdle right now: making sure that they develop the renderer separately from the user interface. Yes. At last 🙂
Once they manage that, two things will happen:
- Internally, they’ll be able to split their teams. The ones currently working on the interface — where adding a checkbox could be enough to crash the renderer beneath! — will be able to work on the interface without having to worry about the rest of the code. This will mean that interface changes will be deployed quickly; and that future versions of SL could be released independently: one team releasing UI changes, the other playing with the renderer. The two teams will be able to work independently and not worry if they are doing things correcty for the “other” team.
- This will open the door to easy interface mods, integration with third parties’ applications, and multiple clients. Yes, eventually, even the SL 2.0 client could be released separately 🙂 But the more interesting bit: they could finally get the whole programmer community of SL residents start to add all sorts of features to SL as “plugins” 🙂
Will this address the major issues of SL — lag and scalability? Well, perhaps. As an optimist, what I see is a major change in the future (or should I say ongoing…?) development model. As long as there is a clear, open, public API, people doing the server code, the renderer code, and the interface code will be able to work independently, making sure none of the teams creates a bottleneck. We have already seen some changes — it has become more common to see rolling upgrades on the servers that don’t require a client change. I expect this to be the norm in the future — with a good API, you would be able to deal with different versioning on the three bits that are collectively what we see as “the SL platform”. So imagine you’d be able to have a simple 3D renderer, but a very efficient one — and all chat would go to your favourite Jabber-based client 🙂 Inventory would show up as folders on your hard disk, as a remotely-mounted drive. That would be simply awesome — no more struggling with the client interface 🙂 For the die-hard lovers of the current interface, you’d get that option as well, on a monolithic client like we have right now.
Then the resident programmers would say: “ah, I don’t really like the way they have implemented the renderer, I can do much more — I’ll just grab a copy of the Quake engine or something, and get it to talk with the SL grid servers” 🙂
Now, we know for a fact that Philip is not afraid of going that route. What seems to be the underlying problem is the required security of all communications and data storage — SL relied upon the concept that no one would “look” at the data stream between server and client, so security was not a big concern. But this needs to change if they really wish to “open” the code more — at least, provide an open, public API to the server. What seems to be the major development effort at LL right now is going this route. When SL had perhaps a few hundred resident programmers, they dismissed (perhaps rightfully so) their willingness to help. As this number has increased to a few tens of thousands, it’s a source of willing development power that they wish to tap. It only makes sense!
So what I can imagine that will happen during 2006 is that all this work to clearly separate the three bits — UI, renderer, server software — will take most of the developing time of SL. Making sure that both client and server are not monolithic structures will be the next most important issue. When that is achieved, they’ll be able to say: “You don’t like this bug? You want this feature? Do it on your own” 🙂 It won’t be “totally open source” but the closest you could get at this stage; and I think it’ll be the major change in the way we look at the development effort at Linden Lab.
This doesn’t mean that this development will be “chaotic”. It will mean that there will be a rather large group of core developers working at Linden Lab, providing mostly hooks and interfaces — and encourage a host of programmers to add third-party tools, plug-ins, or totally different clients. On the server side, more and more should be done to tie-in into external applications. Philip has just announced yesterday that llHTTPRequest() has a 0.1 sec round-trip time (totally consistent with my own measurements made on the preview grid). This will already be a huge step in getting external applications being driven from inside the SL grid. The next step will probably to allow external images to be retrieved and displayed on your SL client (ActiveWorlds has had it for ages — so it’s time SL does it as well). This would then mean that LL wouldn’t have to worry about storage — if your textures load slowly, it’s the resident’s fault, not yours 🙂 As more and more things get integrated on external, third-party servers, LL’s grid computers will look more like “entry portals”, or proxy servers if you wish, to a host of applications that will simply be offloaded from the web.
Put into other words:
- Does your script lag? Run it on your own external server instead.
- LSL is too limited/buggy for you? Use your own favourite language on your own server.
- Is the client too slow for your computer? Develop your own light version!
- Too many bugs? Get a client with less bugs — or develop your own.
What remains to be changed under this model is the whole structure of the grid to achieve better scalability — that should remain for another thread.
So, my hopes are high, although I believe we’ll have to wait another year or so to see these changes actually happening. At the end of the day, this will bring a completely different relationship between Linden Lab and their core programmers, and the whole of the resident developer community. I truly look forward to it. I would also love to have Philip subscribe to these ideas publicly; for now, he just drops a snippet of information here and there, for us to listen, digest, and interpret…
