GFS, BigTable, MapReduce: where’s the competition?

This struck me today while reading about GFS and BigTable: what does Yahoo, Microsoft and Amazon use? I have no idea. I found it curious to observe that Google’s culture of develping their software in-house is well known, and most people have heard of some the three more famous examples listed on this post’s topic. There are papers about them and one can have a general idea of how they work.

Yahoo and Microsoft have some of the same needs as Google, since both do search have and mapping applications, which are very heavy (Google has 100+TB of sattelite image data), and Amazon has EC2, S3, and handles a lot of data itself. Do they use huge expensive servers running Oracle? Or do they also do in-house stuff, developing their own stack from optimized device drivers to ridiculously distributed applications? and if yes, where is that stuff?

The thought came when thinking about how much duplicate effort exists in the computing world. In the context of Google’s big distributed systems, the obvious duplications are the free (as in speech) implementations of their published solutions. But in the case of Google’s competitors, the duplication must’ve been not only in the implementation, but also on the whole design of the solutions.

One can imagine at least some effort saving if the alternative designs were known and free implementators could choose from them beforehand. Of course it’d be all a lot easier if they’d just open source it all right? ;)

P.S.: ok, so it seems at least Yahoo does publish some stuff. Just never heard of them before.

You know you’re a nerd when…

… you start hovering the mouse over Calvin & Hobbes strips to see if there’s a second punchline.

Should config or should I code?

Yeah, I saw the release, installed the Kubuntu packages (for KDE4, I mean), no effort at all. Logged out, switched session, logged in. Couldn’t stay for five minutes.

Everything got bigger. Like, it takes more screen space. I was surprised with this bunch of fat windows and widgets cluttering up my view. My 15″ laptop screen just seemed to shrink instantly to 10. Alt-F2 to fire up Firefox crashed on the first attempt.
Yeah, and the Wi-Fi didn’t work out of the box. Major party pooper. Best feature: it didn’t mess up my beautiful KDE3 installation. Went back to it immediately.

I don’t know about the rest of you people, but I’ve quit taking hours to laboriously scrub configuration files to the point I can see my face on them.

I can and usually do take some time, *after* stuff is “just working”, to customize one thing or another (keyboard shortcuts mainly). But my general philosophy on dealing with the whole installation/configuration issue has for some time now become package based. I install the package, try using it. If it doesn’t work, I’ll wait for the next version. No hurries.

That does have some disadvantages, no doubt. I’ve had my Vaio VGN-FZ150E laptop for almost a year now and mic/headphone jacks still don’t work, and neither do the built-in mic and camera. Or the Fn key. Tough. I’ve tracked the bug reports and tried it a couple times when they say “new package X fixes this”. But that was it. I refuse to go digging up patches, howtos and obscure config options unless it’s something I *really* can’t live without.

The first step in this decision was, of course, stopping to use Gentoo on my desktop.

The epiphany came when a friend told me “if you’re configuring, you’re not writing code.” Wow. That had never crossed my mind. I always felt pretty proud to be the power configurator. And that’s no problem when that’s what you wanna do for a job. Sysadmins love to configure stuff, and I’m glad there are people who do. But a bit before that conversation I already had pretty clear to me that I was probably going to be a programmer and, if not that, at least I knew I would *definitely* not be a sysadmin.

So that’s one distraction down. Now I only have to get rid of all the other hundred thousand useless X for which the sentence “If you’re doing X, you’re not coding” holds true (only the useless ones; there is much more to life than coding ;-).