Tim Oren's Due Diligence

Open Source Code Bases for Multilingual Blogging

This is a quick survey of the open source world to find platforms that might be suited for bringing up blog services or news hub sites in Middle Eastern languages. The best fit depends on whether the goal is a large, hosted service a la Blogger and Typepad, and a more stand-alone system that would support one to a few blogs, or be the core of a newshub operation (a la OhMyNews).

In the large, hosted category, the only real contender is LiveJournal. (Hat tip: Michael Edwards) This is open source code for the system driving the service of the same name. How many of the many scalability features descibed in this PDF slideshow are included is unknown, I haven't dug into the code base as yet. LiveJournal has an active developer base. It is multibyte character capable, and there are a number of teams engaged in translations, however none have yet completed and brought up a non-English version. There is a conference area for translation activities. The projects underway do not include Arabic, Farsi, or Urdu. The level of effort for translation, and the division between the server and (optional) authoring client is unknown.

For standalone, the leader is Mambo, which is really positioned as a full-up Content Management System, with blogs being one use. There is commercial version of Mambo. (And for Mac heads like myself, it's said to run well on OS X.) In this cautionary post on its capabilties, 'multilingual' appears to multiple languages at once, as the same poster cites his work on a German version for a commercial project. There also seems to be an Arabic version, Al Mambo, put up by a small Michigan firm that appears to have ties to Tunisia. (If any Arabic speakers are reading, a comment on this project would be appreciated.)

Finally deserving mention is WordPress. This appears to be a relatively new project, targeted at a standalone site. It lacks the feature sets of Mambo and the numerous scalability enhancements of LiveJournal. There are no translation efforts underway as yet.

Public Ontologies

(I've been taking a bit of time off from blogging over the holiday, spending some of it on catching up on book reading. A few reviews may follow. But there's also a little research via the net...)

Following some comments that public ontologies (hierarchies) might be good way of organizing blog posts into domains that could then be more reliably addressed by machine translation techniques, I took a short look at a few existing sites and projects that might fit. My expecation were very modest, and they were met.

Why such low expectations? Hierarchical ontologies for classifiying blogs or posts are a close morph of something called 'controlled term (or vocabulary) indexing' that's been around for a long time. It has known shortcomings. One of them is consistency - different people (or even the same person at different times) don't categorize things the same way. They are labor intensive, creating a serious cost burden in a commercial enterprise, and a public goods problem when done collectively. They are in no way 'neutral' - an ontology by nature is a theory of what is important, and what is related to what - which may pose problems of its own.

As I said, my expectations were met. One of the most mentioned of related projects was del.icio.us an open blog bookmarking and reading site, including a self-administered categorization scheme. I see a fair number of referrals to this blog from del.icio.us, particularly to technology or blogosphere related posts. The theory about its users and categories that I had formed can be confirmed by looking at the categories. A site that has OSX, BITTORRENT, and FLASH at its top level isn't going for global coverage, shall we say, though it's undoubtedly useful.

Wixonomy was mentioned at the Berkman conference, but the site is barely there. It does seem to aspire to global coverage, but barely has any. I'll watch this one with interest, but I'm not holding my breath. If you can get up a good blog fight over the name of a geographical feature, trying to converge a whole taxonomy seems an exercise in futility. I don't feel any better about global semantic systems than I did over a year ago.

More interesting from my point of view are attempts at emergent organization, including those not trying to create global organizations. Webjay is a site for publishing music playlists, which has a simple 'related' feature apparently based on overlaps in lists. Simpy is a hosted bookmark site, that also appears to use an overlap measure to find other people with similar tastes. Frassle is an intriguing site that apparently allows establishing ones own category system, which is then automatically related to those of other users. But, it apparently requires using its own embedded blogging platform, which has zero chance of global success. (See Geodog's notes on Frassle and some related projects.)

The emergent organization / tastemates projects feel like they have some potential in their own right, as long as they can work with existing blogging systems and standards. I didn't find anything to convince me that they would do a better job at rough categorization of blog posts than rather well explored techniques such as clustering based on link patterns, combined with some amount of full text analysis. Yes, these techniques are inherently noisy and error prone, but I'm looking at them in the context of corpus driven linguistic techniques that are themselves designed to mitigate such noise.

Great Probability and Bayesian Inference Tutorial

From the basics of joint and conditional probabilities, all the way to PHP code, get it here. Awesome job.

Hat tip: Don Park

ITM in Palo Alto

My report of the visit of Omar, Mohammed, and Spirit of America to Silicon Valley is up at Winds of Change. See also Dan Gillmor's report, Silicon Valley Redneck, and Renee Blodgett.

For those interested in a blow-by-blow of the brothers' tour, Kesher Talk has an ITM and SoA linky fest. (And BTW, Judith, I'm Tim Oren, not Tom. Tom is my better looking and taller kid brother.)

The Company of Brave Men

Blackfive reports the day, and the evening, more eloquently than I can manage. I can attest to his conversation with Omar and Mohammed - I was standing right next to them at the time. The brothers come across as quietly determined, occasionally bemused by all the fuss. They know far better than us the worth and cost of what they are seeking. I was honored to be in that company.

The bar's stock of Guinness and various scotches took a pounding, but all survived intact. I can report (from the previous night) that the ITMers have met and overcome at least one Boston lobster. I have pix of that evening which will be passed on to the brothers to use as they wish. On the other hand, I do have one really good compromising shot of Marc "Reasonably Liberal" Danziger and Janice Abrahams of Spirit of America Arabic blog fame, which might get posted if my demands are not met soon. Mwahahaha!

Blogospheric Convergence

Today was one of those days when I really wished that I had the journalistic skills of a Jeff Jarvis, by which I mean the ability to listen, think and write at the same time. While yesterday's Berkman session was on the slow side, due to the academic panel and discussant style, it was capped off with the thrill of meeting the Iraq the Model brothers over dinner. Today's sessions were much more engaging, open forum style that really exploited all of the diversity present - just read Jarvis, I can't do it justice. I kicked in one extended riff on how to fit the blogosphere and the machine translation market together, which I've expanded over at Winds of Change. Now running off to go drinking with Armed Liberal, Blackfive, and the Spirit of America guys. Hung over field report to follow.

Headin' for the Plane

I'm off to the Harvard Berkman conference, where the action will include meetups with Marc 'Armed Liberal' Danziger, Joi Ito, Jeff Jarvis, and a bunch of the good Spirit of America folks. Good times and skullduggery. But the important question of the morning is: 101 or 280?

Détente Achieved!

The Wind of Change team in the Spirit of America blogger challenge, of which I'm a member, has joined forces with my original blogdaddy, Jeff Jarvis of Buzzmachine. A good excuse to join us of the Militant Middle is supporting citizens' media and democracy in Iraq. So pay no attention to those lizardoids, helmet headed military types, and Grain Belt sippin' Northern wankers, the link you want to click on is this one:

Update: The team name has been changed to 'Pajamahdeen', the bane of legacy media everywhere. Remember, what symbol do you want to represent your contributions to the future of Iraq? A bunch of bloggers who hide their heads in tincans, or the dreaded insurgents who have the MSM newsies in fear?

Citizens' Media Money Chase

While I was spending the weekend at holiday parties, there's been a really smart discussion on business models and investment strategies in citizens' media. Jeff Jarvis puts together a great summary, with his own comments. Or jump to picks by Steve Rubel, Tom Watson, and fellow VC Fred Wilson.

On the whole, I'm in Fred's camp (surprise!). VCs bet on the change, which means they have an affinity toward the most differentiating and innovative parts of a trend. From my point of view, the most differentiating characteristics of citizens' media are low barriers to entry, and precision. They are the two faces of the long tail phenomenon: Low entry barriers enable the creation of diverse content by diverse people, precision in access lets readers find matches to their tastes without unendurable overhead.

Low entry barriers on content are problematic to most venture investors; it puts the determination of winning and losing into a domain of marketing and audience taste that we are usually ill-equipped to judge. Unless we're going to do an entire fund around content, it's difficult to use the portfolio model of risk management that is the core of our business. (Caveat: This is a somewhat Silicon Valley centric view. The Valley has never done content well, and we usually know it. About once a decade there's enough new blood to get up a movement in that direction; shortly the market slaps us down and normality returns.)

Low entry barriers are also a characteristic of most of the software supporting citizens' media authoring and basic hosting. Part of this is due to its evolution on open standards and during the rise of open source. A portion is inherent in the need for simplicity: Hang enough bells and whistles on the blogging system, and it's no longer simple enough to be real citizens' media. Shifting to the services business model somewhat addresses the business model implications of low barrier, but it doesn't necessarily address the VC's need for an exit model in what will likely become a market share game.

So I'll take my stand on the precision side, which makes me lean towards the same sort of things as Fred: content discovery, advertising targeting, data exhaust analysis, and so on. The scale of the problem, and the multiplicity of needs and approaches guarantee an interesting landscape, in both business model and defensible technology. It's way too early for certainty on industry structure, value chain, revenue extraction and lots of other essentials to understanding how the game plays out. I have a couple of touchstones that I will share:

* Enhance the value of the long tail, and extend it further

* Every ad a wanted ad

A precison creating play that pushes in both of those directions is probably not wrong. As for the rest - have a smart team, get close to the market fast, listen to the music and dance like mad.

Update: Comments have been opened on this post.

Update 2: Ed Cone says VCs make him nervous (Boo!), but it's OK as long as I don't talk about 'monetizing eyeballs'. Um, no. If your level of appeal is to the undifferentiated eyeball, you're legacy media - I don't care what your bizcard says. Offer some value to the brain behind the eyes, that's another deal.

Joe Trippi and Michael Turk on the Internet and Campaign 2004

Later this week I will be attending the Harvard Internet and Society 2004 conference, this year focusing on 'Has the Web Changed Politics?' (We all know the answer, right?)

The kickoff event, on Thursday evening, will be a moderated discussion between Joe Trippi, campaign manager for Howard Dean, and Michael Turk, eCampaign director for Bush '04, at 6PM in Harvard's JFK Forum.

The seating there far exceeds the size of the conference, so the organizers are making it open and free to all. Due to wretched airline connections, I'll still be on my way into Logan that evening, so someone can warm my seat and blog the affair. I'll be doing some posting from the conference's regular sessions on Friday and Saturday, in my usual chew-and-digest mode.