Tim Oren's Due Diligence

The Hardware In Question

I'm back home after surgery and resting comfortably in bed. Hurrah for Vicodin! (And a controlled surgical incision is a lot less painful than the trauma of shattered tib/fib.) Here's the hardware collection that came out of the leg:

If you count carefully, that's 26 screws. And if you look even more carefully, you'll notice that two of the longest ones are missing their self-threading tips, lost to metal fatigue. They'll be with me the rest of my days, but getting the rest of the metal shop out of my leg should certainly help the irritation problem. Maybe I need to get these cast in Lucite as a memento?

Uninstall? Y/N:

Those who've been following my scribblings for a while may recall that nearly two years ago I broke my right tibia and fibula rather severely. The repair and rehab from that accident left me with two metal plates in my lower leg. Their presence causes internal irritation after walking more than 3 or 4 miles, or being on my feet for several hours. The medical professionals as well as friends who've dealt with implants agree that such problems only get worse with age.

So tomorrow bright and early I'm going under the knife again to get the hardware removed. Expect a short hiatus here, possibly followed by some Vicodin-hazed posting. I'm assured that the back-to-normal-physical-activity time for this is about six weeks, largely waiting for the holes left in the bone by removed screws to fill in. Until then, they are a sort of 'break here' dotted line inviting more trouble. That's a far cry from the six months for partial rehab from last time (it was really about a year to regain full strength), and I've been able to schedule the down time for the worst Pacific coast weather period. This time I should be back on my feet ain time for the wildflower and hiking seasons.

Back To Code: Blog Community Watcher

Last week I wrote about getting back into coding, in part to pursue some pet projects. So here's one thing that I've been tinkering with off and on since late January. (Anyone waiting for a big technology revelation is going to be disappointed - in some ways this is quite retro.)

While there are some deeper agendas involved (once again, to be discussed later), the proximate cause for this project was my simultaneous frequent use and continued frustration with Gabe Rivera's Techmeme and Memeorandum blog discussion trackers. The good news is that they are a mostly ad-free way to keep up on top stories in their respective domains, high tech and politics. The bad news is that the blogs and topics followed are Gabe's choice, not mine. What I was really wanting was 'automatic peripheral vision' to pick up breaking stories from blog communities that I find interesting, but can't afford the time to follow in detail.

According to legend, Gabe knocked together the core of his platform in a year, so I figured I should be able to build something useful in less than that if I was willing to cut back the functionality where it didn't support my goal. Making this my first serious project in Perl would take me right into the heart of its library functionality, dealing with HTTP requests, XML crunching, Web services calls and piles of regular expressions and other text grinding. A decent sized journeyman's project.

I made several functionality compromises to keep the effort bounded. The first was not to try scraping arbitrary blog HTML, but instead to use only blogs that publish a feed of some sort, ATOM or one of the flavors of RSS. And that feed would need to be either full or long enough to contain a decent number of outbound links. The second compromise was a direct consequence: Not to worry about being exhaustive; this was to be my own alerting service not a generic utility such as Technorati. I would hand-build the blog community lists, rather than try to gather them automatically - I considered this degree of hand control to be a feature. Finally, I wouldn't attempt any sort of full text analysis, other than collecting HREF texts - this was strictly about citation counting until and unless I proved something else was needed.

The first bootstrap step was obvious, since I needed a blog community list, and the easiest way to get it was via OPML export from iGoogle, which I use as a reader. Making that choice as a config file format also let me borrow OPMLs from around the blogosphere as blind tests. (Thanks to Ole Eichhorn and others who unknowingly provided test data.) Getting the OPML import working got me into XML parsing. I started out using the CPAN XPATH library, but that eventually turned out to be a mistake: The parser involved was not very robust, and while it worked fine for OPML, it fell apart when exposed to malformed feeds (which are out there, and in the oddest places - slashdot!?). I eventually had to back it out and go with the full LibXML interface, with attendant gnashing of teeth since their interfaces are similar, but not identical.

Getting to the feed parsing milestone pushed me through fun things like XML name spaces and the specs for ATOM and the various RSS flavors. It also produced some interesting revelations about their rather variable semantics in actual use, around things like GUIDs, alternate links and even publication dates. The good news was that I found plenty of feeds meeting my requirements. Those bloggers with a feedburner source have obviously thought about the matter. There are also a large number of blogs that are publishing feeds without even having an RSS/ATOM bug on their home page - evidently as a result of default or trivial settings on their blogging platform. Those who think RSS is dead or dying haven't been trying to actually use it.

This got me to the point of actual output for the first time, which was simply a matter of collecting outbound links from the feeds, using a hash to build an inverted table, and sorting the referenced pages by link count. Followed by an interval of writing code to stomp out ad links and the more egregious self-linking behaviors. An elapsed-time based weighting function also turned out to be necessary to keep popular stories from staying on top the pile for a week.

Somewhere along the line some CSS was thrown in to make the output look a little less like the 90s. I also added a Technorati cosmos call for the top stories, in part as a means of finding further blogs to be evaluated for inclusion in a particular community list. It's a nice feature when the Technorati API service is running and stable, which is not always the case. (It's evidently not part of their core business model, whatever that may be.) I'm also generating multiple sections in my output lists, using different link weighting schemes to differentiate newly hot topics from those that have held interest over a longer time.

This was intended to be a Perl journeyman's project, and I have suffered accordingly. There was one complete pass through the code to rewrite in more Perlish idiom, which I consider penance for my C accent. And two rounds of refactoring, almost inevitable in an 'organically grown' project, and resulting in a style (at least for now) of rather coarse-grained cover classes and abstractions for feed components and analysis steps.

So how'd it come out? Here's a sample output so you can see for yourself. This was generated from my 'right & libertarian politics' community list. This list has been a useful test target since the posts and referenced stories turn over frequently, and it's an interesting counterpoint to memeorandum's spin.

I've found it takes 30 or more sampled blogs with varying posting frequencies to regularly produce interesting results. The bloggers also need to be part of a topically coherent community. Otherwise, the leading stories are simply generic top headlines that everyone tends to mention, regardless of their blogging focus. Different communities turn over their focal stories at different rates. Political bloggers are the most frenetic, milbloggers and scientists shift topics more slowly, and those with a finance or technology bent are somewhere in the middle. Make of that what you will.

What's next, other than continuing to tweak the community lists and output format? The project has met my initial goals. I let it run while I'm drinking the first cup of coffee and reading a couple of usual blogs, and by the time I'm done it'll have found the top topics in the communities I want to monitor. The peripheral vision gained is a win already. Getting into HTML scraping and full text analytics doesn't seem like it will offer much improvement for the effort. A little more automation seems in order, perhaps whatever cron-like thing OS X will do, along with auto-posting the results.

What this approach does miss is the action in blog comments, web-based bulletin boards, or in whatever real time channels (e.g., Twitter) are also used by a community of interest. If I wanted to turn this from 'peripheral vision' into an 'early warning system', it would likely be necessary to widen out the data sources accordingly to pick up interest closer to real time. Shorter messages tend to have few or no links, and following dialog can require discourse analysis on top of full text parsing. That's a long, deep rathole...

Uncharacteristically, I've opened comments on this post for questions, reactions and suggestions.

Back To Code

So if I'm only doing the VC thing part time, and don't currently have my fingers in a startup pie, why the relatively slow rate of posting around here?

I've gone back to writing code, and it's been sucking up a fair amount of time and bandwidth.

Why? One level of answer is that I missed being able to build my own tools. Since HyperCard finally died I've had no environment for knocking out quick hacks, unless you count Excel, and I don't. I also found my engineering feasibility and schedule BS detector becoming fuzzier and less useful given the accumulating years since being hands-on.

But Rich Miller didn't buy that motivation when I laid it on him, and you shouldn't either. The second level answer is that I'm playing around with a set of notions about social software, and have some specific ideas that I think worth implementing at prototype level. Exactly what is a matter for later posting. Being acutely aware of the overhead involved in building a team and raising funding, and wanting to remain agnostic about where these ideas were on the spectrum from hobby to serious business, I decided to just have a go at it myself.

I've been working in Perl, for a variety of reasons:

My last serious code (defined as something used by another human) was in a combination of C and various scripting languages. Perl has a familiar feel to someone used to C syntax, immediate gratification, and loosey-goosey typing. Object orientation in Perl is only skin deep, so I can decide how deeply I want to buy into it for a specific project.

CPAN has an awesome collection of libraries with mature functionality aimed right at my tasks of interest. Even the not-so-mature bits are useful cribs for a newbie. While the excitement may have moved on to other languages, the archives of accumulated Perl wisdom at places like Perl Monks saves a lot of frustration when hitting a real puzzler.

I deliberately punted the choice between the leading languages du jour, Ruby and Python. A combination of avoiding premature commitment, and deliberately creating the potential (necessity?) of shifting environments when going from batch-like personal prototypes to possible future web services.

Finally, I've got 40 years worth of computer language history and syntax buried in my hind brain, all the way back to assembler and Fortran II. Reading the camel book alone can't suppress all those years of bogus instincts. I need live code and a motivation to screw around with it to make a new language natural. Fortunately, one of my other avocations provided both: geocaching, of all things. One type of geocache is the 'puzzle' cache, where one has to solve a problem to find the actual coordinates. Given the Silicon Valley environment, a lot of the puzzles around here are rather nerdy - math problems, word puzzles, and cryptograms, many of them most efficiently solved with code. And the lingua franca of the code snips passed among would-be puzzle solvers is Perl. That gave me lots of little challenges and quick rewards while getting started, culminating in writing a hill-climbing solver for substitution ciphers.

Those toy programs barely touched the Perl libraries, and went just fine with a plaintext editor and the command line. That's not enough power for a real project, and misses the point of modern development environments anyway. So I've ended up working in an Eclipse environment, using the EPIC plug-in, all on my trusty MacBook Pro. I'm using git for source code control (it's good enough for perl, it's good enough for me), but mitigating its rather arcane command line interface with the GitX GUI for commits and other common tasks. The developers' suite that comes with OS X provided a nice stable Perl 5.8.8 configuration, which I'm sticking with for now. All in all, probably overkill for a one man project, but you never know.

None of this is meant to provoke a religious discussion about languages and environments (YMMV, and I've seen so many flame fests that they are just boring) but as a bit of background for upcoming posts about the actual topics of interest.

Is iGoogle Hosed?

I use iGoogle to collect a number of RSS feeds originated from blogs, in addition to a few of Google's MSM-originated feeds. The updating rate on the blog feeds has gotten slower and slower over the last few weeks. Right now, none of my non-MSM feeds have updated on the iGoogle pages since Thursday, two days ago. When I click through to the blogs, most of them have at least one post since then. I doubt all of their feeds happened to break at once.

What's going on, Google? I'll have to move my feeds elsewhere if you've gone unstable.

The Roving Eye: 2/12/08

LT G's 'Kaboom' blog is the best milblog writing to come out of Iraq since Neil "Red Six" Prakash rotated out.

Reverse engineering Mother Nature. Very cool, working backwards from current organisms to reconstruct a protein sequence from a common ancestor, and then determining some of the adaptations and environment of that early organism. It's hard to believe there are still evolution deniers out there, just as we're taking big strides in exposing and analyzing its action over time. Via Ole Eichhorn.

Japan's Aerospace Exploration Agency is building and testing prototype components for orbital powersat systems. Glad to see someone serious about this, we need to be trying out all the alt.energy approaches that look feasible, for both strategic and environmental reasons.

I got a good prognosis from my first post-op X-Ray yesterday. The price is six more weeks cooped up on a walker. Well worth it for a full recovery.

The Roving Eye: 2/9/08

Ole Eichhorn is blogging again, and has been for about a month. Welcome back!

My favorite new read of the holiday season was Richard Preston's "The Wild Trees"" (Fluffer to Kevin Kelly for the recommend.) It tells the story of men and women compelled to find the world's tallest tress - and climb them. Many of these monsters are in North Coast redwood country that I know fairly well. Not that I've got ambitions towards the climbing, but I'd love to walk those groves. [Shack happy already? — ed. Yes.] The author has a photo gallery section on his site that's well worth a look. Sort of puts my own old growth hunting activities in the shade, so to speak.

If you're anticipating major surgery, voluntary or otherwise, here's a word for you: Acidophilus. Seems the antibiotic cocktails used to prevent post-operative infections are also powerful enough to blow away your internal set of intestinal flora. That's the collection of bacteria that have become our symbiotic digesters in return for a usually congenial host environment. Once they're gone, the old GI tract - um - doesn't work so well. So the Acidophilus capsules with replacement flora, usually ignored by all but diet supplement fiends, are suddenly important in rebooting the system. You can find them at the nearest health food store - that would be Trader Joe's in our case. HT: our doc, who we love for his minimalist outlook on prescriptions.

Best YHOO post. Cruel, but fair, just like the Piranha brothers.

BlogWorld: A Mouse That Roars

Marc 'Armed Liberal' Danziger and Glenn Reynolds profess themselves pleased by the inaugural BlogWorld Expo, last week in Vegas. I had a more mixed reaction, perhaps based on my jaded appetites: I've been going to shows back to the old COMDEX in its heyday and the West Coast Computer Faires. And I've been to a fair number of 'first time' shows, due to a career split between developing and investing in new technologies - CD-ROMs, Atari STs, HyperCard, 'hypertext', WiFi wireless data, and on it goes. Since a first time show by definition addresses a market that is only partially defined, and due to its low cost attracts a fair number of out-and-out hucksters, it's often hard to extract a consistent theme, or to forecast the survival of the show and market.

BlogWorld certainly fits the pattern. In no particular order, there were exhibitors representing:

• Aggregators: directories, topical specialists, horizontal blog portals, branded content networks, audio and video


• Tool vendors, from basic text blogs to sound and video and DIY blog-to-book


• Far too many blog advertising networks and gimmicks


• Far too many 'feature level' technologies


• Platform providers: Yahoo, Windows Live, AOL


• Old media about new media: books, magazines, movies


• Consultancies


• Corporate and other PR presences

Many of these have the hallmarks of first time exhibitors: minimal signage, no clear message, no attempt to qualify the people that came past the booth. You can easily guess that half of them won't be back next time.

But that cacophony is normal at this stage. What matters is whether the underlying market is viable, and whether the show turns out to be a nexus bringing together potential partners and customers and vendors. That's a more relevant measure than the fact the show took up no more than one third of one of Vegas' exhibit halls, and the attendees at the sessions rattled around in the large meeting rooms. I had only a limited sample, but at least some attendees thought there was value in the connections made on the floor, in the hallways, and at the parties.

And yet, there's a difference between this show and many of the other first-timers. If you compare the still-minimalist 'large' BlogWorld presences to what you'd find at even a specialty technology show (e.g., RSA), you'd be ignoring the reality: This is an upstart sector that has discomfited the traditional media - stolen audience, discredited stories and brands, and credibly threatens to build completely new distribution networks. It has meaningfully affected everything from the value of major corporate brands to national military strategies and political campaigns. How many wannabe shows draw official representation from both the White House and Dept. of Defense communications staffs?

So, lesson one: This is a crowd that punches way above its weight. You can't judge it by the metrics of an established or even emergent technology market.

Perhaps a better comparison would be a Cable Show. There you get juxtapositions like the Playboy Channel (displaying its wares, shall we say), located right across from hard core tech vendors of video servers and optical fiber. BlogWorld was a small, low rent version of that melange of everything from content to infrastructure. You had your cheese cake, your celebrities, your live shows from the floor, as well as the techie bits. (Credit for first two images: Glenn Reynolds)

So here's my second lesson: I have seen the future of media, and it is low rent. Those big booths at the Cable Show and other mass media conclaves are made possible by high margins, which are in turn enabled by a stranglehold over distribution. What I saw in Vegas is competing effectively for part of that audience, distribution and hence margins. BlogWorld will never really look like a Cable Show, and the latter will never go away. But everything about the old media, from brands to margins and audience is at threat of being corroded by the participants in this little show.

Update: A Hollywood insider reaches similar conclusions, coming from a completely different direction. Via Mickey Kaus.

Off to Blog World

I'll be off at Blog World Expo in Vegas for the next two days. Reports as events warrant. Sometimes you just need to meet people f2f.

What's Going On Here?

So what's the deal, you might ask? Nothing but sporadic posts here for months on end, and then suddenly back to something like a regular writing schedule. Here's at least part of the story:

Over a year and a half ago I wrote that I had taken on a part time CEO role at a startup. That venture was in the area of database security, particularly centered on privacy and reidentification control of shared, but matchable databases. We were in stealth mode, and in spite of bold and hopeful promises to the contrary, working on it largely absorbed the creative energy that formerly produced bloggable content, while my actual output was being directed to an internal wiki. (Thanks, Ross!)

We put together some very modest angel funding to allow creation of a working prototype and recruited a capable security-oriented coder to do so. As that approached some definition of ready, I recruited a more business development oriented CEO to take my place, and shop the idea to both early adopters in the finanicial institutions sector, as well potential OEM partners in the security and analytics domains. I continued as CTO during that process.

Long story short: The returns are in, and what we have produced comes out as a feature, in multiple markets, rather than a free standing company. In spite of having written on the topic, sometimes you have to walk the road to find out the answer. Fortunately, we minimized the capital consumed as we did so, but at the end of the day the ability to extract revenue as a stand-alone doesn't measure up to the costs of sales given both the technology and the market. So rather than taking any more investors' money, we have stood down the company and are in the process of looking for a buyer for the technology.

So for the first time in a long time, I've got some free time and head space for blogging and other activities. There's a fair amount of 'scrap research' from the recent venture that will likely find its way here, suitably laundered as always. Pacifica Fund itself is nearing seven years old, and a number of our ventures are going through the exit process. Once these deals have closed, I may be able to write more about the trials and tribulations of the liquidity process without compromising my fiduciary role. Finally, there's a fair number of topics of interest that were laid aside over the last months due to lack of bandwidth, and some of those are already getting back into my reading and writing mix.

Often the best way to find out where you need to go next is to find out what's interesting enough to inspire the energy for researching and writing. So, no long term promises, but I will likely be more active here for at least the immediate future.