I Hate Computers - Log entry #762

Here are some helpful tips, if you ever find yourself using 'computers', especially the Windows species.

• Never ever disable the "VgaSave" video adapter. Ever. This is the fallback software used by Windows to display things on the screen if no other video driver works. If this is disabled, Windows will start but your screen will show nothing but the finest shade of black. If you fail to follow this advice, be prepared to follow these instruction to re-enable vgasave service from the Windows recovery console. Not that it worked for me, but hey, good luck with that.



• If you perform a 'repair' installation while your screen shows nothing but the black darkness which has enveloped the heart of every poor Windows operating system developer, hoping for your video adapter driver to be repaired, do not (no not ever) turn the power off. Not even once, just for fun.





• If you happen to boot up your Windows computer and it fails to start due to 'Registry is corrupted' or some such, and you choose to re-install Windows rather than becoming a Tibetan monk (who would likely have fewer problems than a Windows user with a failing disk drive, even considering the Chinese government's approach to freedom), be prepared for a long stretch of file recovery and application re-installation. I recommend Fat Tire Amber Ale from the New Belgium Brewing Company.
  




• If you happen to have not followed these tips and you have re-installed Windows and you now have a default user account with a default green and deceptively bucolic grassy field, where before you had many files and folders and possibly (if you are reading this) insipid desktop wallpaper, here is an actual, useful tip : you can change the settings for this newly created user account to use the folders and settings of your previous user account. This won't solve all your problems, but really, do you think that's even possible? Even with advice from me (and I'm composed of nearly 100% pure awesome). Here is what you do :
  
  
  
  • use the regedit program (and if you don't know what that is, give up now) to view HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList
  
  
  • Look for a sub-entry that has a really long value with an entry of ProfileImagePath that points to the fairly empty and useless 'new user account' (e.g. "%SystemDrive%\Documents and Settings\Mike.NAUTILUS". Change the value to point to the location of the old and wondrous user account (e.g. "%SystemDrive%\Documents and Settings\Mike").
    
  
  
  • Go back to the ProfileList registry key, and update the "AllUsersProfile" and "DefaultUserProfile" settings - you'll probably want to continue to use the old folder that had settings for all the old applications you had previously installed and which are most likely useless, since you've re-installed Windows.
    
  
  
  • Log out, then log back in. Hopefully you now see your old desktop. In either case, remember - Fat Tire Amber Ale from the New Belgium Brewing Company.
    
  
  
  • To be honest, you could just copy the files from the old user account folder into the folder for the new user account. But I'm lazy. Just be aware that by pointing the new user account to the old user folder, there may be permission issues with accessing some of the folders or files. I haven't seen that happen but it seems a likely next failure.
    
  
  
  
  

Catalina - point-of-sale ad network

I wonder if they support cookies...

Armed with two years of purchase data for 80 million individual consumers, Catalina Marketing is this week launching a new in-store ad network called the Pointer Media Network.

The information comes from frequent shopper cards covering most of the nation's supermarket chains, thousands of drugstores and other retailers.

[...]

Pointer Media works like this: Catalina has installed color printers at the checkout counters of close to 50,000 stores around the country that are linked to the company's massive database of consumer purchases. When a shopper's order is rung up, the printer instantly creates a print ad on a receipt-size piece of paper based on the unique purchasing history of that shopper. The ad is handed to the shopper along with the receipt for the current purchase.

SpaceX reaches orbit

SpaceX became the first private company to launch a liquid fueled rocket into Earth orbit. Totally awesome! This was their fourth launch of the Falcon 1 configuration and was carrying a test payload. Their third launch two months ago didn't achieve orbit and actually was carrying paying payload - oops.

This is a huge step on the road to space.

SpaceX - Flight 3 of the Falcon 1

SpaceX is launching their Falcon 1 today from Kwajalein - about 1 hour to go before liftoff, if things go according to plan.

Here's a live webcast - http://www.spacex.com/webcast.php

WebHooks

This looks interesting - in a 'teach people how the Web really works' kind of way. WebHooks is a catch phrase for Web application development where notification are sent from theh source to the listener via HTTP POST, rather than the other way around via polling (which, as some have said, doesn't scale).

Somewhat related to my earlier post on how HTTP can be used as an alternate to XMPP/Jabber in a publish/subscribe scenario without too much problem or angst.

MySQL and LAST_INSERT_ID()

We've recently migrated our software servers from a dedicated server environment (BlueGecko - really good people and service) to Amazon's EC2 'virtual compute cloud' environment.

The new system has a very nice performance monitoring capability based on Ganglia that gives us visibility into the performance of service requests as well as details on more fine-grained functions that take place within each request. We can now see not only the number of requests or functions but also the average duration and the time it took for 50%, 95% or 99% of the requests to complete in a five minute interval. This percentile breakdown gives a quick feel for how 'spiky' performance is and how common outliers are.

So far, things have gone very well but there was one function of the system that seemed to suddenly have terrible performance. I was able to quickly see which area of the code was involved, which pointed me to some SQL statements.

As our system adds new anonymous profiles, the code retrieves the unique number assigned by MySQL using the special query that MySQL provides:

SELECT LAST_INSERT_ID() AS user_id FROM oo_anon_user

However, it turns out that this SQL is actually incorrect - the extra "FROM oo_anon_user" caused the database to return every single record from the table back to the client, on every request that created a new record. This of course took some time.

The correct syntax to retrieve an auto-increment field from a MySQL table is

SELECT LAST_INSERT_ID() AS user_id

Remember to omit any FROM clause.

It's much much much faster now.

REST and pub/sub

It's unfortunate that technologists continue to propagate serious mistakes like "[...] its also clear that REST and its inherent polling mechanism isn't the best way of building a user notification system [...]"

REST is about state transfer - and event notifications are also state transfer.
As for HTTP, it isn't only "polling" - anyone that has posted a blog entry knows that. The 'client' can 'post' updates to the 'server' - exactly the same as event notifications via XMPP. The great thing about XMPP is the federated multi-hop capability with 'trust' built-in. Just like email, only with everyone using settings for very low latency delivery.

There have been multiple publish/subscribe over HTTP mechanism (comet, mod_pubsub, KnowNow, etc) over the years.

Twhirl Adds Identi.ca

I don't really follow the social/friend feed streaming application space, but this post about Twhirl from
ReadWriteWeb had a quote that caught my eye (emphasis mine):

However, for the regular user, always on social networking doesn't have to be a source of stress - it just means that when you go online, socializing with others just becomes part of the overall experience of being on the internet.

MoonBoy

We've been doing GeoCaching for a number of years, enjoying learning about new parks and finding Geocaches on our trips. Earlier this year we created our own Geocache near our house and placed a Travel Bug in it - MoonBoy. His goal was to view a space shuttle launch and to someday travel on a Space Shuttle.

Amazingly, he's made his first goal and was an observer to this weekend's launch of STS. This is totally cool! Here are the photos on the GeoCache site

Oh, the irony of shallow WSDL

I'm looking into the API for the Hi5 social network and unfortunately found some WSDL. They also have a REST API, but it's documentation appears auto-generated from WSDL that nobody actually filled in. Somewhat useless.

Normally I wouldn't post about WSDL, but I couldn't pass up the irony of the WSDL documentation for the authentication service API having an HTML form describing how to authenticate the user. If that's not irony, I don't know what is.

Ultimate Twitter revenue model - chatbots??

From ReadWrite Web

"Essentially, this would entail Twitter parsing over the Tweets of a given user, as well as the Tweets of the users he/she is following. Common keywords, themes, and phrases are then pulled from this data and associated with that user. As a result, highly-targeted ads can be displayed based on the user's network of content ("web design", for example). These simple text ads would look very similar to regular Tweets, but would be clearly marked as "Sponsored Content"."

I think chatbots haven't work for a reason - people want to chat not shop.

Reading RWW and other pundit blogs that describe "how the future will work" reminds of reading Popular Science as a kid and gazing in wonder at the flying cars and transparent house soon to be built.

Web scale pubsub

Looks like people are now starting to talk about a de-centralized, Web native pubsub system. Now is the time, if applications are going to avoid being based on a single company's service. Too bad pubsub.com and KnowNow aren't driving this.

CouchDB

I'd heard the CouchDB name but but didn't realize that it's an HTTP-accessed, Erlang-implemented, schema-free, distributed datatabase - and it supports Javascript for defining views. Time to dive in!

From Apache CouchDB: The CouchDB Project:

"Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API. Among other features, it provides robust, incremental replication with bi-directional conflict detection and resolution, and is queryable and indexable using a table-oriented view engine with JavaScript acting as the default view definition language."

What programmers like

Quote of the day, found on the High Scalability blog while looking into Amazon's SimpleDB

"Programmers like problems they can solve with more programming."

Online services and availability

I've been working with the design and development of online services for quite some time and am very familiar with what 'high availability' means, but today I had the opportunity to learn first hand how the lack of reliability can directly impact people. This is a minor example, to be sure but as more 'software as a service' companies are built out more people will be affected in ways that they don't quite understand.

My daughter attends a Middle School that uses the online curriculum from Key Curriculum Press - I have no idea if the content is decent or not (but knowing the teachers at the school I'd bet it's good) and in order to access specific materials a login process has to take place and then access to some PDFs become available. This weekend, the login service was unavailable and so my daughter could not work on that part of her homework - very annoying.

The problem is likely something as simple as a server misconfiguration or a host being down (I can't imagine there's too much horsepower required for this sort of service), but as a 'customer' we are unable to do anything. There is no 24x7 tech support or online contact information. I could make a phone call, but that would only work between 8am and 5pm (PST), and I'm pretty sure kids do homework outside of those hours.

I've sent an email to their president and we'll see what kind of response that gets.

Update - I received a nice email from their president and she also sent a follow up email with more details. Very nice!

MyOpenID for Your Domain

This post from ReadWriteWeb about
MyOpenID for Your Domain is interesting. I've looked into OpenID several times in the past but it didn't have enough immediate value to get me to do anything about it. Now that more sites are 'supporting' OpenID and with the simple to use MyOpenID provider maybe I'll set something up. It would be interesting to see decentralized profiles and authentication, it seems a shame that everyone is getting locked into a few social networks for hosting and managing personal profiles and shared contact info.

Yaws and reverse proxy

It seems that there are a lot of situations where efficient connection handling in an HTTP reverse proxy is needed to help with scaling and keeping costs down, and my first guess was that Erlang would be the way to go.

So I just searched to see if there was a reverse proxy written in Erlang and sure enough Yaws has one.

Next, I need to find some folks that have actually used it and ask some questions. Especially for persistent connections (accepting a request but sending no response until there is data available, like in a message queue or pubsub scenario).

Of Liquidity, Competition and Platforms

This post by Bob Wyman "Of Liquidity, Competition and Platforms" reminds me why I subscribe to tech blogs - occasionally you get a deep thinker that clearly and concisely describes something that likely took them a long time to figure out.

"As many others have said, much of what people are building today is 'features' not products. As long as that is the case, raw economics is the real problem with the software business. Competition strengthens the platform builders while eviscerating the component builders... The rich get richer and the builders of innovative features must be satisfied with the 'personal rewards' of doing a job well unless they get lucky in the buy-out lottery game."

With the various cloud computing efforts well underway, and with each providing a different approach as a 'platform', it would be good for developers and entrepreneurs to think about which road to travel.

More REST and Erlang

Man, I wish I had the time to dig into these posts about Erlang, Yaws and RESTful services.

Cross domain web apps

This post about cross domain Web apps is spot on. The growth of the current Web was due to millions of silos - no data shared between sites. Over the past few years technology like iframes, ajax and other styles of 'mashups' have made it possible to create applications composed from cross-domain services. I believe this will drive the next major growth curve of the Web.

Elephant on a Bicycle

Here's an interesting blog on China from someone living there. The photos are both beautiful and intriguing.

Elephant on a Bicycle

An example of the writing (which I really like)

I'm now in Xining, Qinghai, in a youth hostel 15 floors up - which you think would offer remarkable views of the mountains that crowd the city on all sides. But it doesn't, this is China. 15 floors up generally just presents you with worse pollution, as it seems to procrastinate in mid-air limbo.

I'm in a quiet little room with two computers. Beside me sits a young monk of about 18, who has rather rashly decided to accessorise his austere crimson robes with a pink feather boa. I couldn't make that up. Furthermore, he is currently downloading provocative pictures of Nicole Kidman; certainly harder to find fault with this inevitable phase of teenage experimentation.

Xining city is a fascinating stopover because it offers a unique diversity. It is still predominantly Han Chinese, in both it's populace and it's blindfolded embrace of modernity - noisy, dirty, difficult to look at, pulsing
with energy. But alongside this, it contains a deeply religious aspect from the presence of size-able pockets of Islamic and Tibetan communities. In places there even exists a meditative quiet - a stillness - which amidst the commercial bustle and the perpetual foot-race of human progress can seem almost revolutionary. This is especially apparent in the areas surrounding the great mosque where I spent most of the day after befriending a young Koranic scholar named Farooq.

Party like it's 1994

You learn something new every day. Although sometimes it takes 5,110 days.
From Steve Vinoski's blog - here's
A Note on Distributed Computing from Sun in 1994.

We argue that objects that interact in a distributed system need to be dealt with in ways that are intrinsically different from objects that interact in a single address space. These differences are required because distributed systems require that the programmer be aware of latency, have a different model of memory access, and take into account issues of concurrency and partial failure.

What Sucks About Erlang

Very funny yet enlightening post about Erlang. And I still want to build a real system with it.

From Damien Katz: What Sucks About Erlang:

"The only purpose of the true -> ok line is to give it an else condition to match. That weird taste in the back of your throat? It's probably vomit."

Seattle's nuclear reactor

I knew that some universities have or had nuclear reactors for research, but I didn't know they could look this beautiful.


From Crosscut:

Abby Martin hopes that, whatever its fate, the Nuclear Reactor Building finally gets public acknowledgment of its role in history, and credit for its architectural originality. That would be in keeping with its original intent. It's hot core may be gone, but it can still teach us something.

No more ugly desktop software

From
ReadWriteWeb it seems Mr Kirkpatrick is all a quiver with Adobe AIR :

AIR lets developers use Adobe Flash, Adobe Flex, HTML and AJAX to create desktop apps. That means no more ugly desktop software!

No more ugly desktop software. Do you really think it was the software that was the problem before? Unless you're part of the short-attention-span-journalism industry, it'll take more than drop shadows and bouncy icons to get past ugly software.

Dare Obasanjo aka Carnage4Life - How "View Source" Broke the Web

I haven't been following the Web blogging elite recently (really, 'view source' broke the Web??), but this post - How "View Source" Broke the Web - made me think of Bob Dylan.

Broken lines, broken strings,
Broken threads, broken springs,
Broken idols, broken heads,
People sleeping in broken beds.
Ain't no use jiving
Ain't no use joking
Everything is broken.

SpaceX signs more business, builds more engines

It looks like SpaceX is continuing to make good progress, both with engineering and with business.
From the SpaceX status page:

"For the last few months, optional activities such as website updates have gone on the back burner while we finished the regeneratively cooled Merlin 1C engine, got the Falcon 9 first stage integrated, proof tested and fired, signed up our first GTO (geostationary transfer orbit) customer - Avanti Communications Group, and took our COTS system past the big CDR milestone."

Signing a contract for placement into a geostationary orbit is huge! I'm really hoping these folks don't spend all their capital on development of bigger engines without establishing the operational business that will generate revenue. They seem to be looking to raise $100M in the first half of 2008, hopefully that will happen and keep this company going to profitability.

Yowza! MicroHoo in the future?

Looks like Microsoft is serious about buying advertising's future - they have offered $44B for Yahoo.
Favorite punditry (from Stowe Boyd)

"Personally, I think the Microsoft and Yahoo matchup is like two tired swimmers who bump into each other and then wind up drowning each other in their scramble to survive. But Yahoo will be the first to go under in this embrace."

This one from iMe on Twitter is good too:

"Microsoft and Yahoo! Its like a blind man trying to lead a deaf guide dog...."

Stonebraker on MapReduce

I've too busy lately to post on all the exciting things happening in the database world - more people getting into column-oriented storage, Amazon's SimpleDB service, consumer database Web services like blist and LongJump, Sun buying MySQL. But I couldn't pass up commenting on Joe Gregorio's post on Stonebraker on MapReduce.
It seems everybody is panning Stonebraker's evaluation of MapReduce as incorrectly comparing it to a DBMS. One commenter even said (ironic comment of the year) Michael Stonebraker should learn what a DMBMS is.

The point, which I'm sure someone has pointed out, can be found by looking at the summary of their findings. Here are a few:
- A sub-optimal implementation, in that it uses brute force instead of indexing
- Missing most of the features that are routinely included in current DBMS
- Incompatible with all of the tools DBMS users have come to depend on

These read like a quote from The Innovator's Dilemma. People enjoy the benefits of MapReduce for large scale data processing because of these points. The lack of support for these DBMS features and tools are the reason it scales like a mother fucker. That is the feature people want. And it does it 10x better and cheaper than anything else. Those other things simply don't matter to them. It's a new audience and a new market.

Animation, Pivot and Maya

Over the past few months I have been volunteering in my son's gradeschool class to teach art and animation using computers. The class has several computers in the room and the school has a good computer lab with lots of equipment, and of course most of the kids have a computer at home - quite a change from when I was growing up.

Before I started, I did a little research into available programs for art, sound and animation. There aren't a lot of solid applications freely available, but here are the ones that I found to be interesting:

• ArtRage - beautiful digital oil painting


• Pivot - stick figure animation (very addictive)


• Scratch - visual programming of sprites from MIT (essentially visual logo)


• Microsoft Movie Maker - simple track-based video and audio compositing

I found one sound effects generator but haven't found the perfect sound effects program appropriate for these kids. Yesterday I found Reaper, a full featured audio track and effects program - it looks wicked cool, but complicated for gradeschoolers.

The kids have amazed me by picking it all up really quickly. Several of the kids already knew about the Pivot cell animation program, and others have already made stop motion animation with Legos, digital cameras and Microsoft Movie Maker. Normally I would have expected the boys to be the ones more into working with the computer, but in this class everybody is very excited and has been diving in. All the kids are trying their hand at different skills - some are more comfortable creating a background painting in ArtRage, but still make a go of animation in Pivot. Others quickly grabbed some background photos off the Web and brought them into Paint.net and faded them out to make a good background.

The teacher has told me that she has to shoo them out during recess because they'd rather work on their animations than play outside! Makes me feel so proud...

Right now I have them working on a project in teams of two. To explain why working in teams is a good thing to learn, I told them of time I visited Pixar and learned how they used teams - an 'artist' and an 'engineer'. They got a kick out of my describing walking the halls and looking at pencil sketches of a cowboy and a spaceman - I thought the movie idea was great but didn't know if it would have mass appeal. I still regret not having the gumption to ask for one of those sketches - I was so in awe of meeting Ed Catmull at the time, it was hard to speak! I'm sure I made a fool out of myself...

Recently I've thought about introducing the class to 3D graphics, but it can be very time consuming to create the models and I haven't found a good animation program for use with pre-existing models. I did recently download the personal edition of Maya, and maybe there's something in there but I'm guessing it'll be too much for the kids.

Edgeio shutting down

Looks like Edgeio is shutting down. That's too bad, I liked the idea of supporting decentralized listings of offers. I suppose being a centralized aggregator wasn't the way to go.

Good quotes from Michael Arrington's (a co-founder) comments:

# Andrew
December 7th, 2007 at 12:31 am
what exactly did you spend 5 million dollars on?

# Michael Arrington
December 7th, 2007 at 12:32 am
Andrew - parties, scotch, hookers, blow. you know, the usual.

The Old Web = Ten Million Catalogs

Stowe Boyd always has something interesting and thought provoking to share on his /Message blog. If you are interested in the intersection of technology and society, I highly recommend giving a little attention.

Today's post is
The Old Web = Ten Million Catalogs (part of his The Social Web: What's The New Web Worth series) which looks back at the past ten years of the Web to draw out the distinction of what the new Social Web has become.

These services are based on a catalog metaphor, where sellers can offer goods or services, and buyers (generally consumers) can find them and acquire them. The volume and low overhead of online services hollowed out the markets in most areas thet they touched, for example, sideswiping brick-and-mortar bookstores, blowing up the travel agent business, and strongly cratering the head hunter marketplace.

I like where this is going - pointing out a 'catalog metaphor' for Web services helps me describe to people how my current company isn't just a directory of people - it's a way to add a social dimension to the Web, centered on people not pages.

Steve Vinoski’s Blog

Steve Vinoski is one cool dude. I've always had the utmost respect for IONA and even after leaving a while back, Steve continues to show the attitude of professional engineering that garnered that respect. With his measured explanation of his view of REST and Dare's theory and practice post I think I'm ready to walk away from cooling embers of the dying REST .vs. SOAP flame war. Time to unsubscribe from rest-discuss.

From Steve's blog, here are some good quotes:

People seem to get really upset when I say that the static typing benefits of popular imperative languages are greatly exaggerated, and when I say that developing real, working systems in dynamic languages is not only possible, it’s preferable.

Either way, no interface definition language is ever going to keep you or some other real live person from having to figure out what the service actually does and how to actually use it, and then coding your client accordingly.

Remember, REST is an example of applying well-chosen constraints to achieve desired architectural properties for a broad class of distributed systems, and so that’s what its constraints are all about.

Either way, anti-REST folks commonly claim that REST’s success is due only to the fact that there’s a human-driven browser in the mix, but that’s one of the dumbest things I’ve ever heard.

Having a solid thinker like Steve Vinoski blogging makes the Web a better place. Can't wait to hear the details of what he's working on.

Amazon PR: Neither Open Nor Social

Looks like there was a bit of a messaging snafu between Marshall Kirkpatrick and Amazon and the results are not pretty - ouch. There's messaging then there's messaging. Let's see what the 'go-forward strategy' is...

Is this just communication gone awry, or is there a culture clash looming?

From Read/WriteWeb
- Amazon PR: Neither Open Nor Social

Did you know that there's been no RSS feeds for top selling items in categories at Amazon.com? Well, there is now - and they were so excited that they figured it out, that they wrote it up in a press release.

Nielsen on Generic Commands

I ran across this interesting bit from Jakob Nielsen's UseIt site about user interface design. It's about using the same few commands in a UI and is an interesting twin to the 'uniform interface' aspect of REST. Although a 'user interface' and a 'network interface' are very different beasts I can't help but wonder if there's some fundamental reason for the similarity.

Summary: Applications can give users access to a richer feature set by using the same few commands to achieve many related functions.

In application design, there's a tension between power and simplicity: Users want the ability to get a lot done, but they don't want to take the time to learn lots of complicated features.

OpenSocial - what? no logo?

Everybody has been commenting on the news of the week - Google and MySpace spinning out their own system for applets within social networks called OpenSocial.

It seems everyone has missed the biggest gap in the OpenSocial system - they have no logo! Rather than talk about the disappointing lack of open content in their 'extension' to Atom for profile data (anyone heard of hCard or FOAF?) or the lack of anything innovative like client-side includes (that might actually lead to social networks being spidered by a big search engine) I've decided to contribute my l33t grafx skillz to the community in the form of an OpenSocial logo. It includes the mandatory Web 2.0/startup color scheme of blue and orange. I need to figure out how to include the RSS radio waves into the design somehow...

or this...

Next, a motto : "Where do I belong today?"

Armadillo Aerospace almost wins prize

So close... Armadillo Aerospace had a couple good flights but tipped over just before touchdown and the next day caught on fire on takeoff. They were this close to winning the Level 1 prize in the Lunar Lander Challenge.


More about the Lunar Lander Challenge over here.

Friday humor

Gotta love inner-thought-humor on a sunny Friday.

• You want four million users by DECEMBER?? You have four hundred active licenses for your product currently! Nothing - and I mean NOTHING - is going to add four zeros to the end of that number in three months short of hiring Arthur Anderson to handle the bookkeeping.



• Wait... First you wanted to clone Digg... Then you wanted to "add the social aspects of Facebook to it," and NOW you want it to be Wikipedia? Where the HELL did you spend your morning? In the "Web 2.0 Company Names to Memorize" symposium sponsored by the local Linux Enthusiasts club?



• Uh... Four million active users means minimum 20,000 concurrent users at any given moment, and you want to do all of this on ONE co-located virtual server in India? On .Net and MS SQL Server? Honestly? You really, really think that's how it will go? In that case, can I punch you? Please? I mean, I only ask because you seem like the type of person who'd ponder the question and then just blurt out "Yes," and I've been dying to hit something since I pressed "1" to join your conference.

The Web! It's People!

Tim Bray always has interesting things to say - I've enjoyed the distributed discussion he's started around Erlang - but his latest piece is a direct echo from the Others Online playbook (emphasis added):

Here’s the thing: the Net’s killer app has always been other people. There are side benefits, like access to all the world’s information. But the links that matter aren’t between pages but people, and they’re strong and rich and subtle. Multiply the infinite flavors in human relationships by a thickening bundle of means-to-connect; that product is what’s new and what’s good and what’s exciting. People who are looking for the Next Big Thing are mostly looking in the wrong places. And anyway, you don’t need to look, it’ll find you.

Good tips on starting a startup

This is a very decent (but short) article on things to do and think about if you are starting to develop your own small Web-based software startup. It's from Read/Write Web and part of a larger series. If you have fifteen years of experience, you'll know this already - otherwise take a look.
How To Create a Web App

Amazon Payment System

I was looking at the Amazon Flexible Payment System service API and it's a whopping big set of docs. Unfortunately, I was sorely disappointed to see the operations in the HTTP based API (called a REST Request) seem to all be GET requests with an Action= query term. Ugh.
The documentation doesn't even mention what HTTP method to use, what content-type to submit or expect as a return, etc. I'm pretty sure that they simply don't know the difference. Sad but true. At least there are docs.

I suppose they didn't have the time to make it simple.

NASA Tech Briefs

A long time ago I used to subscribe to NASA Tech Briefs - a slim monthly magazine full of hard-core engineering articles and light scientific reporting.
A few months ago a friend had an issue and I decided to see what kind of online presence they have now - and they have a good one. No Atom feed, but still lots of techno-bits!

NASA Tech Briefs

Metaplace: open DIY virtual worlds

This I have to check out... it sounds very much like what I've wanted to build for the past ten years or so.

Metaplace: open DIY virtual worlds for everyone

Facebook - the end is near

Ah, yes. Facebook is on the decline already - when I logged in this week, I got a an ad for Zwinky. Animated, colorful, annoying. Completely not me. Except the annoying part.

CouchDB: Thinking beyond the RDBMS

From Assaf at labnotes,
CouchDB: Thinking beyond the RDBMS

It stores document in a flat space.

There are no schemas. But you do store (and retrieve) JSON objects. Cool kids rejoice.

And all this happens using Real REST (you know, the one with PUT, DELETE and no envelopes to hide stuff), so it doesn’t matter that CouchDB is implemented in Erlang. (In fact, Erlang is a feature)

Here’s where it gets interesting. There are no indexes. So your first option is knowing the name of the document you want to retrieve. The second is referencing it from another document. And remember, it’s JSON in/JSON out, with REST access all around, so relative URLs and you’re fine.

Hmm, Erlang. Hmm, no schemas. Hmm, no write consistency. Sounds perfect.

HTTP errors - a photoset on Flickr

(From BB)

HTTP - a protocol so simple, even a cartoonist can understand it. (and why haven't you bought a shirt yet?)

404 Arrgghh - (insert pirate code joke here)

SubAtomic future

I just had a thought - what if the Atom Publishing Protocol and Subversion had a mashup - we could call it SubAtomic!

Object serialization raises it's insidious head

Funny post from Dare Obasanjo decrying the horrible state of affairs created by the company he works for.

This part is about object serialization and version incompatibility.

However, the insidious thing is that the failure wasn’t because their application was improperly coded to fail if it saw a fruit it didn’t know, it was because the platform they built on was statically typed. Specifically, the Web Services platform automatically converted the XML to objects by looking at our WSDL file (i.e. the interface definition language which stated up front which types are returned by our service) . So this meant that any time new types were added to our service, our WSDL file would be updated and any application invoking our service which was built on a Web services platform that performed such XML<->object mapping and was statically typed would need to be recompiled. Yes, recompiled.

Insidious indeed! Who in their right mind could have predicted such things!

And this...

It’s sad that as an industry we built a technology on an eXtensible Markup Language (XML) and our first instinct was to make it as inflexible as technology that is two decades old which was never meant to scale to a global network like the World Wide Web.

Who's this industry of which you speak, Kemosabe?

Getting all meta

Dare Obasanjo seemingly always has time to create detailed and thoughtful posts, I've gone from being annoyed at early posts to reading them regularly.
His latest two posts are worth reading as well, but I can't help but comment on his strawman characterization of ReST.

I should probably start out by pointing out that the title of this post is a lie. By definition, RESTful protocols can not be truly SQL-like because they depend on Uniform Resource Identifiers (URIs aka URLs) for identifying resources. URIs on the Web are really just URLs and URLs are really just hierarchical paths to a particular resource similar to the paths on your local file system (e.g. /users/mark/bobapples, A:\Temp\car.jpeg). Fundamentally URIs identify a single resource or aset of resources. On the other hand, SQL is primarily about dealing with relational data which meansyou write queries that span multiple tables (i.e. resources). A syntax for addressing single resources (i.e. URLs/URIs) is fundamentally incompatible with a query language that operates over multiple resources. This was one ofthe primary reasons the W3C created XQuery even though we already had XPath.

That said, being able to perform sorting, filtering, and aggregate operations over a single set of resources via a URI is extremely useful and is a fundamental aspect of the Web today. As Sam Ruby points out in his blog post Etymology, a search results page is fundamentally RESTful even though its URI identifies a query as opposed to a specific resource or set of resources [although you could get meta and say it identifies the set of resources that meet your search criteria].

I don't think there's any harm in the mis-characterization of ReST and the detail of the post about the two query systems from Google and Microsoft is rather interesting, but I was surprised at his misunderstanding since I thought he knew better. Dare does have an out - as any good program manager does - by saying "... you could get meta...", so I will take that opportunity and get meta.

• "URIs on the Web are really just URLs and URLs are really just hierarchical paths to a particular resource [...]" - no, they are not really just hierarchical paths. They are an identifier. Sometimes they are a 'path', but usually not.
  
• "Fundamentally URIs identify a single resource or a set of resources." - always just a single resource.
  
• "A syntax for addressing single resources (i.e. URLs/URIs) is fundamentally incompatible with a query language that operates over multiple resources." - this is core missed direction in the explanation of ReST. The term 'resource' as applied within RDBMS isn't the same as 'resource' as applied within ReST. A ReST 'resource' is the view from the outside, and datbase 'resources' are the data elements on the inside. Read up on Data on the Outside vs. Data on the Inside by Pat Helland (not exactly relevant to this post, but a catchy title).
  
• "a search results page is fundamentally RESTful even though its URI identifies a query as opposed to a specific resource or set of resources [although you could get meta and say it identifies the set of resources that meet your search criteria]." - the URI actually does identify the results (as Dare suggests) and not the query. The URI is the query. This means you can operate on the set of data that are the results - you could delete them, retrieve them, replace them, etc.
  

I'm not suggesting that URI are better than or a replacement for queries, just that they are different. They do different things and are both very useful, and when you think in terms of "a URI is a query" you think of the processing of the implementation and miss out on some aspects of Web architecture which naturally arise when you think of "a URI identifies the results".

Ruby on Rails Job Trends

From the indeed.com job search site (which looks very nice by the way), the trend for Ruby on Rails mentions in job postings is on a steep growth curve. The absolute percentage is smaller than other terms, and there may be other factors that contribute to the trend but it's pretty telling.

But to keep things in perspective, compare Ruby on Rails with J2EE job postings

More happy fun job trends for Ajax, Web2.0, et al.

Branding ads and ad listings

I just saw this post on BoingBoing about this ars technica post on the psychology of banner ads which is interesting in light of Google's and Microsoft's recent acquisition of 'creative/banner ad' networks.

"The research concludes that repeated exposure to a product via banner ads generates a positive feeling towards that product. The good news for consumers is that a critical reevaluation of the product can make these positive feelings vanish." and
"This suggests that familiarity-based advertising may work best for impulse buys, where more detailed evaluations aren't likely to occur."

I found these two papers yesterday when reading up on economic theory and internet advertising. This one Internet Advertising and the Generalized Second-Price Auction - Selling Billions of Dollars Worth of Keywords is the most readable of the two and talks about the bidding process for online ad placement. They cover the history and describe the shift from "pay what you bid" to the current "pay the next lower bid" (called a 'generalized second price auction'). Really very interesting stuff.

The second paper Brand and Price Advertising in Online Markets looked at different fundamental forms of advertisements - brand advertising and 'price' advertising. I had high hopes for learning from it, but it was very dense with too much lingo specific to this research area for me to fully understand. Here's an example: In contrast to models where loyalty is exogenous, these crosschannel effects lead to a continuum of symmetric equilibria. Yeah, I, uh, was thinking the same thing.

Their models and assumptions also seem questionable and so I don't know that their results apply to the world we live in, as opposed to the simplified models they used to prove their theory. In any case, the questions being asked and the attempt to find answers are still valuable.

Here are their findings (and again, these may not really apply to the real world)

While each firm finds it optimal to advertise its brand in an attempt to “grow” its base of loyal customers, in equilibrium, branding (1) reduces firm profits, (2) increases prices paid by loyals and shoppers, and (3) adversely affects gatekeepers operating price comparison sites. Branding also tightens the range of prices and reduces the value of the price information provided by a comparison site.

Their research shows that brand advertising allows a firm to have higher prices, since loyal consumers aren't as price sensitive, but their conclusions are that profits are less - which I don't understand, unless the cost of creating the brand is very high. The Ars Technica post shows that other research continues to confirm that viewing brand ads creates a positive impression, which is one step towards converting shoppers into loyal customers.

This got me thinking about the various forms of advertising. The second paper distinguishes 'brand advertising' from 'informational advertising', which I agree is a useful distinction. I think of it in terms of how actionable the advertising is - how delayed is the payoff. For branding, the payoff is very indirect but could be profitable (or not, depending on which research you subscribe to) due to higher prices or repeat business or cutting out the competition through causing the customer to not engage in comparison shopping. For ad listings that show a very specific product or category, which are often a gateway to a purchasing decision, the payoff is fairly direct. Some even feel that the advertising cost model may evolve into 'cost per action' and go beyond 'cost per click'. However, cost per click is currently much easier to gather metrics in a two-way trusted fashion than measuring the final transaction in a two-way trusted fashion.

Another example of a very actionable advertisement is the Amazon or EBay offer listings - these are immediately purchasable, and through syndication via Associates are widely used as advertisements. I don't know of any company that does placement optimization of Amazon Associate links (raising placement for offer with better click through rates, better commission for the associate, etc), but I think Amazon has started doing some of that with Omakase links. One interesting thing to consider is that both Amazon and EBay are similar to price comparison sites due to the large number of offers for a single authoritative item (EBay doesn't have very good item authority, but people work around that issue by using manual searches). Getting top placement on an Amazon offer listing page, or in the 'buy box' on the details page, doesn't use an auction bidding process the way Google or Yahoo paid search listings do. The offer listing position is based on the offering price and estimated shipping costs.

Online Ad industry consolidation

So, what's up with all the acquisitions of online ad networks within the past 30 days?

• Google purchased DoubleClick for $3.1B on 4/13/2007 (on revenues of $150M).
• Yahoo purchased the remaining interest in RightMedia for $680M on 4/29/2007 (on revenues of $70M) They had purchased a 20% stake back in Oct 2006.
• Microsoft acquired European mobile ad network ScreenTonic on 5/3/2007.
• Microsoft acquired aQuantive for $6.1B on 5/18/2007 (on revenue of $442M).
• AOL acquires major interest in Adtech AG on 5/16/2007.
• AOL acquires mobile ad network Third Screen Media on 5/17/2007.
• WPP Group acquired 24/7 RealMedia for $650M on 5/17/2007.

A few billion here, a few billion there, pretty soon you're talking real money.

What's happening is different for each player, but the overall trend is the same - expanding beyond paid listings into creative branding. Paid search was $6.7B last year and brand advertising was $3.3B. This interest in brand advertising may be a reaction to the expectation that television - which is mostly branding style ads - is moving online.

In Google's case, they are buying a company that has been successful with creative ads, essentially banner ads. Banner ads are the most common choice for branding rather than being used for actionable ad listings. They also inherit distribution agreements for AOL and MySpace, and more distribution capacity helps draw advertisers into Google to bid for placement.

Microsoft hasn't done well in any online ad segment - listings or branding - and with their acquisition they will be more involved with holistic ad campaigns and deep in the creative arena of advertising. The includes the ad agencies that do the actual construction of creative ads and create very innovative branding experiences like custom website which are blurring the lines between interaction and advertising. There may be some future tie-in with 'rich internet applications' and Silverlight. I may be wrong, but I don't see aQuantive providing additional distribution capacity - 'inventory' as the ad industry calls it. They of course claim it 'extends their platform'. Everything's a platform to Microsoft. Maybe they should simply try providing value instead.

The hope is that the contextual and behavioral profiling that is done for ad listings will be applied to better target brand advertisements. A requirement for this to work is for the ad network to know a lot about their audience - something that a single site cannot accomplish. Effective audience profiling is orthogonal to Web sites - orthogonal to the Web's organization. However, by looking at the architecture and technologies of the Web you can see the areas where this multi-site capability can exist :

• clients such as rich internet applications, browsers and browser extensions like toolbars
• intermediaries such as the proxies that ISPs like Comcast operate
• compound resource structure of current Web documents. Since each resource can be retrieved from different domains, information can leak between domains.

Look for control or partnerships in these areas in the future.

Command Links

Uh oh. I see a mess coming up...
From Jakob Nielsen's Alertbox post on
Command Links

"Windows Vista introduced a new GUI widget for commands: the command link. Once something is in the system that people use on a daily basis, it becomes a de facto standard. Because they'll encounter them frequently in Vista, users will come to know and expect command links."

From what I gather, Vista 'command links' appear to be glorified buttons for native applications, not 'underlined text' links on Web pages.

However, Jakob continues with the following regarding web page links:

To reduce confusion, link text should explicitly state that it leads to an action and not just to a new page. It's not enough to communicate this info in the surrounding text; users often scan Web pages for the areas they can act on. Thus, you should assume that most users will only read the link text. In fact, users often read only the link text's first few words, so it's important to start with a word (typically a verb) that indicates the action that results if they click the link.

It appears Jakob is also implicitly approving the use of a link on a web page as an 'action'. From my reading, I think this is seriously wrong. I believe web page 'action links' need to satisfy the following two requirements

• visually distinct from normal 'safe' links
  
• syntactically distinct from normal 'safe' links
  

The second point is important because of the large number of automated agents that traverse the Web through hyperlinks. Adding unsafe links into the web will cause confusion among both people and software agents.

It’s my Vineyard

I found this blog about a couple that have purchased a vineyard in France - what a dream!

It’s my Vineyard

"Sitting in glorious sunshine on the terrace of The Restaurant du Pont having a delicious lunch we are reflecting that it is almost two years ago that we fell in love and bought Maison des Bulliats and its vines in Regnie, Beaujolais, one of the most idylic spots on earth."

This reminds me of the book "The Olive Season" which is about an impetuous British actress that purchases a run down villa and Olive 'garden' in the South of France. We found that book in the apartment we rented while we stayed in France two summers ago. Interesting book - although the scattered personality and writing of the author can get exasperating - and I'm looking forward to following this French vineyard blog.

XML is in the House

I ran across this page - Legislative Documents in XML at the United States House of Representatives: "The purpose of this website is to provide information about the ongoing work of the U.S. House of Representatives in relation to the eXtensible Markup Language" - and thought the host name was very cool - xml.house.gov.
This page has pointers to DTDs and schemas for the governmental processes involved with making and amending laws. So the next time we need to form a more perfect union, we've got this going for us.

And that page led to this page of a summary of floor proceedings of the US House of Representatives. It reads like a blog, only with less detail and there is no RSS feed.

Apollo, Silverlight, blah blah blah

Looks like Hugh W is one of the few questioning the value of Silverlight and RIA for the Web.

Flash, applets, Silverlight, Javascript -- the more you use them, the suckier your web apps are at exploring the web information space. I don't think it has to be this way, but it takes a design discipline few seem to have. These programming models are from the 80s. They have web APIs, but they're not web oriented. Programs end up as little desktop applications, not web apps. I don't see Silverlight changing that. It is good to have super expressive widgets -- hear hear. But if you're not pushing a bunch of hypertext down to my browser, you're not helping me explore the space.

I agree with his sentiment - you might say that RIA is to user interfaces as RPC is to messaging interfaces : more is not better. There probably will be a few years of smooth looking but hard to use (and harder to re-use) applications, while we wait for people to re-learn the basics of usability. I can only hope folks read Nielsen's UseIt column.

Mathematica 6

It looks like Wolfram Research has release a huge new release of Mathematica. I have only dabbled in Mathematica but I could spend all day playing and learning with it. It's hard to believe Mathematica came out almost twenty years ago!

Something they've added recently - and apparently improved on - is server based computing and visualization. Imagine what could be done by putting this together with something like Amazon's Elastic Compute Cloud (EC2).

See the Wolfram Blog for more details.

REST - it's inevitable

A couple weeks ago I was having lunch with a friend at Amazon - who coincidentally used to work at MS with the data access team and is a brilliant architect and engineer - and we talked about REST and how he was helping use REST concepts in refactoring some core back-end services. I made the observation that I was never stressed about how long it has taken for REST to achieve common industry understanding and acceptance - I just said "It's inevitable". I didn't realize just how short it would be for inevitable to show up.

It looks like REST has taken Redmond by storm. First I read that Mark Baker did some consulting with Microsoft (I missed the chance to have dinner with him when he was in town due to email snafu - major bummer). Then I read Dare Obasanjo's post that says "REST is totally sweeping Microsoft."

We passed the tipping point quite a while back, but it's still good to see pragmatic architectural sensibilities take root finally.

Sparkly things factory

Oh, this quote from Performancing.com is beautiful!

"Snap's preview anywhere gizmo is ruining the reading experience for millions of people. Its intrusive, obstructive and unuseful in almost every respect and use case. The fact that so many big blogs are using it, big well respected blogs, does not mean that it's useful, it just means that they, like most bloggers, have all the self restraint of a magpie in a sparkly things factory."

Amazon.com Widgets and hypertext

It looks like Typepad and Amazon are collaborating to help bloggers link to Amazon products. They have three widgets outlined, and one of them - the quick linker widget - uses custom attributes on an HTML anchor tag to make it easier to reference a set of products. The 'old school' would just use a URI, but those are hard to construct, hard to type and the wizards slow people down, blah blah blah. This not-quite-micro-formats approach is more understandable and more forgiving for hand-crafted markup. They define a new 'type' attribute for the anchor with the value "amzn". Then there are several more attributes like 'search' or 'category' or you can create a direct link via an 'asin' attribute. I assume some snippet of javascript would scan the page after it was loaded and construct the URI on the fly and set the href attribute on these anchors.
This is a creative solution to the "how do you construct a URI" problem, but it does leave spiders out in the cold, breaking the hyperlinking which defines the Web.

However, there is a simple approach they could use which would make a fully declarative and locally described document work and continue to allow auto-discover of hyperlinks to work - add a 'meta' tag to the head of the document with the URI template that corresponds to the 'type' attribute. I think the URI template proposal may need to do a bit of work related to optional or conditional patterns, or the URI template could stay static and the type="amzn" could change to be type="amzn-direct-link" or some other more qualified value.

Chocolate. Real chocolate.

The other day I was lucky to tag along with Jordan to the ESIF (Early Stage Investment Forum) in Seattle. The event was held by NWEN (Northwest Entrepreneur Network) and is sort of a 'graduation' milestone for local entrepreneurs and startups. We weren't the only software startup, but there were many other types of companies represented - bio-tech, high tech, textiles and gourmet foods. Not just any gourmet food - gourmet, hand-crafted and wonderfully made chocolate. Real chocolate. Not the burn-your-throat, over sugared, waxy poo that comes from Pennsylvania. I had heard rumors of a coconut curry chocolate so during a lull in booth duty I wandered around to score some chocolate. Several years ago a friend visited Mexico and brought home some chocolate that was flavored with chili powder and granulated sugar and the taste was fantastic so I assumed this curry chocolate would be just as good. I found the booth with Theo Chocolate and talked with two very friendly folks and asked about the curry chocolate. Naturally they offered a taste and I must say it was great - spicy but smooth and rich with chocolate. I told them that I'm always on the lookout for new chocolate because my wife really likes good chocolate, especially Dutch dark chocolate. Suprisingly, they started pulling out bars and explaining each - this is 91% cacao, this is 65% cacao and so on. Their flavors are intriguing, not only do they have coconut curry chocolate, but also Chai Tea chocolate and exotic chocolate using cacao beans from Ghana and Madagascar and Venezuela. I have only tried the Venezuela chocolate (91% cacao) and it's really becoming addictive - crisp chocolate with a dusky bitterness without being unpleasantly bitter. I hope the 'Special Limited Edition' label doesn't mean it will be hard to get in the future.



(Oh, and I just found this useful Flickr tool to upload photos directly from your desktop with a right-click menu. Quite a time saver.)

Tim Bray stole my flowers

I regularly read Tim Bray's posts, but admit that I skip the details of a few posts about hardware now and again... however his recent flurry of spring flowers made me laugh because it's like he's talking about my backyard.

It first started with a star magnolia, his was blooming a bit after ours had started flowering - here's a photo of ours.

Then it was a rhododendron, which is fairly common in the Pacific Northwest. We have a white with pink tinge and half a dozen more that haven't bloomed yet - they come out at different times of the year each with different colors, very cheerful.


Then there was the trillium which is my favorite flower. My wife bought a bulb and planted it out back by a little pond because she knew I loved them so much. Three came up this year and I missed the chance to take a good photo, but Tim's will do very nicely.


And there are others - tulips, more rhodies, daffodils. The grape hyacinths are pretty, but their fragrance is the best. Put these near your front door and you'll really know when Spring arrives. You might need to check the particular kind of grape hyacinth - ours looks a bit different.

If Tim next publishes photos of hibiscus and sunflowers, I'm going to get suspicious.

CSS - new style, same old sheet

I've been working heavily in the world of HTML and browsers over the past six months and for the most part enjoy it. It's much much better than the scene six years ago when I was at KnowNow writing fairly advanced Javascript client code that was supposed to work on Netscape Navigator and Internet Explorer - what a nightmare. Today's browser world is infinitely better. Now people only complain about box models being a pixel or two off. Well, and there's that z-index bug Microsoft hasn't fixed and probably doesn't even know about, even though everyone else does...

Anyway, my most recent learning experience has been with information extraction from Web pages - essentially extracting meaningful keywords from HTML. I must say, there's a lot of room for learning to take place here. There are several research papers I've found that are really educational, especially those that talk about extraction in the absence of a large body of other documents (corpus) to measure relevance.

As I was going through some experiments I realized that doing a decent job of extracting text from HTML requires knowledge of what 'markup' is and what the particular elements of HTML are defined to mean. Extracting meaningful phrases from markup means to ignore the markup and get to the underlying text which was marked up. But then I began to notice something - in all the advanced HTML pages that use the latest CSS to accomplish 'semantic HTML' (a phrase that I've heard tossed around pretty loosely) something is going wrong. The underlying text that is marked up is becoming gibberish. This is due to the use of CSS for layout and ignoring the effect of the tags on the text. For example, when a span tag is applied to text it is considered an 'inline' element - the underlying text is not meant to be fragmented and split apart and any extraction tool (especially a naive one that I was experimenting with) should merge the text fragments before, within and after the span element with no whitespace. But often designers will add layout and margins to the span tag in order to visually separate the text - yet the underlying markup indicates the text fragments are contiguous. How annoying. There is a simple solution - tag the text as it is intended to be read and understand the difference between 'inline' and 'block' semantics for narrative text.

Inertial electrostatic fusion

I haven't heard any news recently about inertial electrostatic (confinement) fusion, so I was pleased to read a note (on the Google Research blog of all places) mentioning Robert Bussard (former Asst. Director of the AEC) talking about inertial electrostatic fusion. Apparently it's a video - I hate video. I'd rather read about it, but this should be good. I tried to get some information from EMC2 (Bussard's company) a while back, but they didn't answer emails. Hopefully this video will give some interesting info on what they've been doing.

Wow - this is very very cool. Bussard has been working for over ten years on Navy contract to research and build a spherically confined fusion reaction - and they've done it. The funding from the Navy dried up, his company has the patent and now the next step is to get funding to build a full scale prototype. After that - cheap and limitless energy for the world. Wow.

DoubleClick and Google

There's been a lot of industry angst over Google's pending acquisition of DoubleClick. I'm fairly new to the game of online advertising so I don't know enough to predict the fallout.
Here are two good posts to read - the first is from Pulse360 Blog and is a sky-is-falling viewpoint. The second is better and is from Jordan's blog (the CEO of the startup I work for) and gives a good analysis of the value to Google and why they made the acquisition.

There are a lot of terms that I wasn't familiar with several months ago, so here is my cheat sheet of terms:

• banner advertising - wide advertisement usually at the top of a page
  
• inventory - this is what web page publishers have to offer, the space on their pages and the audience that will view those pages
  
• remnant inventory - areas of a website that are not very popular and the web page publisher cannot charge lots of money for ad placement (because there are no viewers)
  
• ad network - provider of ad listings
  
• eCPM - effective click-per-mille (which is click-per-thousand page views)
  
• creative - the 'creative content' that shows up in the ad listing. Google made big bucks doing just text and links and left the annoying flashing ads to others.
  
• AdSense network - the Google system that provides ad listings to folks publishing web pages. It's supposed to provide an ad listing that is highly relevant to the page it is injected into, but that sometimes doesn't work.
  

(more words here)

It's interesting that the space on web pages is called 'inventory' - coming from Amazon I had a different view of 'inventory'. There are a lot of recurring themes between the world of product catalogs and inventory management that I am familiar with and the new world of online advertising. Maybe it'll all make sense eventually.

Sinterklaas Boot

And now for something completely different...
Over on my music blog I posted about a great techno dance song called Boten Anna - but this Sinterklaas Boot video which is a spoof from Holland (those crafty Dutch!) is the real gem!

Space Conference

Here's something interesting from Wired Private Launches, New Tech ... This Isn't Your Parents' Space Age

This week an estimated 7,000 government officials, corporate representatives and space enthusiasts will converge at the annual National Space Symposium here to hash out the technological, cultural and political issues surrounding the next decade's push for manned exploration of space.

I hadn't heard about the Constellation program from NASA, but it's very exciting to see the money put into space resulting in good work.

Here are a couple space blogs with a little news Space Politics and Space Report.

How to (Teach how to) Write a Spelling Corrector

Through Bill de hÓra's Bzzt Questions blog post, I found this post from Peter Norvig on How to Write a Spelling Corrector. The spelling corrector post was interesting initially because I've been doing a little text processing recently and have his code echoed the simplicity of approach I needed to use to squeeze the algorithm into JavaScript for use in a widget. But it wasn't strictly the code that really struck me - it was the multi-faceted learning opportunity that the post represents. On one hand, there is the lesson that thinking well before coding is a Really Good Thing. In this case, thinking of the problem in terms of lists, sets and maps clarifies the tasks that the software should perform. On the other hand, demonstrating the simplicity and expressiveness of Python shows that the actual tool used can be important. Especially a tool that removes obstacles between theory and practice. And on the third hand, Peter Norvig's post is a great example of education. It educates readers about processing natural language in the wild, it educates programmers on how programming languages reflect the mental model of the developer, it educates designers on how theory can and should influence the practice of software development and at a higher level it educates everyone on what a real engineering looks like.

Oh, and check this out

Fortunately, Google has released a database of word counts for sequences of up to five word sequences, gathered from a corpus of a trillion words.

Spring in the Northwest

Spring has definitely arrived here in Kirkland. One of the rhododendron and the Star Magnolia tree are in bloom, brilliant white blossoms finally brightening the yard as well as adding their perfume to the air. There is something about the warming of the forest at the beginning of the year that gives a smell of the outdoors that charges me up. I'm very lucky to be able to work from my home over the past six months and I took a break from work this afternoon to split some wood from trees that had fallen in our winter storm. (If you ever get tired of spinning your wheels and want to do something with visible results, come on over, pick up an axe and split some wood!)
Being outside is so wonderful - the sound of birds (especially the Northern Flickers), the random bumbling path of a bumble bee and even the first butterfly I've seen this year all make me grateful for the place I live. There was even a pileated woodpecker hopping around the lower part of the Doug Fir outside my office window. If my cat hadn't been asleep on my lap he would have gone nuts!

SimpleBits

Although I spend most of my time working in and around server-side systems, once in a while I find myself designing and building client side software. Some of that work is the visual and interaction bits, which is more challenging than designing servers - at least for me. Over the past few months I've worked heavily with CSS as applied to HTML and XML. Using CSS is fantastic. Sometimes infuriatingly inconsistent, but fantastic. I have a fascination with good 'design' - the visual kind of design that I appreciate but can't quite create. One site I stumbled on recently is SimpleBits by Dan Cederholm. This seems like a good starting point for exploring the world of page design, styling, typography and user interaction.

SpaceX launching demo flight #2 today

I don't know where I've been, but I had no idea that SpaceX corporation is so far along with their technology and their business. They three configurations - light, medium and heavy. These can reach LEO, GEO and the heavy can support planetary missions! A commercial company that can reach another planet! They are broadcasting the launch also.

This video of their Falcon1 launch in late 2006 kicks so much ass over the BlueOrigin test launch - although when BlueOrigin launches that cute iPod of a rocket for real, that will be so very cool.

Update - the launch was delayed 45 minutes, but the live webcast is on. Cool! I can see vapors from the fuel topping off! "T minus 20 minutes and counting..." Wicked!

Update - AARggh! They aborted launch at T minus 1:15. Not sure why yet.

Update - The launch engineer has announced termination of the webcast, but the launch will be rescheduled for tomorrow same time.

(Check out the google maps of the Omelek Island launch site - someday we'll have live Google Earth with the 3D trajectory!)

PS - if you want to chat about this, download this Firefox sidebar and go to the SpaceX home page

The Younger Generation

So I was sitting at my computer when my daughter walks up and asks - "What are you working on?" I told her I was reading about Web technology and happened to see a page on Wikipedia about HTTP. Since I knew she had seen Wikipedia before, I thought I'd impress her by saying that I had added a sentence to this page. So she says - "Oh yeah! I added a sentence to this page here." and she proceeded to take over the mouse and navigate to the page on Cubism where she showed me the sentence she added there - the second sentence at the top of the page. I was the one impressed! And she's not yet a teenager...

Job openings at OthersOnline

I've been busy and having a lot of fun cleaning up and building out different pieces of the OthersOnline system. So far I've fixed a few issues in the InternetExplorer toolbar, implemented the Firefox toolbar, implemented a widget for bloggers, built Web pages for profile search and browse, and variously tweaked and tuned the profile matching algorithms on the server. But now it's time to add a few more engineers. We have two job openings for an experienced software engineer and an experienced UI designer. Feel free to contact me at mike at othersonline dot com, or forward this to someone that may be interested.

Senior User Interface Designer
As a Web applications developer with OthersOnline you will define and create the user experience for all our applications that results in a highly social and interactive community of online users.
You will be responsible for operational support as well as prototyping features and brainstorming ideas. You will design, document and implement usable, enjoyable and engaging features using DHTML, Javascript, CSS, XML and Ajax and work within a Linux and Java based server environment. Candidates must be creative, flexible, self-directed, and understand how to design and write elegant, responsive and maintainable user interfaces.

Senior Software Engineer
As a software engineer with OthersOnline, you will create innovative algorithms and data services to dramatically improve search result relevance and create highly scalable, reliable Web services that extend the reach of our platform.

You will be responsible for operational support as well as brainstorming features and ideas. You will design, document and implement scalable, flexible features using Java, XML, SQL and HTTP in a Linux and Oracle environment. The ability to document technical approaches completely, correctly and concisely is required. Candidates must be creative, flexible, self-directed, and understand how to design and write high-performance, reliable and maintainable code and data models.

Social Content Services and REST

This article from Leigh Dodds about Connecting Social Content Services is an extremely well written summary of several social content services, a greatly pragmatic description of REST and an extremely useful review of the service APIs from those social content services.

If you need a quick list of aspects for engineers to consider when building a ReST based system (the list is derived from Joe Gregorio's site), try this:

• How does the API use URLs to identify resources?
• What HTTP methods does the API support?
• What status codes are returned?
• How are users and applications authenticated to the API?
• Are resources linked to one another (use of hypermedia)?
• What data formats does the API support?
  

This review notes that nearly all services do not use hypermedia which I think is unfortunate but understandable. I've always had a problem resolving the desire to be flexible in allowing the internal data identifiers to be used in many situations and the desire to be trivally easy for clients to access other resources by simply using links - the mashup problem. One issue I have is that the server-side software that generates the representation might not know all the possible resources made available by sibling services. Think of a US postal zip-code - if you have a service that provides weather based on zip code, should that representation also be responsible for linking to all other services - either provided by your system or some other server - that could potentially take in a zip-code? My approach is to return both direct links to known resources (tagged appropriately) as well as the short-form of the identifier, the plain zip-code for example. Microformats sort of do this, but it's a style that isn't well applied by data services.

One comment I have on this review of RESTful services is the author mistakes "URL" for only the portion before the query terms. This results in judging these services to be less RESTful than they actually are. Use of query terms does not mean there is one 'controller' with parameters, it actually increases the number of addressable resources which is good.

Funny story about a bank

Want to hear a funny story?
Okay.
So, this guy walks into a bank south of Seattle, WA.
"Hi" he says "my name is Mike Dierken".
"Here's my social security card, bank account number and Utah driver's license." He hands over a check for $5,000 and wants to withdraw $4,000.
The bank says, "Duh, okay. Sounds good to me." and hands over $4,000 in cash.

Here's the funny part…
The next day, same guy walks into a different bank.
"Hi" he says "my name is Mike Dierken".
"Here's my social security card, bank account number and Utah driver's license." He hands over a check for $5,000 and wants to withdraw $4,000.
This time the bank teller calls the other Mike Dierken who says "What the #!**?" while hearing the teller saying "Sir, please just have a seat over there. No, don't leave…"

I'll be out of the office tomorrow morning. I have a meeting with my local Bank of America branch. I must say, the teller that called me was very smooth and professional, pretending to talk to a 'signature authority' while talking to me. She made her side of the conversation sound normal while still responding to my questions. I imagine even check fraud could be frightening to a teller, as you don't know how they will respond.

Second Life client under GPL

I've never used Second Life, but I like the idea of people creating hyper-spaces. I used to do work with 3D graphics and have always thought the multi-player online games had poor visuals - the shading (coloring of models) and lighting really need work to be realistic. I'd like to see 'scenes' with near movie quality lighting and shading that you can invite a few friends into, yet hyper-link to other spaces. I've even toyed with the idea of a 'better' markup language for 3D - less modeling and more auto-layout capability.

Anyway, I don't know what it will mean to have their client library freely available, but this post on BoingBoing had a comment that really struck me:

"Customers only ever get to love it or leave it. Citizens get to change it."

Happy Birthday Grandma!

Today is my Grandmother's 90th birthday. Happy Birthday!
She lives in Montana, by the Little Blackfoot River just a few houses down from my Uncle. It's always great to go visit. Unfortunately we didn't fly out to see her for her birthday, airline tickets were just too expensive. We'll probably drive out in the Spring - at least the snow should be gone by then!

Windstorm 2006

Taking the limbs off.
Originally uploaded by dierken.

The recent windstorm here in Washington knocked down three trees in our backyard. One snapped at the base and landed on our swingset and narrowly missed our deck and hot tub. The other two tipped over, pulling up a huge rootball and landed on the edge of the yard and into the forest.
We were without power for five days, which wasn't so bad for us. Our fireplace in the basement kept us warm and we learned you can actually bake cookies on a barbecue grill (if your daughter really really wants to make cookies). Some friends dropped their kids off for a sleepover (since our house wasn't freezing) and we all had a good time playing games by candlelight.

Now it's time to finish taking the limbs off the trees and clean the yard - fun fun.

Out of Amazon

Today is my last day as an official Amazon employee. I've been on leave of absence for three months and have decided not to return at this time.

This was not an easy decision - recently I was talking with a team at Amazon about a project they have started and it's exactly the kind of thing I've been wanting to build for years - but there are some other things I'd like to work on elsewhere.

I think Amazon is great, they have smart people and are in a great position to grow the Web. I expect to continue to see great things launched by the people at Amazon and really, really look forward to the project my friends are building.

Thanks for everything!

Happy Sinterklaas and St. Nicholas Day

Happy Sinterklaas and St. Nicholas day!
When I was little, every December 5th was the day we would wake up and find a little package or an orange in our shoes by the fireplace. This is a family tradition from my mother, who grew up in Germany and taught us about St. Nicholas.
And since my wife's family is Dutch, we also get to celebrate Sinterklaas on the 5th - today we opened a package from Sinterklaas and Zwarte Pieten (Black Pete) with my favorite ginger spice cookies and we each found a giant chocolate letter - the first letter of our name. I think there weren't any 'M' letters left, so I got an upside down W!

Why I like blogs

This post over at
edgeio is why I like blogs. It's real, it's about people and and it's about what people are doing. In this case, we hear a brief overview of their first rollout of real-time searching of a hundred million offer listings.
Cool stuff.

Agents, toolbars and adware

Here is a thought provoking post from Brian Smith about agents, adware and toolbars on ComparisonEngines.com. I like that he presents the core value of 'adware' and phrases it in a more general context of 'agents'.

Long ago I had been very excited about agents and event driven applications but spent most of the recent past working just on messaging systems (subscriptions, notifications, etc) rather than the agents themselves. Maybe it's time to go back and revisit agents. Recently I've had this notion that an interesting paradigm for empowering people to build agents might be to use a spreadsheet with event inputs and message outputs - but I haven't even started down the road of working out what it would look like. I think general folks wouldn't understand specialized agent building languages but might be able to poke things together with a visual tool.

The Spreadsheet as Mashup Fabric

The title of Joe Gregorio's recent post - The Spreadsheet as Mashup Fabric - really caught my eye. For the past two days I've been thinking about the spreadsheet as a simplifying tool for building event driven web applications - accepting events, summarizing many messages, routing and republishing the results to a new set of subscribers.

I've just started thinking about this so I don't have any actual situations it might be useful for, but the idea of opening up a compact, rich and information dense document in a very responsive interactive application for ad-hoc applications and messaging components sounds interesting.

Sort of like message-driven beans on acid.

What's old is new again

Check out this short article from 1998 on making more responsive web applications using - get this - javascript.

We can create a more responsive experience if we look at the available technology in a different way. In particular, we are going to forget the assumption that a Web page is loaded directly from a Web server. Instead, we are going to use the request/response structure of the Web in a slightly different way.

I wonder how far back we can see this approach being used or clearly talked about.

37Signals and Google Web Accelerator

From the rest-discuss list, here's a post from 37 Signals that missing the point about the Google Web Accelerator fetching URLs.

This wouldn’t be much of a problem on the public web since it’s pretty tough to be destructive on public web pages, but web apps, with their admin links here and there, can be considerably damaged. If you have a web app, it might be worth returning a 403 when the HTTP_X_MOZ is set to “prefetch” header is sent. This will keep Web Accelerator from clicking destructive links.

I like this "destructive links". A major point of the Web is that links are not destructive.

I think the better approach is to not use GET to modify or remove data. That's simply unnecessary and against the word and spirit of HTTP.

Remember, a link is not a widget. You can't use a simple anchor element as if it were the same as a push button. Whenever an HTML page has an anchor tag, it is explicitly advertising that the URL is safe to retrieve. By sprinkling your HTML with these landmines, it's the author that made the mistake, not the browser or web accelerator.

If you are writing web apps, it might be worth more to think about what's happening and why, rather than hack a workaround that only occasionally avoids a bug introduced by your own application. Fix the bug, and no workaround is necessary. Again - a link is not a widget.

New Direction is new theme for Democratic plan

Is it just me or does the phrase
'New Direction' sound just like 'nude erection'?

I have a bad feeling about this...

Participation Inequality: Lurkers vs. Contributors in Internet Communities

A few years ago I had stopped reading Jakob Nielsen's Alertbox even though I had been reading it for a long time before then and had enjoyed his fact filled articles and analysis. Just the other day I popped back in to check out his recent postings and this one caught my eye - Participation Inequality: Lurkers vs. Contributors in Internet Communities - as I'm looking into communities, social aspects of the Web and such.

The article gives some history and data that covers the basics of a 90-9-1 rule with respect to participation - most people just lurk, some interact occasionally and a few are very active. Many people know this and if you've read Clay Shirky's "Power Laws, Weblogs and Inequality" article you'll be familiar with the idea. Although there have been comments elsewhere that Jakob doesn't "get" RSS (and his alertbox site doesn't have an RSS feed), I think Jakob does understand the implications of RSS and this post provides some value by highlighting some issues and suggesting how to design useful systems in the context of participation inequality.

Search. Search engine results pages (SERP) are mainly sorted based on how many other sites link to each destination. When 0.1% of users do most of the linking, we risk having search relevance get ever more out of whack with what's useful for the remaining 99.9% of users. Search engines need to rely more on behavioral data gathered across samples that better represent users, which is why they are building Internet access services.

Make participation a side effect. Even better, let users participate with zero effort by making their contributions a side effect of something else they're doing. For example, Amazon's "people who bought this book, bought these other books" recommendations are a side effect of people buying books. You don't have to do anything special to have your book preferences entered into the system. Will Hill coined the term read wear for this type of effect: the simple activity of reading (or using) something will "wear" it down and thus leave its marks -- just like a cookbook will automatically fall open to the recipe you prepare the most.

(Bummer - Jakob Nielsen had a 'User Experience 2006' in Seattle a few weeks ago and I missed it...but maybe I can catch some of it on a blog somewhere)

Added: Here is another good article from Jakob Nielsen about traffic log analysis.

The top 10 queries accounted for 10% of the total traffic, so each one of these queries is obviously more important than those that brought only one visitor. Taken together, however, the single-use queries accounted for three times as much traffic as the top 10 queries. This statistic shows the folly of focusing search engine optimization solely on a few high-performing queries. Your site must be found when users enter relevant queries -- and the possibilities are typically vast.

Abundance .vs. Scarcity

This is a good post by Dare Obasanjo commenting on economies of abundance .vs. scarcity with respect to the digital abundance of Internet and the scarcity of time and attention with respect to human intelligence. The Economy of Abundance and Other Fairy Tales.
I first heard the phrase 'economoy of abundance' many years ago in a story called "Manna" by Lee Correy (pseudonym of G. Harry Stine) - a neat science fiction story. I wonder where it first came into use...

Most successful Web companies today are exploiting the scarcity of attention and time that plagues all humans. In a world where there a hundred million websites the problem isn't lack of content, it is finding the right content.