SpaceX reaches orbit

SpaceX became the first private company to launch a liquid fueled rocket into Earth orbit. Totally awesome! This was their fourth launch of the Falcon 1 configuration and was carrying a test payload. Their third launch two months ago didn't achieve orbit and actually was carrying paying payload - oops.

This is a huge step on the road to space.

SpaceX - Flight 3 of the Falcon 1

SpaceX is launching their Falcon 1 today from Kwajalein - about 1 hour to go before liftoff, if things go according to plan.

Here's a live webcast - http://www.spacex.com/webcast.php

WebHooks

This looks interesting - in a 'teach people how the Web really works' kind of way. WebHooks is a catch phrase for Web application development where notification are sent from theh source to the listener via HTTP POST, rather than the other way around via polling (which, as some have said, doesn't scale).

Somewhat related to my earlier post on how HTTP can be used as an alternate to XMPP/Jabber in a publish/subscribe scenario without too much problem or angst.

MySQL and LAST_INSERT_ID()

We've recently migrated our software servers from a dedicated server environment (BlueGecko - really good people and service) to Amazon's EC2 'virtual compute cloud' environment.

The new system has a very nice performance monitoring capability based on Ganglia that gives us visibility into the performance of service requests as well as details on more fine-grained functions that take place within each request. We can now see not only the number of requests or functions but also the average duration and the time it took for 50%, 95% or 99% of the requests to complete in a five minute interval. This percentile breakdown gives a quick feel for how 'spiky' performance is and how common outliers are.

So far, things have gone very well but there was one function of the system that seemed to suddenly have terrible performance. I was able to quickly see which area of the code was involved, which pointed me to some SQL statements.

As our system adds new anonymous profiles, the code retrieves the unique number assigned by MySQL using the special query that MySQL provides:

SELECT LAST_INSERT_ID() AS user_id FROM oo_anon_user

However, it turns out that this SQL is actually incorrect - the extra "FROM oo_anon_user" caused the database to return every single record from the table back to the client, on every request that created a new record. This of course took some time.

The correct syntax to retrieve an auto-increment field from a MySQL table is

SELECT LAST_INSERT_ID() AS user_id

Remember to omit any FROM clause.

It's much much much faster now.

REST and pub/sub

It's unfortunate that technologists continue to propagate serious mistakes like "[...] its also clear that REST and its inherent polling mechanism isn't the best way of building a user notification system [...]"

REST is about state transfer - and event notifications are also state transfer.
As for HTTP, it isn't only "polling" - anyone that has posted a blog entry knows that. The 'client' can 'post' updates to the 'server' - exactly the same as event notifications via XMPP. The great thing about XMPP is the federated multi-hop capability with 'trust' built-in. Just like email, only with everyone using settings for very low latency delivery.

There have been multiple publish/subscribe over HTTP mechanism (comet, mod_pubsub, KnowNow, etc) over the years.

Twhirl Adds Identi.ca

I don't really follow the social/friend feed streaming application space, but this post about Twhirl from
ReadWriteWeb had a quote that caught my eye (emphasis mine):

However, for the regular user, always on social networking doesn't have to be a source of stress - it just means that when you go online, socializing with others just becomes part of the overall experience of being on the internet.

MoonBoy

We've been doing GeoCaching for a number of years, enjoying learning about new parks and finding Geocaches on our trips. Earlier this year we created our own Geocache near our house and placed a Travel Bug in it - MoonBoy. His goal was to view a space shuttle launch and to someday travel on a Space Shuttle.

Amazingly, he's made his first goal and was an observer to this weekend's launch of STS. This is totally cool! Here are the photos on the GeoCache site

Oh, the irony of shallow WSDL

I'm looking into the API for the Hi5 social network and unfortunately found some WSDL. They also have a REST API, but it's documentation appears auto-generated from WSDL that nobody actually filled in. Somewhat useless.

Normally I wouldn't post about WSDL, but I couldn't pass up the irony of the WSDL documentation for the authentication service API having an HTML form describing how to authenticate the user. If that's not irony, I don't know what is.

Ultimate Twitter revenue model - chatbots??

From ReadWrite Web

"Essentially, this would entail Twitter parsing over the Tweets of a given user, as well as the Tweets of the users he/she is following. Common keywords, themes, and phrases are then pulled from this data and associated with that user. As a result, highly-targeted ads can be displayed based on the user's network of content ("web design", for example). These simple text ads would look very similar to regular Tweets, but would be clearly marked as "Sponsored Content"."

I think chatbots haven't work for a reason - people want to chat not shop.

Reading RWW and other pundit blogs that describe "how the future will work" reminds of reading Popular Science as a kid and gazing in wonder at the flying cars and transparent house soon to be built.

Web scale pubsub

Looks like people are now starting to talk about a de-centralized, Web native pubsub system. Now is the time, if applications are going to avoid being based on a single company's service. Too bad pubsub.com and KnowNow aren't driving this.

CouchDB

I'd heard the CouchDB name but but didn't realize that it's an HTTP-accessed, Erlang-implemented, schema-free, distributed datatabase - and it supports Javascript for defining views. Time to dive in!

From Apache CouchDB: The CouchDB Project:

"Apache CouchDB is a distributed, fault-tolerant and schema-free document-oriented database accessible via a RESTful HTTP/JSON API. Among other features, it provides robust, incremental replication with bi-directional conflict detection and resolution, and is queryable and indexable using a table-oriented view engine with JavaScript acting as the default view definition language."

What programmers like

Quote of the day, found on the High Scalability blog while looking into Amazon's SimpleDB

"Programmers like problems they can solve with more programming."

Online services and availability

I've been working with the design and development of online services for quite some time and am very familiar with what 'high availability' means, but today I had the opportunity to learn first hand how the lack of reliability can directly impact people. This is a minor example, to be sure but as more 'software as a service' companies are built out more people will be affected in ways that they don't quite understand.

My daughter attends a Middle School that uses the online curriculum from Key Curriculum Press - I have no idea if the content is decent or not (but knowing the teachers at the school I'd bet it's good) and in order to access specific materials a login process has to take place and then access to some PDFs become available. This weekend, the login service was unavailable and so my daughter could not work on that part of her homework - very annoying.

The problem is likely something as simple as a server misconfiguration or a host being down (I can't imagine there's too much horsepower required for this sort of service), but as a 'customer' we are unable to do anything. There is no 24x7 tech support or online contact information. I could make a phone call, but that would only work between 8am and 5pm (PST), and I'm pretty sure kids do homework outside of those hours.

I've sent an email to their president and we'll see what kind of response that gets.

Update - I received a nice email from their president and she also sent a follow up email with more details. Very nice!

MyOpenID for Your Domain

This post from ReadWriteWeb about
MyOpenID for Your Domain is interesting. I've looked into OpenID several times in the past but it didn't have enough immediate value to get me to do anything about it. Now that more sites are 'supporting' OpenID and with the simple to use MyOpenID provider maybe I'll set something up. It would be interesting to see decentralized profiles and authentication, it seems a shame that everyone is getting locked into a few social networks for hosting and managing personal profiles and shared contact info.

Yaws and reverse proxy

It seems that there are a lot of situations where efficient connection handling in an HTTP reverse proxy is needed to help with scaling and keeping costs down, and my first guess was that Erlang would be the way to go.

So I just searched to see if there was a reverse proxy written in Erlang and sure enough Yaws has one.

Next, I need to find some folks that have actually used it and ask some questions. Especially for persistent connections (accepting a request but sending no response until there is data available, like in a message queue or pubsub scenario).

Of Liquidity, Competition and Platforms

This post by Bob Wyman "Of Liquidity, Competition and Platforms" reminds me why I subscribe to tech blogs - occasionally you get a deep thinker that clearly and concisely describes something that likely took them a long time to figure out.

"As many others have said, much of what people are building today is 'features' not products. As long as that is the case, raw economics is the real problem with the software business. Competition strengthens the platform builders while eviscerating the component builders... The rich get richer and the builders of innovative features must be satisfied with the 'personal rewards' of doing a job well unless they get lucky in the buy-out lottery game."

With the various cloud computing efforts well underway, and with each providing a different approach as a 'platform', it would be good for developers and entrepreneurs to think about which road to travel.

More REST and Erlang

Man, I wish I had the time to dig into these posts about Erlang, Yaws and RESTful services.

Cross domain web apps

This post about cross domain Web apps is spot on. The growth of the current Web was due to millions of silos - no data shared between sites. Over the past few years technology like iframes, ajax and other styles of 'mashups' have made it possible to create applications composed from cross-domain services. I believe this will drive the next major growth curve of the Web.

Elephant on a Bicycle

Here's an interesting blog on China from someone living there. The photos are both beautiful and intriguing.

Elephant on a Bicycle

An example of the writing (which I really like)

I'm now in Xining, Qinghai, in a youth hostel 15 floors up - which you think would offer remarkable views of the mountains that crowd the city on all sides. But it doesn't, this is China. 15 floors up generally just presents you with worse pollution, as it seems to procrastinate in mid-air limbo.

I'm in a quiet little room with two computers. Beside me sits a young monk of about 18, who has rather rashly decided to accessorise his austere crimson robes with a pink feather boa. I couldn't make that up. Furthermore, he is currently downloading provocative pictures of Nicole Kidman; certainly harder to find fault with this inevitable phase of teenage experimentation.

Xining city is a fascinating stopover because it offers a unique diversity. It is still predominantly Han Chinese, in both it's populace and it's blindfolded embrace of modernity - noisy, dirty, difficult to look at, pulsing
with energy. But alongside this, it contains a deeply religious aspect from the presence of size-able pockets of Islamic and Tibetan communities. In places there even exists a meditative quiet - a stillness - which amidst the commercial bustle and the perpetual foot-race of human progress can seem almost revolutionary. This is especially apparent in the areas surrounding the great mosque where I spent most of the day after befriending a young Koranic scholar named Farooq.

Party like it's 1994

You learn something new every day. Although sometimes it takes 5,110 days.
From Steve Vinoski's blog - here's
A Note on Distributed Computing from Sun in 1994.

We argue that objects that interact in a distributed system need to be dealt with in ways that are intrinsically different from objects that interact in a single address space. These differences are required because distributed systems require that the programmer be aware of latency, have a different model of memory access, and take into account issues of concurrency and partial failure.