MongoDB replica sets - high level overview

Here is a very very brief overview of MongoDB replica sets and a tip to enable read access to read-only replica slaves.
http://www.codypowell.com/taods/2011/08/a-cloud-hosting-architecture-for-mongodb.html

The full definition from the MongoDB site is here http://www.mongodb.org/display/DOCS/Replica+Sets

Non-blocking operations and deferred execution with node.js

If you write high volume server applications with high concurrency or low latency requirements you have probably heard about node.js This is a relatively easy to understand system that came out in 2009 and has some pretty amazing characteristics. An early presentation by the main author is here - http://s3.amazonaws.com/four.livejournal/20091117/jsconf.pdf


Node.js is an environment for writing Javascript based server applications with a big twist - all IO operations are non-blocking. This non-blocking aspect introduces a concurrency model that may be new to most developers but enables node.js applications to scale to a huge number of concurrent operations - it scales like crazy.

Using non-blocking operations means code that would normally wait for data from a disk file or from a network connection does not wait and waste CPU cycles - your code returns control to the runtime environment and will be called later when the data actually is available. This allows the runtime environment to execute some other code whose data is ready at the moment and gains efficiency by avoiding context switches. This also means there is a single thread accessing data and no synchronization or semaphores are needed to prevent corruption of data due to concurrent access, making your application even more efficient.

Although writing applications in Javascript makes node.js very approachable, the use of non-blocking operations isn't very common in most server applications and results in code that looks similar but is oddly different from what is familiar to most developers. For example, consider a simple program that reads data from a file and processes that data. In a typical procedural program the steps would be :

file = open("filname");
 read(file,buffer);
 close(file);
 do_something(buffer);

This pseudo-code example is easy to understand and probably familiar to most developers. The step-by-step sequence of operations is the way most languages work and how most application logic is described. However, in a non-blocking version the open() function returns immediately - even though the file is not yet open. This introduces some challenges.

file = open("filename");

 // the 'file' is not yet open! what to do?
 read(file,buffer);
 close(file);
 do_something(buffer);

If the open() function were a blocking operation, the runtime environment would defer execution of the remaining sequence of operations until the data was available and then pick up where it left off. In node.js the way that code after a non-blocking operation is paused and picked up later is through the use of callback functions. All the steps listed after using the open() function are bundled into a new function and that bundle of steps is passed as a parameter to the open() function itself. The open() function will return immediately and your code has the choice of doing some work unrelated to the data that is not yet available or simply returning control to the runtime environment by exiting the current function.
When the data for the opened file actually does become available your callback function is invoked by the runtime and your bundle of steps will then proceed.

open("filename",function (f) {
 read(f,buffer);
 close(f);
 do_something(buffer);
});

The parameters to the callback function are defined by the non-blocking operation. In node.js opening files uses a callback that provides an error object (in case opening the file fails) and a file descriptor that can be used to actually read data. In node.js most callback functions have an error object and a list of parameters with the desired data.

In the non-blocking example above you may have noticed the read(f,buffer) function call and guessed that this might be a non-blocking operation. This requires an additional callback function holding the remaining sequence of operations to execute once the data is read into a buffer.

open("filename",function (f) {
 read(f,buffer, function(err,count) {
  close(f);
  do_something(buffer);
 });
});

Some people feel this is a natural way to structure your code. Those people would be wrong.

Here is an actual node.js example of reading from a file

var fs=require('fs'),
 sys=require('sys');


 fs.open("sample.txt",'r',0666,function(err,fd) {
  fs.read(fd,10000,null,'utf8',function(err,str,count) {
   fs.close(fd);
   sys.puts(str);
  });
 });

Although this may appear a bit complex for such a simple task, and you can imagine what happens with more complex application logic, the benefit of this approach becomes more apparent when thinking about more interesting situations. For example, consider reading from two files and merging the contents. Normally a program would read one file, then read another file, then merge the results. The total time taken would be the sum of the time to read each file. With non-blocking operations, reading both files can be started at the same time and the total time taken would only be the longest time to read either of the two files.

Making the Web faster - SPDY

Those crafty people at Google are doing some cool work to "make the Web faster". The first I had heard of this initiative it turned out to be how to make "pages" faster - a decent thing, but fairly well known. But recently some folks over there have started to look at the actual underlying issues with the gears grinding out the Web - mainly networking latency. Trying to improve the network protocol of the Web is a tricky thing - lots of people (and egos) can get involved. Surprisingly their effort seems to be off to a good start and everybody is taking it at face value and being supporting and questioning things in a positive way.

One really cool thing mentioned in their whitepaper isn't a direct 'latency' thing - it's about 'server push'. If they can really make this happen a whole knew world of application development would open up.

To enable the server to initiate communications with the client and push data to the client whenever possible.

Real-time web, take 2

Bernard Lunn has a good post over on ReadWriteWeb putting the recent PubSubHubBub/RSSCloud news into context. Very funny that he calls KnowNow a "blow out", but I think he correctly identified their issue being a focus on the enterprise market (when that market had fairly established solutions).

Wish I hadn't been so busy over the past two years and could have worked on helping build PubSubHubBub-style technology.

Scalable, reliable key-value lookup service

Here's a great summary of most of the available key-value storage services from Richard Jones of last.FM - great stuff.

From what I can tell, Scalaris is only memory-resident at the moment and doesn’t persist data to disk. This makes it entirely impractical to actually run a service like Wikipedia on Scalaris for real - but it sounds like they tackled the hard problems first, and persisting to disk should be a walk in the park after you rolled your own version of Chord and made Paxos your bitch.

I'm leaning towards Voldemort, but need to look into this more.