One of the kinds of art I really enjoy is art that shows off the draughtsmanship of cartographers, especially from the early 1900s or earlier. We have a small collection of framed old maps from places we most closely associate ourselves with: London, Derbyshire, Yorkshire, Normandy, Colorado. Road maps, railway maps, topographic maps. I find them utterly fascinating, tracing in my mind’s eye how the countryside has changed or the urban expanses filled out, squinting to read the names.

One of the examples I have is a page from The Gentleman’s Magazine from 1766. This was a monthly periodical that was published from 1731 to 1922 (the first such to use the term “magazine” in fact) and printed a summary of news, information, letters, and commentary, mostly about what was happening in Britain. Samuel Johnson, he of the astounding A Dictionary of the English Language, worked as a writer for the magazine for a while.

A 20-year stretch of issues of the magazine – from 1731 to 1750 – has been scanned and published online as part of the ILEJ project. They are engrossing to read, but can be opaque in places, what with all the long S’s and ligatures and abbreviations. Here’s a flavor:

The inhabitants of Westminster petitioned to the H. of C. [House of Commons] for the free fish market, like that of Billingsgate, in order to encourage fishermen, and reduce the price of fish ; and a bill was order’d accordingly.

Four petitions from the high sheriff, grand jury, justices of the peace, and gentlemen of Suffolk, met at the assizes and quarter sessions, were presented to the House, for an act to hold the former assizes at Ipswich ; as it is very unsafe, as well as expensive, to carry Smugglers and other prisoners as far as Bury ; to which place many gentlemen, and others, who are obliged to attend on business, have fifty miles to go ; and therefore, as Ipswich is even way more convenient, being nearer the middle of the county, etc. leave was desired to bring in a bill accordingly ; and a motion being made for that purpose, it passed in the negative.

Phew. Thank heavens the printer’s long S disappeared, it’s way too much like a lowercase F: fiſh, aſſizes, buſineſs, and so on, so forth. 

Anyway, in one of the issues for 1766 this map appeared:

(I really would click to expand it. The full size is some 1600 × 2400 pixels, 3Mb, so be warned.)

It’s a lovely map, despite the visible ink shadow from the other side in the lower half, and shows a good part of Southwark on the South Bank. It depicts some possible new roads to be built to connect the new (it was still being built) Blackfriars Bridge in the north to the main routes to Surrey and Kent in the south. (The bridge opened in 1769.) If you look carefully at the north end of the bridge you can see where the River Fleet empties into the Thames. Presumably there was an article that went along with it to describe the choices, although I don’t have it.

The map is full of fields and gardens, with Lambeth Palace and Westimster Bridge just visible. The closer to the river you get the more built-up the area becomes. There are some lovely names of places and streets and businesses, redolent of the times: Johnny Groots; the Dog and Duck, clearly marked at the junction of Lambeth Road and the Road to Clapham, Melancholy Walk, Dr. James’s Laboratory (I wonder what went on there?), Watson Calico Printer, Hay Market with a little icon of a windmill by it. The river has the various quays and landing stages clearly marked: Arundel Sta. ("Sta." is short for stairs), Surrey Sta., Essex Sta., Morris’s Causeway, Paul’s Wharf, and the like.

After a while of looking at it, you want to see what the current place looks like, how things have changed. Did any of those roads get built and survive to this day?

My plan was to superimpose a current map of the area over this one, so that the old map bleeds through. Not entirely successful, I’m afraid:

The main reason is that the current map is dense with streets. Southwark has changed incredibly in the past 250 years. It is extremely built up when compared to the old days, and in the building frenzy of the 19th and 20th centuries, entire streets were thrown away to be replaced by others. Some new streets took on an old name, even though they were in different places (look for Maid Lane, which became Maiden Lane). And some streets are still there: Red Cross Street had a slight change to Redcross Way, but Dead Man’s Place (where it ended) is now Park St. Part of the building changes were due to the construction of the railways in early Victorian times. Whole swathes of Southwark are now under rail: there’s Waterloo Station to the west and London Bridge station to the east. The position of the old Blackfryars Bridge (it only lasted one hundred years) is now where Blackfriars Rail Bridge is.

The other incredible thing is the direct result of Bazalgette’s remarkable plans to construct a modern sewer system one hundred years after this map was published. If you look at the present course of the Thames compared with the old map, you can imagine the monumental work that went into building the Embankment on the north side in Victorian times. He narrowed the river quite a bit with this construction, and some of those stairs I mentioned became streets instead (I went to King’s College, which is just south of Somerset Garden on the old map, and to one side of it is Surrey St, roughly where where Surrey Stairs are depicted).

The other fascinating part of the map for me (because I used to live in Camberwell to the south and hence travelled through it a lot) is the Elephant and Castle. On the old map it’s already pretty built up with several houses marked, although at that time, the name “Elephant and Castle” was an informal name for the village – it was the coaching inn there – the official one being Newington. It amuses me a bit to think that the Metropolitan Tabernacle is built on the site of the Fishmongers’ Alms Houses. I know, I know, I’m easily pleased.

Next time, I’ll briefly talk about how I photographed the map: it’s still in its frame.

Now playing:
Level 42 – Running in the Family -Dave O’ Remix
(from Running in the Family (Platinum Edition))

In the previous two installments (one, two) we explored the use of callbacks through creating a mapp function for arrays (so called because the latest JavaScripts have a native map method already), and through creating a mapAsync function where the work is done asynchronously rather than serially. The reason for this was so that we could avoid triggering the browser’s “script running a long time” warning and, also, more importantly, provide the user with a responsive UI.

photo © 2007 sputnik | more info (via: Wylio)In this final part I just want to tidy up the mapAsync method. When I left it last time,the function would process each element using a delay. This is, in all probability, way too much work and way too slow. We could potentially do more work each cycle without inconveniencing the user and thereby reduce the number of delays we have to go through. But how much work? How long do we have? Jakob Nielsen says that

“0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.”

OK, then, 100 milliseconds it is. Actually, thinking about it, that leaves no time for maneuver and so I’ll go for half that for the mapAsync method.

So, what we’d like to do is process as many elements as we can in 50 milliseconds (instead of just processing the one) and then recurse using a delay for the next batch. We therefore need the ability to time our processing.

JavaScript comes with the Date class. If you new up a Date, what you get is a variable containing the current date/time, much as DateTime.Now does in C# and .NET. What we’ll do is get the start time and then start processing as many elements as possible. For each element we’ll process it, then get the current time. If the current time minus the start time is less than 50ms, round the loop we go again. Unfortunately, we can’t just subtract one Date from another, the minus operator only works for numbers. However all is not lost: if we convert a Date to a number, we get the number of milliseconds from some base Date. And to convert? Just use the unary plus operator. Here’s an example:

var start = +new Date();
do {
  // work
} while ((+new Date() - start) < 50);

We get the current date/time as the number of milliseconds by newing up a Date and then converting it to a number by use of the unary plus. We then enter a do..while loop, and continue round doing work while the current date/time minus the start time is less than 50. Pretty easy, no?

Incorporating it into the current version of mapAsync is fairly mundane:

Array.prototype.mapAsync = function (process, done, context) {
  var i = 0,
      result = [],
      last = this.length,
      self = this,
      processAsync = function () {
        var start = +new Date();
        do {
          if (i in self) {
            result[i] = process.call(context, self[i], i);
          }
          i++;
        } while ((i < last) && (+new Date() - start) < 50);
        console.log("*"); // just for showing progress
        if (i === last) {
          done.call(context, result);
        }
        else {
          processAsync.delay(10);
        }
      };
  processAsync();
};

Of course, there’s a bit more to that inner loop condition since we also want to break out if we manage to process the final element. I’ve thrown in a logging statement after the loop for now, just to show that we’re nicely chunking the work and still performing several delayed calls to processAsync. We’ll delete it when we’ve shown it working.

Of course, my little example of an 8-element array just isn’t going to cut the mustard with this new code, so let’s go for broke and use a 100,000 element array:

var myArray = [];
for (var i = 0; i < 100000; i++) {
  myArray[i] = i;
}

myArray.mapAsync(function (element, index) {
  return "<" + index.toString() + ": " + element.toString() + ">";
}, function (a) {
  console.log("done");
  delete myArray;
});

All the completion callback does this time is to log “done” (printing the array will overload the console – I know because I did it) and delete the original array – it is fairly big after all. When I run this code in Firebug, I get this:

*
undefined
*
*
*
*
*
*
*
*
*
done

Again, the “undefined” at the top is Firebug printing the return value of calling mapAsync. As you can see, it chunked the 100,000 element array into 10 delayed calls, roughly 10,000 elements per call. Or, if you like, I managed to process 10,000 elements in 50 milliseconds, which ain’t too shabby.

That’s it for this series on callbacks. I hope it proved useful. Remember: if you want to use this mapArray method (I use an MIT license, so go for it), do take out the logging statement in the middle.

(This JavaScript callback series: part I, part II, part III.)

Last time, we wrote a map method for arrays (which I had to call mapp, so that we didn’t clash with the native version present in many browsers). To use the map method, you have to provide a callback function that would be called for every element in the array.

The first change I want to make today is to allow for missing elements in the array. In that case, the callback shouldn’t be called. Here’s the obvious way of implementing this change:

Array.prototype.mapp = function (process, context) {
  var i,
      result = [],
      last = this.length;
  for (i = 0; i < last; i++) {
    if (this[i]) {
      result[i] = process.call(context, this[i], i);
    }
  }
  return result;
};

var myArray = [2, 3, 5, 7, 11, 13];
myArray[7] = 19; // missing out myArray[6]
console.log(myArray); 

var newArray = myArray.mapp(function (element, index) {
  return "<" + index.toString() + ": " + element.toString() + ">";
});
console.log(newArray);

Since the seventh element is missing the expression this[6] will resolve as false and the if block won’t be executed. Unfortunately there are five other values that an existing element could have that will also evaluate as false: false, 0, NaN, the empty string, and null, all of which might appear in a real array. So this first cut just won’t work.

Another way is to expressly compare to undefined, like this:

    if (this[i] !== undefined) {
      result[i] = process.call(context, this[i], i);
    }

This works with the usual caveat that someone could have set undefined to some actual value. In which case we could use (void 0) instead (the void operator returns undefined for any operand), or we could enter (typeof this[i] !== "undefined") as an alternative.

Yuk, how about some proper JavaScript idiom? We’ll use the in keyword, it’s not just for the for..in statement:

    if (i in this) {
      result[i] = process.call(context, this[i], i);
    }

There. Much more legible.

Now we’ve got that sorted out, suppose that the array we’re mapping is large or that the processing for each element of the array is lengthy. We could be in danger of triggering the browser’s “script is taking too long” warning if we just blithely used the mapp method no matter what. What should we do?

photo © 2005 Jason | more info (via: Wylio)Let’s explore how we can split up the work done by the mapp method. First, though, it behooves us to understand why the browser might put up a warning dialog saying a script is taking too long. In essence, all of the JavaScript code on a page executes in one thread. Indeed JavaScript doesn’t have any way of spinning off other threads to do work. Since the one and only thread is also the UI thread, it means that a long-running piece of code would freeze the entire UI of the page. Not a good experience at all, which is why the browsers have a monitor to check that events are still flowing through the message pump. If a piece of code takes too long, the events are no longer being processed and after a given length of time, the browser interrupts the interpreter and puts up the warning dialog.

So, given all that, how can we split up our code so that the message pump still gets processing time? The answer is to use setTimeout. What this function does is to set a function executing after a certain length of time has passed. You pass both the function to execute (it’s a callback, of course) and the time to wait to setTimeout. What happens under the hood is that these requests are queued and some process pushes the callback onto the message loop after the timeout period has expired. At which point, of course, the function executes. (The same kind of thing happens with AJAX calls: when the AJAX call returns, it pushes the callback onto the message loop to get executed.)

What we would like to do is to split up our processing into “chunks”, each of which won’t take long to execute at all. After each chunk completes it queues up the next chunk to execute using setTimeout. However, the timeout used is going to be very short; not a large time like a second, but in the order of a few milliseconds at most. Essentially we are executing the chunks asynchronously rather than sequentially or synchronously. In between each there’s a bit of breathing room for the message loop to do other stuff.

Here’s a new function that will help us.

Function.prototype.delay = function () {
  setTimeout(this, 10);
};

It’s defined as a method on the function prototype, so it’s available to all functions. All it does is to delay the execution of the function it’s called on by 10 milliseconds. Here’s a silly example of it in action:

var o = {
  count: 10,
  tick: function () {
    if (o.count--) {
      console.log("*");
      o.tick.delay();
    }
  }
};

o.tick();

What we have here is an object with a count property initially set to 10, and a method called tick. This method outputs an asterisk to the console and then it calls itself using the delay method we just wrote, delaying for 10 milliseconds. It does this 10 times by decrementing count down to zero. If you run this, you’ll get 10 asterisks one after the other really quickly. Nice.

Note though that the initial call to o.tick returns immediately. If you ran it in Firebug you would actually get this result:

*
undefined
*
*
*
*
*
*
*
*
*

What’s that “undefined” doing in there? It’s Firebug telling us that the initial call to o.tick has completed and it returned undefined. Then we get the asterisks from the setTimeout followed by setTimeout, followed by setTimeout, etc, etc, each one firing off the other. The point I want to get across is that the initial function call completes before the delays fire. Delaying like this means that we have to be careful and not assume that all of the work we’re trying to do is completed straightaway. If we are concerned about knowing when a chained asynchronous process like this completes, we shall have to provide a completion callback so it can be fired when all’s done.

Back to the next version of our mapp function. We’ll create a new map function, call it mapAsync, that will process the array in chunks. To make it easy to begin with, we’ll make each chunk big enough to process a single element. First of all, we shall have to add a new parameter: the completion callback, as I mentioned above (it would be nice to know when the function had mapped the entire array). Then we shall have to define an internal function that will be the equivalent of our tick method above, and we shall have to make sure it knows how far we’ve got into the array.

Enter a closure. Of course. I’m sure you were expecting it. Here’s the code:

Array.prototype.mapAsync = function (process, done, context) {
  var i = 0,
      result = [],
      last = this.length,
      self = this,
      processAsync = function () {
        if (i in self) {
          result[i] = process.call(context, self[i], i);
        }
        if (++i === last) {
          done.call(context, result);
        }
        else {
          processAsync.delay();
        }
      };
  processAsync();
};

Let’s take this slowly. First of all we declare the same three locals as before. Then I save the value of the this variable, that is, the array this method is acting on. (I like using the name self for this purpose.) Then I’ve declared a function and it is this function that’s going to be called through our delay mechanism. If you notice at the end of the mapAsync method, I kick it all off by calling processAsync() for the first time.

I want to stop here and ask you to consider that call. It’s known as a function invocation. I’m not calling processAsync on an object (there’s none to be seen), in which case it would have been a method invocation call. Since there’s no object on which it’s called, JavaScript will call it using the global object. Inside the function the this variable will be bound to the global object. And that’s why, when you now look at the implementation of processAsync, you can see why we need to save the value of the this variable from the outer function – we would have no other way to get at the array.

So, let’s take a look at that inner function. First of all it processes the current element, determined by the captured value of i. Yes, we’ve created a closure with the mapAsync function and all its local variables have been captured and are available to the inner processAsync function. Now we increment the counter and if we’ve reached the end of the original array, we can call the done callback, setting up the right context (that is, the this variable for done), and passing the resulting, mapped array. If we haven’t reached the end of the array, we call processArray again, but delaying it slightly.

Here’s the code that’ll test it:

var myArray = [2, 3, 5, 7, 11, 13];
myArray[7] = 19; // missing out myArray[6]
console.log(myArray);

myArray.mapAsync(function (element, index) {
  return "<" + index.toString() + ": " + element.toString() + ">";
}, function (a) {
  console.log(a);
});

Notice that we now have two anonymous functions acting as callbacks: the first will process each element and the second will log the value of the mapped array passed in, once all the elements of the array have been processed. In Firebug, I get this:

[2, 3, 5, 7, 11, 13, undefined, 19]
undefined
["<0: 2>", "<1: 3>", "<2: 5>", "<3: 7>", "<4: 11>", "<5: 13>", undefined, "<7: 19>"]

Again notice that the result of the call to mapAsync (that is, “undefined”) is logged before the mapped array is logged.

If you think about it, this mapAsync code is showing the use of many callbacks: there’s process, done, and there’s processAsync, which is passed as a callback to the setTimeout function. Yep, callbacks in JavaScript are used everywhere. Get used to them and the use of anonymous functions to define them.

Next time we’ll work on refining the chunks of code we execute at one time. Processing every element through a delay is a little too granular.

(This JavaScript callback series: part I, part II, part III.)

Now playing:
Erasure - Chorus
(from Pop!: The First 20 Hits)

As I’ve said pretty much from the very start of this series: functions are objects. You can pass them around in variables, pass them into functions as parameters, return them from functions, the whole nine yards. When you pass a function into another where it will be called, it’s generally known as a callback.

photo © 2007 Nonie | more info (via: Wylio)(Note: as usual I’m using Firebug in Firefox to run these examples. All browsers have similar development tools built-in or that are available as a download.)

Here’s a simple example. There’s nothing too complicated going on here: all we’re doing is passing a callback into the function that’ll get called once the function’s real work is done.

var printDone = function () {
  console.log("done");
};

var doSomeWork = function (whenDone) {
  // do some work here
  whenDone();
};

doSomeWork(printDone);

So, we’re declaring a function doSomeWork that takes a callback called whenDone as parameter. Once the function has completed, it calls the callback. In the statement at the end, I’m calling doSomeWork passing in the printDone function.

Now, I totally agree that this is a somewhat daft example. After all I could have just added a call to printDone after the call to doSomeWork, but at least it shows the basics. Many times, if not most times, developers won’t formally give the callback function a name like I just did, but instead use an anonymous function declared right there in the call.

var doSomeWork = function (whenDone) {
  // do some work here
  whenDone();
};

doSomeWork(function () {
  console.log("done");
});

Exactly the same effect, but this time we declare the function anonymously as a function literal. You should get used to this style: it’s used universally in all kinds of code.

Now that we’re used to how to declare and use callbacks, let’s up the ante a little. We’re going to write a map method for arrays. A map is a function that applies some kind of operation to each element of an array and returns the resulting values as another array. First of all, let’s write some code that’s specific but works, and then refactor it to make it more general. We’ll start off by writing a function that doubles every element in an array (which we’ll assume to contain just numbers) and returns the resulting array.

var doubler = function (a) {
  var i;
  var result = [];
  for (i = 0; i < a.length; i++) {
    result[i] = 2 * a[i];
  }
  return result;
};

var myArray = [2, 3, 5, 7, 11, 13];
console.log(myArray);

var newArray = doubler(myArray);
console.log(newArray);

Now we’ll refactor it into a method on all arrays. (Note: the JavaScript version in Firefox 4 already has a map method defined on arrays, so I’m calling mine —unimaginatively, perhaps – mapp().)

Array.prototype.mapp = function () {
  var i;
  var result = [];
  for (i = 0; i < this.length; i++) {
    result[i] = 2 * this[i];
  }
  return result;
};

var newArray = myArray.mapp();
console.log(newArray);

Notice that the call to mapp() uses the method pattern of function invocation, and therefore the this variable inside the method points to the array it was called on.

Now, for a generic map method, this is still very specific to doubling elements. But consider the code: we have the stuff that iterates through the array and we have the statement that gets called for each element. It’s this latter part we want to genericize. So, create a special local (nested) function that does the work:

Array.prototype.mapp = function () {
  var i;
  var result = [];

  var process = function (element) {
    return 2 * element;
  };

  for (i = 0; i < this.length; i++) {
    result[i] = process(this[i]);
  }
  return result;
};

And now we see that we can promote this local function into a parameter of the outer function: that is, a callback. And it gives us the form of that callback too: it takes one parameter and returns a new value.

Array.prototype.mapp = function (process) {
  var i;
  var result = [];
  for (i = 0; i < this.length; i++) {
    result[i] = process(this[i]);
  }
  return result;
};

var newArray = myArray.mapp(function (element) {
  return 2 * element;
});
console.log(newArray);

With this last step, the introduction of the callback, the mapp method has become general and we can use it for other transformations or mappings. Supply a callback that does the work, and invoke the mapp method.

var newArray = myArray.mapp(function (element) {
  return "<" + element.toString() + ">";
});
console.log(newArray);

Of course, now that we have a generic mapp() method we can add sanity checks, start to optimize it, or maybe alter the signature of the callback function we accept. For now, let’s make the callback function not only get called with the element in question, but also its index. I also want to call it in such a way that we can pass in the value that the this variable inside the callback should point to.

Array.prototype.mapp = function (process, context) {
  var i, 
      result = [],
      last = this.length;
  for (i = 0; i < last; i++) {
    result[i] = process.call(context, this[i], i);
  }
  return result;
};

var newArray = myArray.mapp(function (element, index) {
  return "<" + index.toString() + ": " + element.toString() + ">";
});
console.log(newArray);

A quick note on the code is in order. I’m calling mapp() without a second parameter. JavaScript will then set the context parameter to undefined. If context is null/undefined, it will default to the global object in the call to call(). (Recall the call method calls the function and sets its this variable to the first parameter.) Also I’m moving the repeated access to the length property out of the loop and saved its value to a local variable: it’s slightly faster.

Next time, we’ll make a slightly different version of the mapp() method to continue our exploration of callbacks.

(This JavaScript callback series: part I, part II, part III.)