Entries tagged ‘javascript’

While working on hosting Dojo within Node I arrived at a neat solution to isolate the Dojo code from Node itself: evaluate it in a new context. Here’s how it’s done.

var sandbox = {};
process.binding("evals").Script.runInNewContext('this["-eval-"] = function(code){ eval(code); };', sandbox);

runInNewContext evaluates the code in a separate JavaScript context, sandboxed within our sandbox object. We then define this["-eval-"] to call the eval method from the context. This exposes the -eval- method on the sandbox.

We can call it:

sandbox["-eval-"]('this.foo = function(){ return "bar"; };');
require("sys").puts(sandbox.foo());
// bar

Unfortunately declaring global variables inside the context won’t cause them to be set on the sandbox, so this doesn’t work:

sandbox["-eval-"]('baz = function(){ return "thud"; };');
require("sys").puts(sandbox.baz());
// TypeError: Object #<an Object> has no method 'baz'

Same for accessing foo as a global:

sandbox["-eval-"]('this.baz = foo');
// ReferenceError: foo is not defined

I’m not quite sure why this is the case, global variables are treated a bit differently within the context.

In our previous installment we discussed how to create a simple extension for Chrome, using content scripts to interact with web pages visited by the user. Content scripts can be somewhat limiting however, so this time we’ll look into how we can run our extension code at all times, not just when a page is loaded. Coincidentally this is also a nice introduction to the general approach taken by the Chrome team when implementing extensions.

As you might know, each tab in the Chrome browser runs in an isolated process. This is done both for stability and security but it also means that Chrome is very good at running many small “web browsers”. Extensions run in isolated processes as well. In fact, you could say that extensions run inside hidden tabs. To put it differently, an extension is just a web page you can’t see.

Let that sink in a bit. Chrome extensions are just web pages. They have HTML, they have the DOM, they have JavaScript. They have all the browser features you have in a normal website!

Chrome has quite a few more tricks up its sleeve than content scripts. One of them is background pages. A background page is one of those web pages you can’t see. This is how you specify one in the manifest.json file:

{
  "name": "Hello World",
  "version": "2.0",
  "background_page": "background.html"
}

background.html is the path to the background page in the extension directory. It can be called anything really, but by convention it’s called background.html. There can be only one background page per extension and it’ll exist as long as the browser is open (and the extension remains installed). You can even open the Web Inspector for a background page: simply click on its name in the Extensions overview.

Now, if the background page were really just another web page that happens to be hidden, it’d be rather useless. Web pages can’t talk to each other, so what could the background page possibly do!? This is where we get to the really great part about the Chrome extensions: there’s a special set of JavaScript APIs that can be used to give your extension way more power. These APIs are all inside the window.chrome object and are accessible from background pages and content scripts.

For example, the chrome.extension.getViews() method returns a list of window objects of the various tabs the extension has access to. Let’s try to recreate the Hello World example with a background page.

background.html:

<script>
chrome.extension.getViews().forEach(function(view){
  view.alert("Hello world");
});
</script>

Since the background page is an HTML document, we do need to wrap our code inside a <script> tag. Note that we can use Array.prototype.forEach (introduced in JavaScript 1.6) here. One of the niceties of Chrome extensions: more powerful JavaScript!

When you install this extension it will alert “Hello World” straight away, since this code runs when the background page loads. That’s nice, but not really what we had in mind. And, its alerted only once, so apparently the extension only has access to itself. chrome.extension.getViews() (see docs) doesn’t return all tabs, it only returns those that the extension is loaded into through a content script, plus its background pages. Since we don’t want to use a content script this time around, let’s see if we can get access to the tabs that are currently open.

As the background page is still isolated from the rest of the browser, it needs to talk through the Chrome APIs to access other tabs. For this to work though we need to request the appropriate permissions. To access the tabs, we need to request the tabs permission. Update the manifest file:

{
  "name": "Hello World",
  "version": "2.0",
  "background_page": "background.html",
  "permissions": ["tabs"]
}

We can get all tabs in a given window using chrome.tabs.getAllInWindow() (see docs). This is an asynchronous method, so we need to pass it a callback. Also, we pass null as the first argument to get the tabs in the current window:

chrome.tabs.getAllInWindow(null, function(tabs){});

You’ll find that many API methods in Chrome are asynchronous.

Here’s our updated background.html, which alerts “Hello World” for each open tab and includes the URL of each tabs current page for good measure:

<script>
chrome.tabs.getAllInWindow(null, function(tabs){
  tabs.forEach(function(tab){
    alert("Hello world\n" + tab.url);
  });
});
</script>

It’d be extra nice if we could run the alert() from within the tab itself. To do so we can use chrome.tabs.executeScript (see docs), which executes a script (either from a string or a file) inside a given tab:

<script>
chrome.tabs.getAllInWindow(null, function(tabs){
  tabs.forEach(function(tab){
    chrome.tabs.executeScript(tab.id, { code: "alert('Hello World')" });
  });
});
</script>

This, however, doesn’t work! It turns out that we don’t have permission to execute scripts inside the tab. If you open the Web Inspector for the background page, you’ll notice this:

Error during tabs.executeScript: Cannot access contents of url "http://supercollider.dk/blog". Extension manifest must request permission to access this host.

Remember how with the content script we had to specify which URLs it matched? We’ll do the same here, but add the URLs to the permissions instead:

{
  "name": "Hello World",
  "version": "2.0",
  "background_page": "background.html",
  "permissions": ["tabs", "http://*/*"]
}

Now when the alert comes up, Chrome focuses the tab the alert originated from. Success! (You might still see some error logs for https:// pages and the internal Chrome pages, these were not part of our permissions.)

This is all good, but it only works when the extension loads. How can we know when a tab loads? Well, there’s the chrome.tabs.onUpdated event (see docs) which fires when a tab starts loading and when the loading has completed:

<script>
chrome.tabs.onUpdated.addListener(function(tabId, changeInfo, tab){
  if(changeInfo.status == "loading"){
    chrome.tabs.executeScript(tabId, { code: "alert('Hello World')" });
  }
});
</script>

You can listen to Chrome API events by calling addListener on the event object. You pass a callback function, which is called with specific arguments depending on the event. As you can see this is not the typical DOM event model, it’s better geared to the API use cases.

In this case tabId is the identifier for the tab, changeInfo contains some information about the update, and tab is a reference to the tab object itself (just like the tabs returned by chrome.tabs.getAllInWindow()).

At this point we’ve reproduced the original Hello World example, but now through a background page. In our next installment we’ll look more closely at the security constraints on Chrome extensions.

Earlier this year Google’s Chrome browser landed official support for extensions. This is incredibly exciting, not only because extending browsers is cool, but because Chrome’s extensions are fully based on open web standards. And who knows about open web standards? Yup, web hackers.

In other words, if you can build a website, you can build a Chrome extension. They don’t consist of anything else but HTML, CSS, JavaScript, HTML5ish features such as localStorage and <canvas>, and some APIs (implemented in JavaScript) to interact with the browser itself. It’s much easier than developing extensions for Firefox, which has a complicated setup and configuration and the XUL language to work with Firefox’s user interface. (Admittedly, Mozilla is trying to rectify this through Jetpack, which I haven’t looked into closely, but is not included by default in Firefox.) Another advantage of Chrome over Firefox is that Chrome can reload extensions without restarting the browser, which is a fair bit nicer!

Let’s see about building a very simple extension that will alert “Hello world” for every page you load. This will help cover the basics of extension development. First, open up the Extensions overview, which is under the Tools menu in the toolbar. This page shows the extensions you’ve already installed. It also has a Developer mode, which you can select by clicking on the “Developer mode” link in the top right. This should show a few extra options: “Load unpacked extension…”, “Pack extension…” and “Update extensions now”.

We’ll focus on “Load unpacked extension…” first. If you click on it a folder selector dialog will come up. It only lets you select folders, which is great because a Chrome extension is nothing more than a folder with a few files! In the folder selector, go to your Documents folder and open it. Chrome will try to open the Documents folder as an extension, which, of course, is silly. Luckily it finds out quickly and gives this error message:

Could not load extension from ‘/Users/mark/Documents’. Manifest file is missing or unreadable.

This then brings us to what a Chrome extension really is: a manifest file inside a folder. More specifically, a file called manifest.json, that contains a valid JSON object describing your extension. At the very minimum, a manifest file looks like this:

{
  "name": "Hello World",
  "version": "1.0"
}

Create a hello_world directory somewhere on your computer, and place the above in manifest.json (inside that directory). Now in Chrome, click on “Load unpacked extension…” again and open the hello_world directory you just created. You’ll see how the extension is added to the list of installed extensions. Congrats, you just build your first Chrome extension!

Uhm, okay, technically the extension doesn’t do anything yet. There are a few different ways an extension can interact with the browser, but for now we’ll just look at content scripts. These allow you to run JavaScript for pages visited. If you’ve ever written a user script for Greasemonkey you’ll be familiar with content scripts, since they do pretty much the same thing. In fact, Aaron Boodman, who originally started Greasemonkey, is a main driving force behind Chrome extensions.

An extension can define multiple content scripts, each of which can load multiple JavaScript files. First create alert.js in the hello_world folder, it should contain:

alert("Hello world");

To load this for all web pages we use the following manifest:

{
  "name": "Hello World",
  "version": "1.0",
  "content_scripts": [
    {
      "matches": ["http://*/*"],
      "js": ["alert.js"]
    }
  ]
}

content_scripts is an array, each item being a declaration of the actual content script. matches lets you specify on which pages you want your content script to work, by providing a set of match patterns. js specifies which JavaScript files (included in your extension folder) to load if the page matches. (And yes, technically speaking this content script only works with http:// sites, not https://.)

In the Extensions overview, click the “Reload” link underneath the “Hello World” extension. This will reload the extension. Then go to any website and you should see a “Hello World” alert as it loads.

You can also specify CSS files that should be included on the matched pages. Create red.css (again in the hello_world folder) containing the following:

html, body { background: red !important; background-image: none !important; }

Change the manifest to:

{
  "name": "Hello World",
  "version": "1.0",
  "content_scripts": [
    {
      "matches": ["http://*/*"],
      "css": ["red.css"],
      "js": ["alert.js"]
    }
  ]
}

Save and reload the extension. Again go to any webpage and it should now have a red background color.

Admittedly this example extension is rather contrived. In future articles we’ll build more advanced extensions, look at different UI concepts supported by Chrome extensions, Chrome’s security model and how you can publish your extensions for others to use. In the meantime, have a look at the official documentation for Chrome extensions.

As a final note, I’ll be speaking about Chrome Extensions on the Scandinavian Web Developer Conference in Stockholm on June 2nd. Go check it out!

Computed style values for clip:rect() are different in WebKit compared to Gecko and Opera. Given this test code:

<div id="test" style="clip:rect(0, 50px, 50px, 0);width:100px;height:100px;position:absolute;background:red"></div>
<script>
window.onload = function(){
  alert(window.getComputedStyle(document.getElementById("test"), null).clip);
};
</script>

You get:

  • WebKit: rect(0px 50px 50px 0px)
  • Opera: rect(0px, 50px, 50px, 0px)
  • Gecko: rect(0px, 50px, 50px, 0px)

Nothing I couldn’t fix in the regular expression I was using to split the values, but an odd difference nonetheless.

Bootstrapping Dojo in Node

posted January 10th, 2010, 5 comments, tagged , ,

Being a Dojo hacker I’ve been wanting to use Dojo in combination with Node for a while now. So on a recent flight to London I added support for the Node environment to Dojo. I cleaned it up yesterday and – after figuring out Git – the code is now available on my Dojo fork.

Dojo is available for other environments than browsers, although the pre-compiled Dojo Base code you might have downloaded will only work in a browser. Other supported environments are Firefox extensions, the Rhino JavaScript engine and the Spidermonkey JavaScript engine. And now, Node.

Dojo is able to support multiple environments by dynamically bootstrapping itself for the given environment. In the non-compiled code, the dojo.js file detects its environment and loads the appropriate hostenv_*.js file. The biggest task of the various hostenv files is to fix the loader code to ensure dojo.require() works. Depending on the environment the file might also add some other features, for example the Rhino environment adds an implementation for setTimeout.

Right now I’m detecting the Node environment by seeing if the process object exists in the global scope. This object provides various utility methods to Node and is, as far as I know, unique to just the Node environment.

We briefly need to interrupt this explanation by discussing how we invoke Dojo within Node. As far as I can tell Dojo needs to exist in the global scope, and the only way to accomplish this is by loading the Dojo code when Node starts. Therefore you’d run node dojo.js to kick things off.

Based on this invocation we can use the path to dojo.js to calculate the root location of the Dojo files. I use process.ARGV[1].replace(/[^\/]+\.js$/, "") to do this. The primary reason for calculating the root is because the Dojo bootstrapping code uses the root location to load the host environment file. Technically I could have left it empty but that’s a bit silly.

Finally we use require() to load further bootstrapping code, the loader, the host environment and Dojo Base. As a special trick for the Node environment, dojo.js invokes dojo._loadInit() to signal that the environment is ready for use. In browsers this function would be invoked on DOM ready, but of course that’s not possible in Node.

The Node host environment file (hostenv_node.js) patches the loader so we can use dojo.require() in combination with Node’s module loading support. I’ve also added support for passing djConfig properties to Dojo. Simply pass the JSON object to the --djConfig argument: node dojo.js --djConfig='{isDebug:true}'.

Because we have to start Node with the dojo.js file we need a way to load additional JavaScript code into Node. This is accomplished by passing the -s or --script argument, which should point to a JavaScript file to be loaded into Node with require().

Currently the Node environment is largely untested, and it is quite probable that there are parts of Dojo that are depending on other features that haven’t yet been ported. I did try implementing the XMLHttpRequest object however, as a wrapper around Node’s HTTP libraries, to make it easier to execute HTTP requests through dojo.xhr().

Check out the code at Github. Some examples are also available. See the documentation on usage.

Crosscasting Dojo PubSub events

posted July 9th, 2009, no comments, tagged , , , ,

Using the localStorage object, the storage event and the postMessage() API, it is possible for browser windows to discover each other and to communicate with each other.

Crosscasting is a small Dojo package that publishes Dojo PubSub events to other browser windows that are running the Crosscasting code and are active within the same domain. Crosscasting of course stands for cross-window broadcasting.

Check out the example page, which of course you should open a couple different windows. Right now this only seems to work in the latest WebKit nightly builds.

The implementation is quite straight-forward. The first part is the Window Discovery Phase. An item is set in localStorage, which triggers the storage event. The Storage Event object has a source attribute which points to the Window object that set the item. (The storage implementation in Firefox 3.5 does not support this.) This is how a window informs other windows of its existence. The other windows subsequently use the postMessage() API to send a message to the original window. The Message Event also has a source attribute, pointing at the Window object that posted the message. At this point all windows know of each other’s existence. This phase is triggered by invoking supercollider.crosscast.init()

The second part is broadcasting Dojo PubSub events. Using supercollider.crosscast.subscribe("topic") you can register topics for broadcasting. The topic and any arguments are serialized to JSON and send to all other windows through postMessage(). The other window deserializes the message back to the PubSub event and publishes it locally.

// Initialize and start discovery
supercollider.crosscast.init();
// Subscribe to a topic for crosscasting
supercollider.crosscast.subscribe('crosscast/test');

// Subscribe to the topic
dojo.subscribe('crosscast/test', function(s) { alert(s) });
// Publish a PubSub event, both locally and to other windows
dojo.publish('crosscast/test', ['hello world']);

Again, have a look at the example. You can find the source at js-collider/crosscast.

P.S. I do not believe this is an original idea, but when I set about implementing this earlier today I could no longer find the site where I read this first. If you do know which site I’m talking about, let me know in the comments so I can give proper credit. Thanks!

Earlier this month, Christian Heilmann wrote about showing your Twitter data on any website, without your explicit permission. He’s posted a good overview of the matter on Ajaxian. His ‘hack’ relied on the user’s browser being logged in to the Twitter API, which allowed any website to call API methods.

In the work I’m currently doing for a London-based startup, providing a 3rd-party API and also providing widgets to be hosted on different sites is an important goal. But, as the Twitter example shows, there are security and privacy issues to be considered. In this post I’d like to focus on using OAuth to authenticate client-side widgets hosted on 3rd-party websites. I’m not at liberty to discuss the actual widgets we’d like to provide, so instead I’ll give a hypothetical example.

The HighScoreTracker Widget

Let’s imagine a web service named HighScoreTracker. They provide a widget that you can put on your own website, without requiring any server-side coding, which allows any website to send in high scores. This would allow users to track their high scores on various games on various websites. You run a gaming website named GamesForKicks and would like to participate in HighScoreTracker, so you place their widget on your site. After a visitor has finished playing the awesome game of RaceForBreaks, you want to automatically send their high score to HighScoreTracker.

When HighScoreTracker receives the high score, it needs to know two things:

  1. The game (website) that submitted the high score
  2. The user for whom the high score is submitted

This maps nicely to the OAuth use case. (For an introduction on OAuth, read the excellent Beginner’s Guide to OAuth). In OAuth speak, HighScoreTracker is the Service Provider. The widget placed on GamesForKicks is the Consumer and the user playing the game is the User.

With OAuth, the Consumer needs a token and a secret, which the Service Provider can use to verify the identity of the Consumer at a later point. Not only does the Consumer needs this token and secret, it needs to receive permission from the User as well, before it can act on her behalf.

As explained, the widget on GamesForKicks is the Consumer. Presumably you had to register with HighScoreTracker, which then generated the HTML code to display the widget on your site. For the widget to work as a proper Consumer however, it needs a consumer token and a secret. And that’s a problem: since the widget is embedded on GamesForKicks as an HTML snippet, both the token and the secret are public information. The secret is not a secret.

Verifying a Consumer without any secrets

If the secret is not a secret, that means anybody can act as if they’re GamesForKicks. Therefore a different way of verifying the Consumer’s identity is required. Luckily, an integral part of GamesForKicks’ identity is its web domain. If we can verify that the requests from the widget come from the domain we expect, we can verify the widget itself.

Here’s what we have so far:

  1. An OAuth Consumer: a widget running on a third-party website
  2. A consumer token, identifying the widget instance
  3. No consumer secret, since the widget is on a public page
  4. A known domain on which the widget is placed

The steps required by the OAuth protocol for a Consumer to be authorized to act on a Users behalf:

  1. Fetch a Request Token from the Service Provider
  2. Direct the User to the Service Provider for authentication, pass along the Request Token
  3. Once the Request Token is authorized, exchange it for an Access Token
  4. Use the Access Token to make requests to the Service Provider

We can validate the identity of the Consumer at steps 1 and 2. First of all, in order to fetch the Request Token, the widget loads a JavaScript file from the HighScoreTracker website:

<script src="http://highscoretracker.com/oauth/request-token.js?oauth_consumer_key=GamesForKicks&domain=gamesforkicks.com"></script>

(For brevity, I’m ignoring signed requests and other OAuth parameters)

HighScoreTracker verifies that the GamesForKicks Consumer is registered for gamesforkicks.com. It then returns a piece of JavaScript:

if(document.domain=='gamesforkicks.com') notify_request_token('123');

notify_request_token is a JavaScript function specified by the widget. It is invoked with the Request Token, but only if the widget is running on the expected domain.

Subsequently, the widget can open a popup window for the User to authorize the widget:

window.open('http://highscoretracker.com/oauth/authorize?oauth_consumer_key=GamesForKicks&token=123');

So far we haven’t seen anything that can’t be spoofed by an evildoer. For example, instead of fetching the Request Token using JavaScript, an evildoer could request it using a server side request and strip out the domain verification. Therefore we need to ensure that the Request Token was fetched using the User’s web browser.

We can ensure this by relying on the browser’s security model and cookies. First, if the Service Provider can ensure that there is no way to read the JavaScript file from a separate domain, we can be sure that if the Consumer has the Request Token, it either came from executing the JavaScript file, or from fetching it using a server side process. To make sure the Consumer executed the JavaScript file, the Service Provider can save a Verification Token in a browser cookie as the file is requested. When the authorization page is loaded, the Service Provider can verify that the specified Request Token corresponds with the Verification Token in the cookie. If this is the case, we can be sure that the only way the Consumer could have gotten hold of the Request Token is if the JavaScript executed normally, and therefore that the Consumer is running on the domain it is supposed to run on.

To summarize:

  1. The Consumer fetches the Request Token by loading a JavaScript file from the Service Provider
  2. This file is actually loaded through the web browser of the User
  3. The Service Provider generates the JavaScript file, which includes a domain verification, before passing along the Request Token to the Consumer
  4. It also saves a Verification Token in a browser cookie
  5. The value of the Verification Token is only known to the User’s browser, not to the Consumer
  6. The Consumer directs the User to an authorization page on the Service Provider
  7. The user’s browser loads this authorization page, and passes on the Verification Token in a cookie
  8. The Service Provider uses the Verification Token to verify the Request Token
  9. Because the only way to have a valid Request Token is to be on the correct domain, this verifies the Consumer identity

With the Consumer identity verified, the Service Provider can continue with the normal OAuth flow and authorize the Request Token. The Consumer can subsequently exchange its Request Token for an Access Token. Once the Consumer has the Access Token, it can be allowed access without further verification.

Essentially, the mechanism outlined here is a different verification algorithm on top of OAuth.

Preserving Access Tokens

One remaining problem is that OAuth Consumers typically preserve their Access Token for later use. This is problematic in our situation, because the identity of the Consumer is verified in the authorization phase. Instead, the Service Provider could provide a time-limited Access Token and provide for an automatic method of exchanging a Request Token for an Access Token if the User is logged in to the Service Provider and has previously authorized the Consumer. If the Request Token cannot be exchanged, the normal authorization process can continue.

In code, the widget fetches the Request Token:

<script src="http://highscoretracker.com/oauth/request-token.js?oauth_consumer_key=GamesForKicks&domain=gamesforkicks.com"></script>

When the browser fetches the JavaScript file, it also saves a Verification Token in a cookie.

The file is loaded, and the domain matches, so the widget now has a Request Token:

if(document.domain=='gamesforkicks.com') notify_request_token('987');

The widget attempts to exchange the Request Token for an Access Token right away:

<script src="http://highscoretracker.com/oauth/access-token.js?oauth_consumer_key=GamesForKicks&domain=gamesforkicks.com&token=987"></script>

The Service Provider verifies the Request Token using the Verification Token stored in the browser cookie. If the Request Token is verified, the User is already logged in to the Service Provider, and the Consumer has previously been authorized, it returns an Access Token:

if(document.domain=='gamesforkicks.com') notify_access_token('zyx');

If the User is not logged in, or the Consumer has not been authorized before, it tells the Consumer to follow the normal authorization route:

if(document.domain=='gamesforkicks.com') authorization_required();

Conclusion

In this post I outlined a technique for verifying the identity of an OAuth Consumer with a public key and secret, using standard browser security limitations. I’d love to discuss this approach further to see if this is a good idea in the first place, whether there are any vulnerabilities or weaknesses I’ve missed, and if this still jibes well with the OAuth approach.

Tip: Catching Exceptions in dojo.Deferred

posted January 16th, 2009, no comments, tagged , ,

dojo.Deferred would have been part two in my single part series on Method Chaining. It’s a pretty awesome way for setting up callback hierarchies for asynchronous programming:

function async() {
  var d = new dojo.Deferred();
  // Start something asynchronous, 
  // call d.callback(response) when done.
  return d;
}

async().addCallback(function(response) {
  alert('Ready!');
  return response;
});

I can add multiple callbacks to async() or even chain them to the result of adding a callback:

async().addCallback(function(response) {
  alert('Ready!');
  return response;
}).addCallback(function(response) {
  alert('Double ready!');
  return response;
});

Parallel to the callback hierarchy, there’s an ‘errback’ hierarchy which is invoked in case a callback throws an error. This makes it easy to handle errors within the callback chain. Unfortunately it also prevents the error from being logged in the browser’s error console.

Luckily, it isn’t hard to log these errors ourselves. Simply run the following before using any deferreds:

;(function() {
  var addCallback = dojo.Deferred.prototype.addCallback;
  dojo.Deferred.prototype.addCallback = function() {
    var cb = addCallback.apply(this, arguments);
    cb.addErrback(console.error);
    return cb;
  };
})();

This overwrites the original addCallback method to automatically register an ‘errback’, which logs to console.error. Then, it returns the original return value, so it doesn’t affect other code.

Happy Dojo’ing!

XStream is a cool library for serializing Java objects to XML files, and back again. Given that I prefer to work with JavaScript, rather than Java, I spent some time today trying to make XStream work with Rhino objects.

First, a quick example of XStream, from their two minute tutorial. We start with two small Java classes:

public class Person {
  private String firstname;
  private String lastname;
  private PhoneNumber phone;
  private PhoneNumber fax;
  // ... constructors and methods
}

public class PhoneNumber {
  private int code;
  private String number;
  // ... constructors and methods
}

Set up XStream:

XStream xstream = new XStream();
xstream.alias("person", Person.class);
xstream.alias("phonenumber", PhoneNumber.class);

And instantiate the classes, plus serialize to XML:

Person joe = new Person("Joe", "Walnes");
joe.setPhone(new PhoneNumber(123, "1234-456"));
joe.setFax(new PhoneNumber(123, "9999-999"));

String xml = xstream.toXML(joe);

And here’s the result:

<person>
  <firstname>Joe</firstname>
  <lastname>Walnes</lastname>
  <phone>
    <code>123</code>
    <number>1234-456</number>
  </phone>
  <fax>
    <code>123</code>
    <number>9999-999</number>
  </fax>
</person>

Good, readable XML, as a serialization from Java objects. Going back is as easy as:

Person newJoe = (Person)xstream.fromXML(xml);

XStreaming Rhino

Unfortunately, things aren’t as easy when you try to serialize a JavaScript object from within Rhino:

xstream = new Packages.com.thoughtworks.xstream.XStream();
xstream.toXML({foo: 'bar'});

This results in a rather long error stack trace, which boils down to:

org.mozilla.javascript.WrappedException: Wrapped com.thoughtworks.xstream.converters.ConversionException: Could not call org.mozilla.javascript.NativeObject.writeObject()
---- Debugging information ----
message             : Could not call org.mozilla.javascript.NativeObject.writeObject() 
cause-message       : null 
cause-exception     : java.lang.reflect.InvocationTargetException 
-------------------------------

The problem here is that Rhino objects can’t be directly serialized by Java classes. Luckily, XStream is quite flexible and allows you to write your own serializer. Or, in XStream terms, Converter. So, that’s what I did:

xstream = new Packages.com.thoughtworks.xstream.XStream();
converter = RhinoConverter.create();
xstream.registerConverter(converter);

xstream.toXML({foo: 'bar'});

This gives:

<org.mozilla.javascript.NativeObject>
  <object>
    <foo type="string">bar</foo>
  </object>
</org.mozilla.javascript.NativeObject>

Before we continue, I’ve put the code up on Github. Here’s an example file.

If we serialize the following object:

{
  'foo': 'bar',
  'answer': 42,
  'truth': true,
  'falsehood': false,
  'nothingness': null,
  'everything': undefined,
  'nested': {
    'birds': 'nest'
  },
  'feist': [1, 2, 3, 4],
  'now': new Date
}

We get:

<org.mozilla.javascript.NativeObject>
  <object>
    <foo type="string">bar</foo>
    <answer type="number">42.0</answer>
    <truth type="boolean">true</truth>
    <falsehood type="boolean">false</falsehood>
    <nothingness type="null"/>
    <everything type="undefined"/>
    <nested>
      <object>
        <birds type="string">nest</birds>
      </object>
    </nested>
    <feist>
      <array>
        <length type="number">4</length>
        <_0 type="number">1.0</_0>
        <_1 type="number">2.0</_1>
        <_2 type="number">3.0</_2>
        <_3 type="number">4.0</_3>
      </array>
    </feist>
    <now>
      <date stamp="2008-11-09T23:13:26.282+01:00"/>
    </now>
  </object>
</org.mozilla.javascript.NativeObject> 

What’s interesting here is that only the XML root element is org.mozilla.javascript.NativeObject. This root element is created by XStream, and is a serialization of the main object. From XStream’s point of view, this main object is a Java object, as modelled by Rhino. The converter itself, however, is written in JavaScript. All descendent elements are serializations of JavaScript objects. That’s why <nested> doesn’t contain a <org.mozilla.javascript.NativeObject>.

The constructors of the JavaScript objects are retained in the XML, in the form of <object>, <array> and <date> elements. Children of these elements are the object’s properties. Primitive data types are indicated by the property having a type attribute, and a raw value. <date> elements have an extra stamp attribute, which is the value of the original Date object in RFC 3339 format. Because XML property names may not start with a digit, the array indices are prefixed by an underscore. (And yes, underscores themselves are escaped by another underscore. _ is serialized to __.)

It’s possible to add custom constructors to the converter:

function Klass() {
  this.property = 'foobar';
}

Klass.prototype.print = function() {
  print(this.property);
};

converter.register('klass', Klass);

k = new Klass;
k.property = 'hello world';

xml = xstream.toXML(k);

Here I’ve created a class Klass, and registered it with the converter we created earlier. This does run against XStream’s alias() method, which you saw earlier in the XStream example. This is because XStream deals with Java objects, and it’s our JavaScript converter that deals with JavaScript objects.

The XML serialization is as follows:

<org.mozilla.javascript.NativeObject>
  <klass>
    <property type="string">hello world</property>
  </klass>
</org.mozilla.javascript.NativeObject>

We can now deserialize the XML, and run Klass.prototype.print(), to see if deserialization worked:

k1 = xstream.fromXML(xml);
k1.print(); // 'hello world'

Marvelous!

Look ma, no constructor!

Here’s the code which creates an instance for deserialized objects:

    object = {};
    object.__proto__ = constructor.prototype;

This is where the JavaScript truly shines. Rather than doing object = new constructor(), where constructor is a previously derived reference to a constructor method, we simply create an object and point it to the prototype of the constructor, without invoking the constructor method!

At this point it may help to explain how JavaScript finds a property on an object. If we have object o, and we try to resolve o.foo, JavaScript first checks if o has a property foo. If it does, it just returns the value of this property. If it does not, JavaScript checks if the object referenced by o.__proto__ has the property. If it does, the value is returned, if it doesn’t, o.__proto__.__proto__ is checked. This continues until either foo has been found, or we run out of __proto__ references. In JavaScript, __proto__ tends to point to a prototype object of a constructor method. Object.prototype for example. The __proto__ references form a prototype chain. Object.prototype is usually the last object in this chain.

As it turns out, we could program, say, new MyClass(), ourselves, like this:

function MyClass() {
  this.foo = 'bar';
};

MyClass.prototype.baz = function() {
  return 'thud';
};

var auto = new MyClass();

auto.foo; // 'bar'
auto.baz(); // 'thud'

var manual = {};
manual.__proto__ = MyClass.prototype;
MyClass.apply(manual, []);

manual.foo; // 'bar'
manual.baz(); // 'thud'

Strings

When working with Rhino, the Java versus JavaScript object issue comes up more often. In the converter code, you’ll often see something like '' + reader.getAttribute('type'). For example:

switch ('' + reader.getAttribute('type')) {
  case 'string':
    value = '' + reader.getValue();
    break;

The problem is that reader is a Java object, and reader.getAttribute('type') returns a Java string. For some reason, Java strings don’t match JavaScript strings in switch() statements, although they often do otherwise:

javaString = new Packages.java.lang.String('string');

javaString == 'string'; // true

switch(javaString) {
  case 'string':
    print('Match');
    break;
  default:
    print('No match');
}

// 'No match'

Something to watch out for!

XStream & Rhino

Back to XStream. What I’ve built now is a basic implementation, which does not support duplicate or circular references. Neither does it serialize JavaScript functions, and I haven’t really tried serializing Java objects referenced by a JavaScript object. But, it’s a start! Let me know if you use it, find faults, or want to commit a fix.

Method Chaining – Part One

posted September 2nd, 2008, one comment, tagged , ,

As suggested by Johan Bouveng, the first real post here is about Method Chaining. This is the technique jQuery uses for its magic.

To take an example from the jQuery home page:

$("p.neat").addClass("ohmy").show("slow");

This takes all <p> elements with a class name of neat, adds the class ohmy and then slowly animates the appearance of the elements. Equivalent code without chaining could look like this:

var neatos = $("p.neat");
for (var i = 0; i < neatos.length; i++) {
  addClass(neatos[i], "ohmy");
  show(neatos[i], "slow");
}
neatos = null;

In this example a few advantages of chaining become clear:

  • Clear, concise code: this makes it easier for fellow programmers to spot the important bits, such as adding a class name and showing the elements.
  • No variables used: the actions on the paragraphs are invoked on the result set of the p.neat selector
  • No worries about memory leaks, because there are no references to DOM elements

Disadvantages include more magical code, that quite possibly runs slower that non-chained code. For example, whereas the non-chained code uses one iteration over all paragraphs, the chained code iterates twice. Once for addClass() and once for show(). That said, this slight performance loss usually doesn’t hurt, and if it does, you can always remove the chains.

How does method chaining work? The general principle is that each function call returns an object on which you can call other functions. Many jQuery functions return the object on which they are called themselves:

var neatos = $("p.neat");
neatos === neatos.addClass("ohmy"); // true

Others, like filter(), return a different object, which supports the same methods.

Chaining Ourselves

Let’s play with some method chaining ourselves:

var chain = { // Create a new object
  set: function(name, value) {
    this[name] = value; // Do stuff here, in this case, set a property
    return this; // Return a reference to the object itself. 
  },

  get: function(name) {
    return this[name]; // Return the value
  }
};

This creates a chain object, which has a get() and a set() method. The set() method lets you set properties on the object, and returns the chain object. get() returns a value.

Now we can run the following:

chain.set("foo", "bar").get("foo"); // "bar"
chain.get("foo"); // "bar"

chain.set("baz", "thud").set("hello", "world").get("hello"); // "world"
chain.get("foo"); // "bar"
chain.get("baz"); // "thud"
chain.get("hello"); // "world"

We can’t run this:

chain.get("foo").get("baz"); // chain.get("foo").get is not a function

chain.get("foo") returns the value, which is not the chain object, and does not have the get() method.

Chaining with Class

Here we have only one chain object. Let’s create a class, so we can create several chainable objects:

function Chain() {};

Chain.prototype.set = function(name, value) {
  this[name] = value;
  return this;
};

Chain.prototype.get = function(name) {
  return this[name];
};

Now we can do this:

var chain = new Chain();
chain.set("foo", "bar").get("foo"); // "bar"

Let’s add a filter() method, which creates a new Chain object:

Chain.prototype.filter = function() {
  var result = new Chain();
  for (var i = 0; i < arguments.length; i++) {
    result.set(arguments[i], this.get(arguments[i]));
  }
  return result;
};

For example:

var chain = new Chain();
var filtered = chain.set("foo", "bar").set("baz", "thud").set("hello", "world").filter("foo", "baz");
filtered.get("foo"); // "bar"
filtered.get("baz"); // "thud"
filtered.get("hello"); // undefined

Conclusion

This concludes Part One of Method Chaining. We’ve seen why method chaining is so useful, and how it works. In Part Two, we’ll look at dojo.Deferred, which uses Method Chaining in order to, well, chain methods!