Intro to IndexedDB

Tuesday, December 14, 2010

Yesterday at the Silicon Valley GTUG meetup, I gave a presentation introducing the IndexedDB API. I’ve thrown the slides on Slideshare, but the transcription there is absolutely miserable. I’ll reproduce it here in a readable format, and add a few notes where appropriate.

Video

Embedded Slides

Slide Transcript

IndexedDB: Mike West, @mikewest, mkwst@google.com, SV GTUG, 2010.12.14
Beta: The IndexedDB API is incredibly beta. It’s only implemented in Firefox 4 and Chrome dev channel, so it’s not anything that can be used for production projects in the near future. Microsoft and Opera are contributing to the spec, however, and Google is working on pushing the code upstream to Webkit itself, so this looks like something that will be more and more relevant going forward.

Since the spec’s not finished, and everything’s in dev mode, this is a great time to examine the API, and experiment. We need to play around with this code, and feed our experience back into the standards bodies and browser vendors: that’s the best way to ensure that things work the way we want them to when everything’s solidified.
Offline: One of the most exciting recent realizations in web development is that the offline bits of the HTML5 suite of specifications are really ready for widespread use. It’s possible to store arbitrary amounts of information on a user’s computer without resorting to opaque hacks like Flash storage, while at the same time making that information available in useful ways to your program’s code. This opens up a whole new world of sites and applications which we’re only just beginning to appreciate. Offline’s important, and not just because of the Web Store.
Storage Options: What I’d like to do here is take a very brief survey of the landscape for context, and then dive into one particular feature that I think will become important in the near future: IndexedDB.
Cookies: These aren’t offline at all, but they’re relevant to the general context of how web applications store data at the moment. The image on this slide is Luigi Anzivino’s “Molasses-Spice cookies” (which look delicious).
Cookies
- Simple, key-value pairs, “shared’ between server and client.
- Excellent for maintaining state, poor for anything else, as they are unstructured, and incur a signiﬁcant overhead for each HTTP request.
Local Storage: The image on this slide is Evan Leeson’s “Toasters”.
Local Storage
- The simplicity of cookies, tuned for higher-capacity, client-side-only storage.
- Dead simple API:
```
localStorage.setItem( ‘key’, ‘value’ );
localStorage.getItem( ‘key’ ); // ‘value’
```
- Values are unstructured strings:
  - Filtering and search are O(n), unless you layer some indexing on top.
  - Structure requires JSON.stringify & JSON.parse
WebSQL: The image on this slide is Nick P’s “file cabinet to heaven”, which is a pretty accurate representation of life with WebSQL. Stacking file cabinets on top of each other certainly provides you with the possibility of well organized storage, but that doesn’t mean it’s a good idea.
WebSQL
- A real, relational database\n implementation on the client (SQLite)
- Data can be highly structured, and JOIN enables quick, ad-hoc access
- Big conceptual overhead (SQL), no finely grained locking
- Not very JavaScripty, browser support is poor (IE and Firefox won’t implement it), and the spec has been more or less abandoned.
File API: The image on this slide is Davide Tullio’s “Hard Disk in B&W.
File API: I know nothing about the File API, but Seth does! And his presentation is right after mine, so I’ll be all ears. :)
IndexedDB: The image on this slide is Robin Riat’s “Kanuga library card catalog”
IndexedDB:
- Sits somewhere between full-on SQL and unstructured key-value pairs in localStorage.
- Values are stored as structured JavaScript objects, and an indexing system facilitates filtering and lookup.
- Asynchronous, with moderately granular locking
- Joining normalized data is a completely manual process.
IndexedDB Concepts
Practically everything is asynchronous. Callbacks are your friends.
Databases are named, and contain one or more named Object Stores
A diagram of how a database might look, containing a single object store and a set of objects.
Object stores define a property (similar to a primary key) which every stored object must contain, explicitly or implicitly (autoincremented).
The same diagram as #18, with IDs added.
Values in an Object Store are structured, but don’t have a rigidly defined schema. Think document database, CouchDB. Not MySQL.
The same diagram as #20, with differing data added for various objects.
Object Stores can contain one or more Indexes that make filtering and lookup possible via arbitrary properties.
The same diagram as #22, with a subset highlighted (as though they were filtered out).
IndexedDB API: Now we’ll dive into some JavaScript. Lovely, lovely JavaScript.
It’s beta. Again. This is a reminder. :)
Vendor Prefixes: webkitIndexedDB & moz_indexedDB

Code:

// Deal with vendor prefixes
if ( "webkitIndexedDB" in window ) {
  window.indexedDB      = window.webkitIndexedDB;
  window.IDBTransaction = window.webkitIDBTransaction;
  window.IDBKeyRange    = window.webkitIDBKeyRange;
  // ...
} else if ( "moz_indexedDB" in window ) {
  window.indexedDB = window.moz_indexedDB;
}
if ( !window.indexedDB ) {
  // Browser doesn’t support indexedDB, do something
  // clever, and then exit early.
} 

Database Creation

Code:

var dbRequest = window.indexedDB.open(
  “AddressBook”,        // Database ID
  “All my friends ever” // Database Description
);

// The result of `open` is _not_ the database.
// It’s a reference to the request to open
// the database.  Listen for its `success` and
// `error` events, and respond appropriately.
dbRequest.onsuccess = function ( e ) { ... };
dbRequest.onerror   = function ( e ) { ... };

Databases are versioned…

Code:

// The `result` attribute of the `success` event
// holds the communication channel to the database
dbRequest.onsuccess = function ( e ) {
  var db = e.result;
  // Bootstrapping: if the user is hitting the page
  // for the first time, she won’t have a database.
  // We can detect this by inspecting the database’s
  // `version` attribute:
  if ( db.version === “” ) {
    // Empty string means the database hasn’t been versioned.
    // Set up the database by creating any necessary
    // Object Stores, and populating them with data
    // for the first run experience.
  } else if ( db.version === “1.0” ) {
    // 1.0 is old!  Let’s make changes!
  } else if ( db.version === “1.1” ) {
    // We’re good to go!
  }
  // ...
};

… and versioning is asychronous.

Code:

dbRequest.onsuccess = function ( e ) {
var db = e.result;
if ( db.version === “” ) {
  // We’re dealing with an unversioned DB.  Versioning is, of
  // course, asynchronous:
  var versionRequest = db.setVersion( “1.0” );
  versionRequest.onsuccess = function ( e ) {
    // Here’s where we’ll set up the Object Stores
    // and Indexes.
  };
}
// ...   };

Creating Object Stores and Indexes

Code:

dbRequest.onsuccess = function ( e ) {
var db = e.result;
if ( db.version === “” ) {
  var versionRequest = db.setVersion( “1.0” );
  // Setting a version creates an implicit Transaction, meaning
  // that either _everything_ in the callback succeeds, or
  // _everything_ in the callback fails.
  versionRequest.onsuccess = function ( e ) {
    // Object Store creation is atomic, but can only take
    // place inside version-changing transaction.
    var store = db.createObjectStore(
      "contacts",  // The Object Store’s name
      "id",        // The name of the property to use as a key
      true         // Is the key auto-incrementing?
    );
    // ...
  };
}
// ...   };

More code:

dbRequest.onsuccess = function ( e ) {
  var db = e.result;
  if ( db.version === “” ) {
    var versionRequest = db.setVersion( “1.0” );
    versionRequest.onsuccess = function ( e ) {
      var store = db.createObjectStore( "contacts", "id", true );
      store.createIndex(
        “CellPhone”,  // The index’s name
        “cell”,       // The property to be indexed
        false         // Is this index a unique constraint?
      );
    };
  }
  // ...
};

Writing Data (is asynchronous)

Code:

// Assuming that `db` has been set somewhere in the current
// scope, we use it to create a transaction:
var writeTransaction = db.transaction(
  [ “contacts” ],           // The Object Stores to lock
  IDBTransation.READ_WRITE  // Lock type (READ_ONLY, READ_WRITE)
);
// Open a contact store...
var store = writeTransaction.objectStore( “contacts” );
// ... and generate a write request:
var writeRequest = store.add( {
    “name”:  “Mike West”,
    “email”: “mkwst@google.com”
} );
writeRequest.onerror = function ( e ) {
    writeTransaction.abort();
};
// Transactions are “complete” (not “committed”?) either when
// they fall out of scope, or when all activities in the
// transaction have finished (whichever happens last)
writeTransaction.oncomplete = function ( e ) { ... };

Reading Data (is asynchronous)

Code:

// Assuming that `db` has been set somewhere in the current
// scope, we use it to create a transaction:
var readTransaction = db.transaction(
  [ “contacts” ],           // The Object Stores to lock
  IDBTransation.READ_ONLY   // Lock type (READ_ONLY, READ_WRITE)
);
// Open the `contact` store...
var store = readTransaction.objectStore( “contacts” );
// ... and generate a cursor to walk the complete list:
var readCursor = store.openCursor();
// Setup a handler for the cursor’s `success` event:
readCursor.onsuccess = function ( e ) {
  if ( e.result ) {
    // You now have access to the key via `e.result.key`, and
    // the stored object via `e.result.value`.  For example:
    console.log( e.result.value.email ); // mkwst@google.com
  } else {
    // If the `success` event’s `result` is null, you’ve reached
    // the end of the cursor’s list.
  }
};

Querying (is asynchronous)

Code:

var t = db.transaction( [ “contacts” ], IDBTransation.READ_ONLY );
var s = t.objectStore( “contacts” );
// ... and generate a cursor to walk a bounded list, for example
// only those names between M and P (inclusive)
var bounds = new IDBKeyRange.bound(
  “M”,  // Lower bound
  “Q”,  // Upper bound
  true, // Include lower bound?
  false // Include upper bound?
);
var readCursor = store.openCursor( bounds );
// Setup a handler for the cursor’s `success` event:
readCursor.onsuccess = function ( e ) {
  // process `e.result`
};

Further Reading:
- The IndexedDB Spec
- Firefox 4: An Early Walkthrough of IndexedDB
- Mozilla Developer Docs
Questions?, Mike West, @mikewest, mkwst@google.com, http://mikewest.org/

— Mike West