Ember.js: Ember Data in Depth

This is a guide explaining how Ember Data works internaly. My initial motivation for writing this is to understand Ember better myself. I’ve found that every time I understand something about how Ember works, it improves my application code.

Main parts

First we need to understand what are the main concepts. Let’s start with a simple example.

App.User = DS.Model.extend({
  username: DS.attr("string")
});

Let’s dive deep into this. There are four important concepts, two of which are basic Ember.js and we’re going to skip them

• App.User represents a User class in the App namespace
• username represents a property on the User class

These are the basics and you should be familiar with them to understand the rest of this guide. Next we have DS.Model and DS.attr:

DS.Model and DS.attr

DS.Model is one of the core concepts in Ember Data and it represents a single resource. Models can have relationships with other models, similar to how you’d model your data in a relational database. But let’s ignore that for now.

DS.Model is both a state machine and a promise. If you don’t understand what promises are, please take a look at this awesome article which explains them in depth.

State machines are used throughout Ember and they basically represent something which can have multiple states and can transition between the states. For example DS.Model can have the following states (taken from the official Ember guide):

• isLoaded - The adapter has finished retrieving the current state of the record from its backend.
• isDirty - The record has local changes that have not yet been saved by the adapter. This includes records that have been created (but not yet saved) or deleted.
• isSaving - The record has been sent to the adapter to have its changes saved to the backend, but the adapter has not yet confirmed that the changes were successful.
• isDeleted - The record was marked for deletion. When isDeleted is true and isDirty is true, the record is deleted locally but the deletion was not yet persisted. When isSaving is true, the change is in-flight. When both isDirty and isSaving are false, the change has been saved.
• isError - The adapter reported that it was unable to save local changes to the backend. This may also result in the record having its isValid property become false if the adapter reported that server-side validations failed.
• isNew - The record was created locally and the adapter did not yet report that it was successfully saved. isValid No client-side validations have failed and the adapter did not report any server-side validation failures.

We can also bind to these with event handlers, which will be explained later, but for now let’s just list them:

• didLoad
• didCreate
• didUpdate
• didDelete
• becameError
• becameInvalid

I would also encourage you to go take a look at the source documentation on GitHub

It is important for us to understand what each state means, because they can affect how our application behaves. For example if we try to modify a record which is already being saved, we will get an exception saying something like this

Attempted to handle event `willSetProperty` on <App.User:ember1144:null>
while in state rootState.loaded.created.inFlight. Called with
{reference: [object Object], store: <App.Store:ember313>, name: username}

The important part here is the rootState.loaded.created.inFlight. If we look at the source of DirtyState, we can see what this means

Dirty states have three child states:

• uncommitted: the store has not yet handed off the record to be saved.
• inFlight: the store has handed off the record to be saved, but the adapter has not yet acknowledged success.
• invalid: the record has invalid information and cannot be send to the adapter yet.

Let’s go through the record lifecycle and observe it’s state. We can do this by doing .get("stateManager.currentState.name")

user = App.User.find(1)
user.get("isLoaded") // => true
user.get("isDirty") // => false
user.get("stateManager.currentState.name") // => loaded

user.set("username", "wycats")
user.get("isLoaded") // => true
user.get("isDirty") // => true, which means comitting the transaction will save the record
user.get("stateManager.currentState.name") // => uncommitted

user.get("transaction").commit()
// while the record is being saved
user.get("stateManager.currentState.name") // => inFlight
user.get("isSaving") // => true
// after the record was saved
user.get("stateManager.currentState.name") // => saved

Transactions and commit()

In the previous example, we’ve used get("transaction").commit() to persist the changes to the server. .commit() will take all dirty records in the transaction and persiste them to the server.

A record becomes dirty whenever one of it’s attributes change. For example

user = App.User.find(1)
user.get("isDirty") // => false
user.set("username", "wycats")
user.get("isDirty") // => true

If we create a new record, it will be dirty by default

user = App.User.createRecord()
user.get("isDirty") // => true

Currently there’s a regression that we change an attribute to something else, and then back to the original value, the record will be marked as dirty.

user = App.User.find(1)
originalUsername = user.get("username")

user.get("isDirty") // => false
user.set("username", "wycats")
user.get("isDirty") // => true
user.set("username", originalUsername)
user.get("isDirty") // => true, even though it should be false

But let’s hope this will be fixed soon.

Transactions

Until now we assumed that there is some global transaction which is the same for every single model. But this doesn’t have to be true. We can create our own transactions and manage them at our will.

I recommend you take a look at the tests for transactions in Ember Data repository. They basically show all of the scenarios which you can encounter. For example

transaction = store.transaction();
record = transaction.createRecord(App.User, {});

transaction.commit(); // this will save the record to the server

record.set("foo", "bar");
transaction.commit(); // nothing is committed here, because the record
                      // is removed from the transaction when it is saved

store.commit(); // this will save the record properly

We can also add a record to a transaction, which will remove it from the global transaction. Important thing to note here is that store.transaction() always returns a new transaction.

user = App.User.find(1);
transaction = store.transaction();
transaction.add(user);

user.set("username", "wycats");

store.commit(); // nothing happens
transaction.commit(); // user is saved

Same goes for deleting records

user = App.User.find(1);
transaction = store.transaction();
transaction.add(user);

user.deleteRecord();

store.commit(); // nothing happens
transaction.commit(); // user is deleted

We can also remove a record from a transaction

user = App.User.find(1);
transaction = store.transaction();

transaction.add(user);
transaction.remove(user);

user.set("name", "wycats");

transaction.commit(); // nothing happens

One scenario when transactions can be useful is when you just need to change one record, without affecting changes to other records. You can put that change in a separate transaction, instead of just doing store.commit().

Important thing to note here is that there’s a defaultTransaction for the store to which you can get via store.get("defaultTransaction"). This is where all of the records are placed, unless you explicitly create a new transaction and assign a record to it.

These two are completely equivalent

store.commit();
store.get("defaultTransaction").commit();

Just take a look at how store.commit() is defined

commit: function() {
  get(this, 'defaultTransaction').commit();
},

commit()

Now that we understand how transactions work, let’s dig deep into store.commit(). First thing we need to understand here is that Ember Transactions use this thing called bucket to store records with various states in. This is first initialized in the init method of DS.Transaction

init: function() {
  set(this, 'buckets', {
    clean:    Ember.OrderedSet.create(),
    created:  Ember.OrderedSet.create(),
    updated:  Ember.OrderedSet.create(),
    deleted:  Ember.OrderedSet.create(),
    inflight: Ember.OrderedSet.create()
  });

  set(this, 'relationships', Ember.OrderedSet.create());
}

Each bucket represents one state in which a record can possibly be. These are used in many different places in the transaction, and every time a method changes it’s state, it will be moved to a corresponding bucket

recordBecameDirty: function(bucketType, record) {
  this.removeFromBucket('clean', record);
  this.addToBucket(bucketType, record);
},

More content will be coming soon