Getting started with Meteor and Derby on your own server

This article explains some background concerning the holy grail of single language unified client/server web app development, and gives a practical hands-on look at two special kinds of frameworks that could be used to bootstrap a project.

Background

So for major web apps on the new frontier, everybody knows, it's server side Javascript and on the front end, everybody knows, it's some kind of MV* pattern based framework... Javascript ... bringing the dream of single language web app development based on, everybody knows, Javascript.

On the back end, serverside, I have been working with Node.js for some time.

And while I've been distracted with some old-fashioned projects for some time now, I was beginning to see how Backbone.js on the frontend could structure views, models and logic to talk to Node.js endpoints on the backend in an intelligent fashion to form a nice little system capable of overcoming the old slow page based request-response cycle:

"Backbone.js gives structure to web applications by providing models with key-value binding and custom events, collections with a rich API of enumerable functions, views with declarative event handling, and connects it all to your existing API over a RESTful JSON interface."

And behind the Node.js endpoints, performing secure, scalable persistence of data, reside the new generation NoSQL kings and queens: CouchDB, MongoDB, or HTML5 local storage, or whatever....

Examples of this Brave New World abound (Twohree of the most interesting: LinkedIn Mobile, Foursquare and TileMill).

I don't have to write my own framework yet

So I began reading the above cited Addy Osmani's Learning JavaScript Design Patterns and studying to see what kind of best practices could be fused into an architecture I could actually get productive with, and studying Addy Osmani's soon to be published book, Developing Backbone.js Applications (also on Safari rough cuts).

I was slowly realizing that in order to tackle a non-trivial application I would have to write my own framework. And even though there was a lot to do just to get started, I was accepting that.

Meteor

Of course, you always want to be as sure as you can about things, and while I was positive Node.js was the best backend foundation, I often check out those pages facing off Backbone.js with other frontend frameworks (even though I am quite sure it is the best, I am persuaded by Addy Osmani on that one). You know the ones.

Now, in one of them, someone mentioned Meteor, which brings this all together without my having to invent my own framework (for now at least) and which unifies the frontend and serverside aspects of the app into a single effort. I can start prototyping those awesome projects I really want to do right now without going through that on my own. And as Meteor says, I'm not Google or Facebook, and I'd do better using Meteor (or Derby or ...) for at least my initial prototyping and work (using Addy's idea of quickchanging the framework in your best practices design based on separation of concerns) since they will be ready for prime-time before I could possibly be on my own.

The essence of Meteor goodness is summed up in the Seven Principles of Meteor, and in the Concepts section of the docs.

Installing Meteor on your own server

When I first installed Meteor I had upgraded the Node.js (with TJ Holowaychuk's fabulous "n" that lets you use multiple versions on the same server and easily upgrade them) and MongoDB versions on my development server. However, Meteor is a universe unto itself and downloads and packages everything it needs, with the versions it is currently using, without disturbing or needing your global server setup.

First visit the Meteor repo. With the "Quick start" (described in the README.md file) Meteor just downloads what it needs when needed.

I went for the "Slow start" for developers:

# cd /usr/local/src
# git clone https://github.com/meteor/meteor.git
# cd meteor
# admin/generate-dev-bundle.sh
# ./install.sh
# meteor --help
# cd docs
# meteor

When you run meteor in the "docs" directory, you are executing the meteor local server, and you can access the docs app at: http://localhost:3000 .

The generate-dev-bundle script is revealing, showing that an "older" version of Node.js is being used, as well as the use of fibers.

Kicking the tires: Meteor

There are many examples bundled with Meteor once it is installed. In the directory /usr/local/src/meteor/examples (you may have installed meteor elsewhere) you can easily find the leaderboard, todos and wordplay apps featured with explanatory videos on the Meteor site.

I also found /usr/local/src/meteor/other/template-demo (very cool) which I executed and brought up just like the others:

# cd /usr/local/src/meteor/examples/other/template-demo/
# meteor
[[[ /usr/local/src/meteor/examples/other/template-demo ]]]

Running on: http://localhost:3000/

You can create your own version of any of the three major examples very easily, in order to tinker with it. If I now login as user learner on my server, I create a learn meteor directory and grab me a copy of a demo:

$ mkdir learn-meteor
$ meteor create --list
Available examples:
  todos
  leaderboard
  wordplay

Create a project from an example with 'meteor create --example <name>'.
$ meteor create --example todos
todos: created.

To run your new app:
   cd todos
   meteor

Here's what is created for you:

$ tree
.
├── client
│   ├── todos.css
│   ├── todos.html
│   └── todos.js
├── public
│   ├── close_16.png
│   └── destroy.png
└── server
├── bootstrap.js
└── publish.js

You can analyze the code via the Meteor API docs.

To create a bare skeleton app, do:

$ meteor create myapp
myapp: created.

To run your new app:
   cd myapp
   meteor
$ cd myapp
$ tree .
.
├── myapp.css
├── myapp.html
└── myapp.js

Derby

Hmm... what if I don't feel like basing my project on a fibers based framework? Derby provides a refreshing alternative. And you can find all your old friends there:

"Derby eliminates the tedium of wiring together a server, server templating engine, CSS compiler, script packager, minifier, client MVC framework, client JavaScript library, client templating and/or bindings engine, client history library, realtime transport, ORM, and database. It eliminates the complexity of keeping state synchronized among models and views, clients and servers, multiple windows, multiple users, and models and databases.

At the same time, it plays well with others. Derby is built on top of popular libraries, including Node.jsExpressSocket.IOBrowserifyStylusLESSUglifyJSMongoDB, and soon other popular databases and datastores. These libraries can also be used directly. The data synchronization layer, Racer, can be used separately. Other client libraries, such as jQuery, and other Node.js modules from npm work just as well along with Derby."

To read about Derby, consult http://derbyjs.com/

Installing Derby on your own server

Before installing Derby, I upgraded the npm package manager (cuz Derby uses what is now this standard JS package manager):

# npm update -g
npm http GET https://registry.npmjs.org/node.io
npm http GET https://registry.npmjs.org/express
...
...
npm http 200 https://registry.npmjs.org/npm/-/npm-1.1.61.tgz
request@2.11.1 /usr/local/lib/node_modules/node.io/node_modules/jsdom/node_modules/request
/usr/local/bin/npm -> /usr/local/lib/node_modules/npm/bin/npm-cli.js
npm@1.1.61 /usr/local/lib/node_modules/npm
# npm --version
1.1.61

I then installed the Node.js binary management tool "n"(see https://github.com/visionmedia/n):

# npm install -g n

Install latest:

 

# n latest

Install latest stable:

 

# n stable

List installed binaries:

# n
  ο 0.8.8
     0.9.1 

I then installed MongoDB: (See http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian-or-ubuntu-linux/

# sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10Executing: gpg --ignore-time-conflict --no-options --no-default-keyring --secret-keyring /etc/apt/secring.gpg --trustdb-name /etc/apt/trustdb.gpg --keyring /etc/apt/trusted.gpg --primary-keyring /etc/apt/trusted.gpg --keyserver keyserver.ubuntu.com --recv 7F0CEB10
gpg: requesting key 7F0CEB10 from hkp server keyserver.ubuntu.com
gpg: key 7F0CEB10: public key "Richard Kreuter <richard@10gen.com>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
# vi /etc/apt/sources.list.d/10gen.list
# cat /etc/apt/sources.list.d/10gen.list
deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen
 
# apt-get update
 
# apt-get install mongodb-10gen
# mongo
MongoDB shell version: 2.2.0
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
> db.test.save( { prueba: 1 } )
> db.test.find()
{ "_id" : ObjectId("5047b9ce7ad5b5366cc01895"), "prueba" : 1 }
> exit
bye
#

Now I simply install Derby via npm:

# npm install -g derby

The Getting started section of the docs go on to show how to create an app and the structure of the resulting skeleton, with and without CoffeeScript. Read on there.

Kicking the tires: Derby

There is a separate repo for Derby demos. Rather than cloning it, as user "learner" I installed via npm and ran one of them:

$ mkdir learn-derby
$ cd learn-derby/
$ npm install derby-examples

$ cd node_modules/derby-examples/todos/

$ node server.js

info - socket.io started

Starting cluster with 1 workers in undefined mode

`kill -s SIGUSR2 16373` to force cluster reload

Go to: http://localhost:3003/

info - socket.io started

Cool!

Conclusions

I don't want this to be a Meteor vs Derby article (especially with two teams who like each other and share their likes and differences openly, for example, in this insightful article), or to get involved in the I love fibers or threads or pure callback coding controversy (although Meteor's use of fibers led me to consider Derby in the first place). Neither Meteor or Derby is ready for prime time now, but then, neither am I. With either one I can kickstart a project, and be optimally ready on all fronts. Certainly for some ("90%") projects I am with Meteor, with other special projects, I am leaning to Derby.

Certainly all "spoiled" developers should stay tuned to Addy Osmani's TodoMVC (including the Labs at the bottom of the page!) to stay abreast "in a sea of so many options".