Browsing:

Category: javascript

Beginning Node.js – Testing – part 5

Testing for the newbie

To test your Node API's you will need to install the following dependencies:

  • Mocha. Mocha is a library to run tests, AKA a test runner, or test infrastructure. Install it globally, like so: npm install -g mocha.
  • Chai. With Chai you can use "assert that.. " or "should be.. " or "expect that.. ". Depends on the test philosophy you are using (BDD, TDD and the likes)
  • Superagent. Superagent is a library that can test a web api, like so: superagent.get('http://localhost:3001/food')

Of course there are alternatives (Jasmine, Should ..) but in my humble experience the above 3 work just fine. At first I was wondering why I needed so many libraries but I guess this all about the freedom to mix and match the modules you like the most. Which is quite nice.

Test the API

This is an example test to the 'list' function of all the food. Does the API return a JSON array? Well, it should, so I expect it to be.

First, Create a folder named 'test'
Then put a file in it. (e.g. test1.js), with these contents:

var superagent = require('superagent');
var chai = require('chai');
var expect = chai.expect;

describe("Food JSON api", function() {  

  it("should return a json array", function(done) {
    superagent.get('http://localhost:3001/food')
      .end(function(err, res) {
        expect(err).to.eql(null);
        expect(res.status).to.equal(200);
        expect(res.body.success).to.equal(true);
        expect(res.type).to.equal("application/json");
        done();
      });
  });

}

Now run the test with the command: mocha.

[jacqueline@pc01 Restaurant]$ mocha

  Food JSON api
    ✓ should return a json array 


  1 passing (39ms)

Another test

With SuperAgent you can also test a POST action (be sure not to use your production db, but more about different environments later).

it("should post a new piece of food", function(done) {
    superagent.post('http://localhost:3001/food/')
      .send({
        name: "awesome food in test db from env", description: "awesome food description", price: "16,00"
      })
      .end(function(err, res) {
        expect(err).to.eql(null);
        expect(res.body._id).to.not.be.eql(null);
        id = res.body._id;
        done();
      });
  });

Run the test:

[jacqueline@pc01 Restaurant]$ mocha


  Food JSON api
    ✓ should return a json array 
    ✓ should post a new piece of food 


  2 passing (38ms)

So, this was just a wee intro to testing web API's. There is lots more to discover about this subject!

I just put the code online at github. This is still a work in progress though.


Beginning Node.js – REST API with Express 4 – part 2

This is the most simple REST API possible. I will explain every single detail in this post.

var express = require('express');
var app = express();

var menu = [{"description" : "papadums", "price":"2,00"}, {"description" : "chicken tikka masala", "price":"14,00"},{"description" : "fisher king beer", "price":"2,50"},{"description" : "sag paneer", "price":"6,00"}];

app.use(express.static(__dirname + '/public'));

app.get('/food', function(req, res){
  res.send(menu);
});

app.set('port', process.env.PORT || 3000);
app.listen(app.get('port'));
console.log("the server is running on http://localhost:" + app.get('port'));

Let's get started

First create a folder named 'restaurant' or whatever you would like to name the API.
Then run 'npm init' to create a package.json file. This file contains metadata, dependencies etc.:

mkdir restaurant
cd restaurant
npm init

You will get a bunch of questions. Answer them (or accept the defaults) and hit enter:

About to write to /Users/jacqueline/Dropbox/devel/restaurant/package.json:
{
  "name": "restaurant",
  "version": "0.1.0",
  "description": "",
  "main": "main.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Jacqueline",
  "license": "MIT"
}

Is this ok? (yes)

Install Express

Next we will install Express with NPM. Express is a minimalist web framework for Node.js.

jacqueline@nsa:~/Dropbox/devel/restaurant$ npm install express --save

You will now see 'express' mentioned in the package.json.
The '--save' parameter adds a "^" before the package, it will make sure the right version is used.

Create the app

Create a file named 'main.js' (that's how I named the main file in my package.json, you can also name it app.js or server.js).

Add these contents:

var express = require('express');
var app = express();
app.get('/', function(req, res){
      res.send('hello world');
    });

 app.listen(3000);

Next install nodemon because it is awesome. It restarts node every time a file in the project is changed.
Add the '-g' parameter because nodemon should be installed globally and not just as a module in the project folder.

npm install -g nodemon

Now run nodemon main.js on the commandline. It will return:

jacqueline@nsa:~/Dropbox/devel/restaurant$ nodemon main.js 
18 Oct 08:04:59 - [nodemon] v1.2.0
18 Oct 08:04:59 - [nodemon] to restart at any time, enter `rs`
18 Oct 08:04:59 - [nodemon] watching: *.*
18 Oct 08:04:59 - [nodemon] starting `node main.js`

Now browse to http://localhost:3000. This will display 'Hello World' in the browser.

Serve JSON data

Alter your main.js file and add some json and change the app.get bit:

var express = require('express');
var app = express();

var menu = [{"description" : "papadums", "price":"2,00"}, {"description" : "chicken tikka masala", "price":"14,00"},
{"description" : "fisher king beer", "price":"2,50"},{"description" : "sag paneer", "price":"6,00"}];

app.get('/food', function(req, res){
  res.send(menu);
});

app.listen(3000);

Now if we browse to http://localhost/food, we will retrieve our delicious menu. We can also a REST client like POSTMAN to test:
food

With curl:

curl http://localhost:3000/food
[{"description":"papadums","price":"2,00"},{"description":"chicken tikka masala","price":"14,00"},{"description":"fisher king beer","price":"2,50"},{"description":"sag paneer","price":"6,00"}]

Consuming the API

Create a folder called public in your app directory.

Next add this line to main.js, with this we'll tell Express to go and find index.html files in a subfolder called public:

app.use(express.static(__dirname + '/public'));

Drop an index.html in the public folder.
This makes the directory structure look like so:

restaurant/
├── main.js
├── node_modules
│   └── express
├── package.json
└── public
└── index.html

Open index.html and add the following code:



  
  
  
    

Welcome to my restaurant!

On the menu:

And the result is..:

food-awesom

Add Bootstrap with Bower

Bower is like NPM, but for front end scripts. You can use NPM for frontend scripts as well, but I like to keept them separate.

npm install -g bower

Create a file named .bowerrc to tell Bower to put the javascripts into a /public/js:

vim .bowerrc
{
    "directory" : "public/js"
 }

You can create a bower.json file to organize the dependencies:

bower init

jacqueline@nsa:~/Dropbox/devel/restaurant$ bower init
? name: restaurant
? version: 0.1.0
? description: menu app
? main file: main.js
? what types of modules does this package expose?: node
? keywords:
? authors: jacqueline <jacqueline@nsa.gov>
? license: MIT

Then install your client side javascript dependencies like so:

bower install --save bootstrap

Now we can add some styling:

nicerestaurant

Last but not least: some feedback when starting the server

In Express you can define variables with app.set. I would suggest to configure the port like this:

app.set('port', process.env.PORT || 3000);
app.listen(app.get('port'));
console.log("the server is running on http://localhost:" + app.get('port'));

process.env.PORT || 3000 means: whatever is in the environment variable PORT, or use 3000 if there's nothing there.

This makes completed main.js look like this:

var express = require('express');
var app = express();

var menu = [{"description" : "papadums", "price":"2,00"}, {"description" : "chicken tikka masala", "price":"14,00"},
{"description" : "fisher king beer", "price":"2,50"},{"description" : "sag paneer", "price":"6,00"}];

app.get('/food', function(req, res){
  res.send(menu);
});

app.use(express.static(__dirname + '/public'));

app.set('port', process.env.PORT || 3000);
app.listen(app.get('port'));
console.log("the server is running on http://localhost:" + app.get('port'));

And this is where this tutorial ends. I love open source. It's great to be able to use all these tools (Node, NPM, Express) and be very productive. Next time I will show how to use a (NoSQL) database. Not sure to use MongoDb or Postgresql yet. We'll see.


Beginning Node.js – callbacks – part 1

Disclaimer: This is a series about me, creating a web application in Node.js. The completed example is available here.

And from there the series end, because frontend frameworks will take over.

In this article about the MEAN stack, I'm not spending much time on Node.js. However, Node.js is quite awesome. It's lightweight, cross platform and it runs JavaScript files. But what is it?

  • Node.js is like IIS.
  • You can use a module like Express.js (or Restify, Sails.js, Hapi etc.) to implement a RESTful API, much like the ASP.NET Web API.
  • You install modules (packages) with NPM, which is comparable to Nuget.

Node.js is a platform built on Chrome's JavaScript runtime (v8). Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

So what does that even mean? And how to get started? Apart from this blog, there are plenty of tutorials out there and also great books. These are probably much better than this intro, however I would like to share that piece of code that made it click for me. And if I not write down my words, they are wasted.

What do we need?

  • Node.js (I'm on v0.10.30). We will use Node.js to to build server side applications. So no IIS, applications pools and .NET. All you need is the 5 MB's of Node.js! Let's sink that in for a moment!
  • A code editor of choice (I prefer Vim and Brackets)
  • Linux OS, Mac OS or Windows 7/8. I'm on a Mac and on Ubuntu 14.04. I only use Windows at work.
  • For Windows, you will need Python 2.7, Visual Studio Express and node-gyp installed locally. Check this. These are prerequisites to build node modules on Windows (a bit of a pain indeed)
  • Just a little bit of a coding background.

Node.js installation

Linux or Mac
To install Node.js on Linux or Mac, I would advise to follow this gist:

echo 'export PATH=$HOME/local/bin:$PATH' >> ~/.bashrc
. ~/.bashrc
mkdir ~/local
mkdir ~/node-latest-install
cd ~/node-latest-install
curl http://nodejs.org/dist/node-latest.tar.gz | tar xz --strip-components=1
./configure --prefix=~/local
make install # ok, fine, this step probably takes more than 30 seconds...
curl https://www.npmjs.org/install.sh | sh

Windows
To install Node.js on Windows just install the .msi.from the Node.js website.
For NPM don't forget to create an NPM folder in C:\Users\Username\AppData\Roaming\npm\

The basics: callbacks

Normally, code gets executed from top to bottom:

function dothis() {
// do some IO heavy work, or get data from a file
}

function dothat() {
//doing that
}

dothis();
dothat();

If 'dothis' is CPU intensive, or IO intensive, it takes a long time before the code reaches 'dothat'. In the mean time, nothing happens. The code is 'blocking'.
In .NET you can solve blocking operations by using another thread, or write the code asynchronously.
But Node.js code is almost exclusively asynchronous. There is but ONE thread in Node.js.

So in Node, code is executed asynchronously. While some IO heavy operations occurs, Node can do other stuff. When the heavy operation is finished, the function involved 'calls back', like saying: " Hey, I'm done. Here is the result".

So how do we write asynchronous code in Node?

Consider this example where I want to display an order of Indian food on the screen.
order.txt contains a list of Indian food.

var fs = require('fs'); // this is like using System.IO

//initialize the variable
var theOrder;

//this is a function that reads the contents of order.txt. 

function orderSomething() {
   //the 'readFile' method takes (at least) 2 arguments: the file, and also a callback (result).
   fs.readFile('order.txt', function(result){
   theOrder = result;
   });
  }

console.log("The customer ordered:\n" + orderSomething());

Now save this code as test.js and run it:

jacqueline@laptop:~$ node test.js
The custumer ordered:
undefined

So this does not display my order. Why not?
Because when console.log is executed, the function orderSomething is not ready yet.

We can solve the problem by adding a parameter to the orderSomething function. In Node, the last parameter of a function is a function, that will be executed when orderSomething is finished: orderSomething(doThisWhenFinished). (In C# you can also pass a function as a parameter, with Func<> and Action<>, however I'm not sure it knows about async behavior).

So, we basically add a function to a function (which can contain another function and so on).
This is very important. The parameter of a function in JavaScript can be another function, that gets executed when the first function is done.

var theOrder;

//first, orderSomething and when it's done, execute 'done'
function orderSomething(done) {
  fs.readFile('order.txt', function(err,result) {
    if (err) { throw err; }
    theOrder = result;
// call this when done:
    done();
  });
}

function showOrder(){
 console.log("The customer ordered:\n" + theOrder);
}

orderSomething(showOrder);

Now run this code:

jacqueline@laptop:~$ node test.js
The custumer ordered:
//Chicken Tikka Massala
//Papadums
//King Fisher Beer

This is how Node.js works. The last argument of a function is the callback, which gets executed when the function is ready.

Wrapping up

We can rewrite the above example as follows:

var fs = require("fs");
fs.readFile("order.txt","utf-8", function (error, result) {
     console.log(result);
  });

Read the file, then display the result.

That was all.


Nodejs and the MEAN stack for the .NET developer part 3

In this series I'm exploring the MEAN stack. In the previous posts I've covered the M (Mongo), the E (Express) and the N (Nodejs).
It's time for the A. I'm going to add Angularjs and start with the frontend for the reMember application.

NPM for the frontend: Bower

Bower is a package manager for the web. It can be used to add frontend scripts to a website, like Ember, Angular, jQuery and Twitter Bootstrap.

Install Bower globally:

npm install - g bower

First, you need to tell Bower where to download its files.

touch server/.bowerrc
vim server/.bowerrc
{ "directory"  : "../public/javascripts/vendor" }

This configuration file will tell Bower to install tools into the "../public/javascripts/vendor" directory.

bower install angular bootstrap

This is the result:

apps/reMember/public/                                                                                                                   
▾ images/                                                                                                                                         
▾ javascripts/                                                                                                                                    
  ▾ vendor/                                                                                                                                       
    ▸ angular/                                                                                                                                    
    ▸ bootstrap/                                                                                                                                  
    ▸ jquery/                                                                                                                                     
▾ stylesheets/                                                                                                                                    
    style.css                                                                                                                                     
  index.html    

Hook up Angular

We did this before!

Open index.html and make it look like so:



  
        reMember
      
      
        
        
        
     

{{title}}

Create /public/javascripts/app.js:

var reMemberApp = angular.module('reMemberApp',[]);
 
reMemberApp.controller('WelcomeController', function ($scope) {
     $scope.title = "The cat just made a strange noise";
     $scope.story = "We better check her out and see if she's ok."
});

Now go ahead and browse to http://localhost:3000:

cat-plain

Add some Twitter Bootstrap styling

I grabbed this theme and copied/pasted the source into index2.html.
The I adjusted all the paths to reflect the folder structure of the reMember app.

cat-plain-twitter

Consuming the API

Add the following to /public/javascripts/app.js:

$http.get('http://localhost:3000/members').success(function(data) { $scope.members = data; });

This is the result:

var reMemberApp = angular.module('reMemberApp',[]);
 
reMemberApp.controller('WelcomeController', function ($scope,$http) {
     $scope.title = "The cat just made a strange noise";
     $scope.story = "We better check her out and see if she's ok."

     $http.get('http://localhost:3000/members').success(function(data) { $scope.members = data; });
});

And add the following html to either index.html or index2.html:

  
  • {{member.firstName}}
  • And there you go:

    cat-twitter-rest

    I'm surprised of the simplicity of all this.
    Mind you, we've created a minimal web app with the MEAN stack. These tools are all free and open source. Even the OS and the IDE I'm working with is free and open source.
    This is why I want to share code with the world.

    Sources:
    http://thecodebarbarian.wordpress.com/2013/07/22/introduction-to-the-mean-stack-part-one-setting-up-your-tools/


    Nodejs and the MEAN stack for the .NET developer part 2

    In the previous article we've set up the web application skeleton with Express for the reMember application and we wrote a first test to see if we can actually save an object to the database with Mongoose.

    In an Express generated app, 'app.js' is the starting point of the application. (Like Global.asax or WebApiConfig in an ASP.NET MVC app).

    Since the goal of this article is to write an api, we can clean it up a little.

    • We will not be using jade at this time, so we can clean out references to the view.
    • We can also clean out the 'user' lines because we are not going to use these.
    • We will comment out the routes as well and create a new default route to an index.html file
    var express = require('express');
    
    //!!don't need this:
    //var routes = require('./routes');
    //var user = require('./routes/user');
    var http = require('http');
    var path = require('path');
    
    var app = express();
    
    // all environments
    app.set('port', process.env.PORT || 3000);
    
    //!!don't need this, we're going to serve plain html:
    //app.set('views', path.join(__dirname, 'views'));
    //app.set('view engine', 'jade');
    app.use(express.favicon());
    app.use(express.logger('dev'));
    app.use(express.json());
    app.use(express.urlencoded());
    app.use(express.methodOverride());
    app.use(app.router);
    
    //!!change this as well to reflect our new folderstructure:
    app.use(express.static(path.join(__dirname, '../public')));
    
    // development only
    if ('development' == app.get('env')) {
      app.use(express.errorHandler());
    }
    
    //!!don't need this (yet)
    //app.get('/', routes.index);
    //app.get('/users', user.list);
    
    //!!add a default route
    app.all('/', function(req, res) {
      res.sendfile('index.html', { root: "../public" });
    });
    
    http.createServer(app).listen(app.get('port'), function(){
      console.log('Express server listening on port ' + app.get('port'));
    });
    

    Now create an index.html file in /public:

    <html>
    <head>
    <title>Still empty</title>
    </head>
    <body>
    <h1>Still empty..</h1>
    <p>But we're getting there..</p>
    </body>
    </html>
    

    And start up the server with node app.js
    You should now be able to browse to localhost:3000 and see the index.html.

    Debugging

    But what if you don't and you see something like this:

    /home/jacqueline/apps/reMember/server/app.js:37
    });
    ^
    SyntaxError: Unexpected token }
        at Module._compile (module.js:439:25)
        at Object.Module._extensions..js (module.js:474:10)
        at Module.load (module.js:356:32)
        at Function.Module._load (module.js:312:12)
        at Function.Module.runMain (module.js:497:10)
        at startup (node.js:119:16)
        at node.js:902:3
    jacqueline@linuxlab:~/apps/reMember/server$ 
    

    Then you probably forgot to close a curly brace or a quote. Stop node (ctrl+c), correct the changes and start again.
    But wait, there's an easier method.
    With nodemon it is possible to debug changes on the fly.
    Install it globally with npm install -g nodemon.

    Now start app.js with nodemon app.js

    Everytime you change and save a file the node server is automatically restarted. It crashed when something is from and it starts running as soon as you correct your mistakes:

    nodemon

    Take a look at http://localhost:3000 now:

    stillempty

    So that works.

    Are we going to write that API or what?

    Yes we are.
    Delete all the files in the routes folder.
    Create new file called api.js

    
    //!!the C# equivalent would be: 
    //using mongoose; 
    //using member;
    
    var mongoose = require('mongoose');
    var Member = require('../models/Member');
    
    
    //!! /members GET  (in ASP.NET MVC: public IEnumerable Get())
    
    exports.members = function(req,res) {
      Member.find({}, function(err, data) {
          res.send(data);
        });
    };
    

    I've written the C#/.NET equivalents in the comments for convenience.

    The Mongoose 'find' method signature is written in their API. For example:

    // executes immediately, passing results to callback
    MyModel.find({}, function (err, docs) {});
    

    The result of the query will be put in 'docs', which is always the 2nd parameter.

    The Mongoose API docs are a very good read and helped me enormously to understand how it works.

    Mapping the mongoose functions to a route in app.js

    We now need to map the exports.members function to the route /members.
    This happens in app.js.

    var express = require('express');
    var http = require('http');
    var path = require('path');
    
    //!!add the Mongoose module:
    var mongoose = require('mongoose');
    
    var Member = require('./models/Member');
    
    //!!add the path to the api.js file:
    var api = require('./routes/api');
    
    var app = express();
    
    // all environments
    app.set('port', process.env.PORT || 3000);
    app.use(express.favicon());
    app.use(express.logger('dev'));
    app.use(express.json());
    app.use(express.urlencoded());
    app.use(express.methodOverride());
    app.use(app.router);
    app.use(express.static(path.join(__dirname, '../public')));
    
    // development only
    if ('development' == app.get('env')) {
      app.use(express.errorHandler());
    }
    
    //!!connect to mongo:
    mongoose.connect('mongodb://localhost:27017/local');
    
    //!! map the /members route to the api.members function:
    
    app.get('/members', api.members);
    
    //default route
    app.all('/', function(req, res) {
      res.sendfile('index.html', { root: "../public" });
    });
    
    http.createServer(app).listen(app.get('port'), function(){
      console.log('Express server listening on port ' + app.get('port'));
    });
    

    Restart node (or don't of you're running nodemon) and browse to http://localhost:3000/members.

    grimm

    I've added some Grimm characters.

    Tip: use Postman to test the API.

    The github repo for reMember is here.

    Next time we'll start building the frontend with Angularjs. (The 'A' in the MEAN stack).