Introduction: Node.js Webpage Part 2

Welcome to PART 2!!

This is part 2 to my Node.js website application tutorial. I broke this tutorial up into two parts as it separates those who just need a brief introduction and those who want a full tutorial on a webpage.

I am going to go through my site creation. Yours may be different, so follow mine and learn the techniques used. Once you choose a different HTML template the flow will be just slightly different. Keep this in mind.

Step 1: App Structure

So my site follows the express generator, however I used handlebars rather than jade. If you like jade go for it! Jade is short hand HTML without all the brackets and div's. If you do not understand that you might want to visit youtube and watch some HTML tutorials.

I prefer and am more comfortable with HTML and handlebars so that is what I used. To create an express project with handlebars run the express command.

express --hbs nameofmyapp 

Then continue to follow the step in Part 1 for installing all middle-ware.

Express creates a very specific app structure and a very useful one most node.js applications follow this form with some variation.

In the attatched photo you can see different folders and files, below I try to explain all of these.

bin

This is the folder that is run first when node.js starts your server. It looks to the www file and follows this file for execution. The www file tells node.js to start a server at port 3000 (this can change to just about anything) and do some other things such as event listener and such. The main important thing is the port in which your app is set up on.

node_modules

In this folder is what is called middle-ware. Middle-ware I like to explain as extra software to make things easier for you to code. Theya re basically other libraries with functions pre-made for you to use. Some additional middle-ware I used for this project was Nodemailer, Passport, Nodemon, bycrypt, and others.

public

This is where all of your images, CSS, and javascript for your website would go. These are directly used by the webpages.

routes

These are defines routes for your site. Such as a homepage, login page, and others.

views

As you can see the views are .hbs files or .handlebars, either will work it just takes some manipulation of the app.js file. These are your handlebars html pages that will be displayed on the browser. Layout is your main layout file and sometimes is in its own layout sub folder. The main layout file calls to your other handlebars files and displays them, this will make more sense when we dive into the code.

app.js

This is your main app file, sometimes this is called server, just depends on set up. This file has all of the configuration for the server and even some special functions. It will also be an error handler.

package.json

This file is created by express and tells npm all the middleware you want to use in your project. Once you run npm install, all of the middle-ware called out in this file, will be installed in the node_modules folder.

Step 2: Layout Your Template

You can create all of your HTML from scratch or you can use a template. I have used a template for this site. Other sites I have helped developed I have coded from scratch. The choice is yours, this step explains the template layout.

My web application uses a bootstrap template which is great at making amazing CSS. To find templates visit this site.
Like stated before in the previous step all of the needed css, js, and img files are under the public folder. These files make the site look better than plain text and its how images are used on the site.

In order to make the handlebars templating style work with a template The pages are split into two pieces. The first is what is refereed to as the "layout". The layout is the properties that you would want to be displayed on every web page within your site. In my case this is the header, which has the navigation bar, and the footer, which is holds extra navigation and display pieces.

The layout file and other handlebars files are in the views folder. I will go over a more simple layout from the express generator you used earlier to display how the concept works, then you can see my code and compare them.

Express generated layout.handlebars file

<!DOCTYPE html>
<html>
	<head>
		<title>{{title}}</title>
		<link rel='stylesheet' href='/stylesheets/style.css'/>
	</head>
	<body>
		{{{body}}}
	</body>
</html> 

The true handlebars magic is in the handlebars {{title}} and {{{body}}}. So these two act differently {{title}} is a variable that is passed from the index.js file in routes, once passed to the template it is displayed. The {{{body}}} tag takes what ever is called in the render function in your route js file. In our case index.js has this line:

res.render('index', { title: 'Express', count: userCount});

This calls the 'index' file of what ever engine your using, jade, handlebars, and so on, so in our case index.handlebars.

Express generated index.handlebars

<h1>{{title}}</h1>
Welcome to {{title}}

The index.handlebars file is passed like a variable to the {{{body}}} tag and displayed on your webpage.

This allows you to have a static portion of your website and a variable portion. This makes headers and footers nice as you do not have to re-render the whole page, when loading a new page, only some information is changed.

Step 3: Contact Form

I incorporated a contact form into my web page so anyone could email my site email, with questions or comments.

This contact form used a npm middle-ware that is called Node Mailer.

Setting up Node Mailer

To install node-mailer you just need to run the code below in your top level file, in our case, myapp.

sudo npm install nodemailer

Once installed you will need to set up a few things in your app.js file.

The first is just the dependency, this tells node that we plan to use this middleware.

var nodemailer = require('nodemailer');

Second is our transporter, the transporter is used to connect to your mail server, in my case gmail.

//Transporter used to get gmail account</p>var transporter = nodemailer.createTransport({
    service: 'gmail',
    auth: {
        type: 'OAuth2',
        user: 'theinternet.onthewifi@gmail.com',
        clientId: '139955258255-a3c6ilqu6rtocigde7cbrusicg7j00eh.apps.googleusercontent.com',
        clientSecret: 'Q775xefdHA_BGu3ZnY9-6sP-',
        refreshToken: '1/0HfdzyzW3FmnDPqeYkv19_py6zWgMCOqI9DSZ9kQWfc',
        accessToken: 'ya29.GlvDBGA2Z_coEKjQOnXAnBLbTB0wQmS-sARqNGC3V2UATiywNb34IhFq4d7UQvhTobE6pi83-FB2-OvMWjC-mk-EKPMYmwxFe9AOZ7mY6kurYyQ7e1Mu8m8INxg7'
    }
})

if using nodemailer with a different mail server please look here for documentation and help.

A few things will change from person to person: user, clientId, clientSecret. refreshToken, and accessToken.

Your userId is the email in which you want to use, I made a new one named the same as my site.

The clientId, clientSecret, refreshToken, and accessToken need to be found through your google account.

If you need more help you can follow this video here.

Once all of those fields have been filled in we will add our message details.

Next we need to validate that all fields in our form have been entered and are valid responses.

// Express Validator<br>app.use(expressValidator({
  errorFormatter: function(param, msg, value) {
      var namespace = param.split('.')
      , root    = namespace.shift()
      , formParam = root;


    while(namespace.length) {
      formParam += '[' + namespace.shift() + ']';
    }
    return {
      param : formParam,
      msg   : msg,
      value : value
    };
  }
}));

We now need to get information from our contact form on our webpage and send a message.

//Post from contact submit button, need to create a homepage with success message for submitted forms<br>app.post('/contact_Form', function(req, res){
    //Get information out of the contact form, from homepage.hbs
    var name = req.body.name;
    var email = req.body.email;
    var phone = req.body.phone;
    var message = req.body.message;
    var mailOptions = { //creates information used when sending a message
        from: 'Automatic Email',
        to: 'theinternet.onthewifi@gmail.com',
        subject: 'Website Contact Form: ' + name,
        text: 'You have received a new message from your website contact form.\n\n' + 'Here are the details:\n\nName: ' + name + '\n\nEmail: ' + email + '\n\nPhone: ' + phone + '\n\nMessage:\n' + message
    }
  transporter.sendMail(mailOptions, function (err, res) {
        if(err){
            console.log('Error');
        }else {
            console.log('Email Sent');
        }
    })
    res.render('index'); //render new homepage, look into how to do this with success message, like logout page
})

Flash

Flash is used to show messages after actions are done. You can see this when you submit a form, or do not enter a field correctly.

Install flash just like other npm middleware.

sudo npm install connect-flash
var flash = require('connect-flash'); //had flash functionality to show on screen messages
// Connect Flash<br>app.use(flash());

Enable flash which pushes and updates messages on the webpage. These are the messages that say things like success, or information was entered incorrectly.

// Global Vars
app.use(function (req, res, next) {
  res.locals.success_msg = req.flash('success_msg');
  res.locals.error_msg = req.flash('error_msg');
  res.locals.error = req.flash('error');
  res.locals.user = req.user || null;
  next();
});

Some need variables associated with flash.

There ya go a made contact form.

Step 4: Login Page

This was just something I wanted to see if I could do and maybe I will use it in the future. I just wanted to explain the code as it is in my git repository.

So this portion uses a few more npm middle-ware. Install the following using the commands below.

npm install passport && npm install passport-local && npm install bcryptjs

The && allows you to run multiple commands with one line.

Login and Users

You will need to creat a login.js and user.js file under your routes folder. This will be used to allow the creation of a user, which will be stored in our database, and allow the user to login by checking the database.

user.js

var express = require('express');<br>var router = express.Router();
var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var User = require('../models/user');

// Register
router.get('/register', function(req, res){
	res.render('register');
});

//Register User
router.post('/register', function(req, res){
	var name = req.body.name;
	var email = req.body.email;
	var username = req.body.username;
	var password = req.body.password;
	var password2 = req.body.password2;
	// Validation
	req.checkBody('name', 'Name is required').notEmpty();
	req.checkBody('email', 'Email is required').notEmpty();
	req.checkBody('email', 'Email is not valid').isEmail();
	req.checkBody('username', 'Username is required').notEmpty();
	req.checkBody('password', 'Password is required').notEmpty();
	req.checkBody('password2', 'Passwords do not match').equals(req.body.password);

	var errors = req.validationErrors();
	if(errors){
		res.render('register',{
			errors:errors
		});
	} else {
		var newUser = new User({
			name: name,
			email:email,
			username: username,
			password: password
		});
		User.createUser(newUser, function(err, user){
			if(err) throw err;
			console.log(user);
		});
		req.flash('success_msg', 'You are registered and can now login');		res.redirect('/login');
	}
});

Breaking this down piece by piece

First we include all middle-ware needed, then we include our model file which is explained below. We route from the register tag and display our register handlebars text. Then the important function comes. These allow us to register a new user in our database. The function checks to make sure all fields are valid and included in the form, if not it will prompt for them. Next it checks for errors, and if no errors occur it creates a new user with the information given. It then re routes to the login page, allowing you to login.

login.js

var express = require('express');
var router = express.Router();<br>var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var User = require('../models/user');

/* GET users listing. */
//Homepage
router.get('/', function(req, res){
	res.render('login');
});

passport.use(new LocalStrategy(
  function(username, password, done) {
   User.getUserByUsername(username, function(err, user){
   	if(err) throw err;
   	if(!user){
   		return done(null, false, {message: 'Unknown User'});
   	}
   	User.comparePassword(password, user.password, function(err, isMatch){
   		if(err) throw err;
   		if(isMatch){
   			return done(null, user);
   		} else {
   			return done(null, false, {message: 'Invalid password'});
   		}
   	});
   });
  }));

passport.serializeUser(function(user, done) {
  done(null, user.id);
});

passport.deserializeUser(function(id, done) {
  User.getUserById(id, function(err, user) {
    done(err, user);
  });
});

router.post('/login',  passport.authenticate('local', {successRedirect:'/', failureRedirect:'/login',failureFlash: true}),
  function(req, res) {
    res.redirect('/dashboard');
  });

router.get('/logout', function(req, res){
	req.logout();
	req.flash('success_msg', 'You are logged out');
	res.redirect('/homepage');
});

<p>module.exports = router;</p>

First we include all middle-ware needed, then we include our model file which is explained below. We route from the login tag and display our login handlebars text. We then use some passport functions to take the entered user name and password and check them against our database. We will use an encrypted password also which can make login a little slow on a raspberry pi. I explain this more next. After validating the user name and password, you are redirected to the homepage which will show the dashboard as we set this up in our index file. We also add here the ability to logout.

Like I previously mentioned we will also need to create a model to check the database for.

This is done by creating a folder under your main application folder called models. In this folder a user.js file is also needed.

model/user.js

var mongoose = require('mongoose');
var bcrypt = require('bcryptjs');
// User Schema
var UserSchema = mongoose.Schema({
	username: {
		type: String,
		index:true
	},
	password: {
		type: String
	},
	email: {
		type: String
	},
	name: {
		type: String
	}
});
var User = module.exports = mongoose.model('User', UserSchema);</p><p>module.exports.createUser = function(newUser, callback){
	bcrypt.genSalt(10, function(err, salt) {
	    bcrypt.hash(newUser.password, salt, function(err, hash) {
	        newUser.password = hash;
	        newUser.save(callback);
	    });
	});
}
module.exports.getUserByUsername = function(username, callback){
	var query = {username: username};
	User.findOne(query, callback);
}
module.exports.getUserById = function(id, callback){
	User.findById(id, callback);
}
module.exports.comparePassword = function(candidatePassword, hash, callback){
	bcrypt.compare(candidatePassword, hash, function(err, isMatch) {
    	if(err) throw err;
    	callback(null, isMatch);
	});
}

This model outlines what our user parameters will look like as well as how we will access them. I mentioned before that we will be encrypting our passwords. this is so that no ones password is store in the database in case of a breach. The passwords are hashed using middle-ware bcrypt.

Step 5: Traffic Counter

I wanted to see how many unique users visited my web page and count the number of "hits". There are many ways to do this, I will be explaining how I went about it.

This uses a mongodb collection to track how many users have visited my page and how many times each unique visitor visited.

Since we have already talked about setting up a mongoDB i will not go through it again.

You may need to add two collections to your database in order to compile. To do this you can either install RoboMongo if using an UI, however if you are using a headless raspberry pi like I am you will fun the following commands.

Mongo shell

To edit a db, get information, or create a collection you will need the mongo shell on a headless unit.

Run

mongo

This will open the shell.

Add a collection

In my case, the database is called loginapp, you can name it whatever you want.

use nameofyourdb

We need a collection to hold all of our ip address of the users who visit our site.

db.creatCollection("ip")

Next we create a collection to count the unique hits to our site. This is initialized with an id and count starting at 0.

db.createCollection("count", {id: "hit counter", count:0})

Track IP Addresses

To do this we will pull the users Ip when they visit our home page, increment our count, and store them to compare them later.

We need to create some models to store our mongoose schemas, and add some code to our homepage.js file.

We create count.js and ip.js and store them in our models folder.

The ip.js file is just a schema for our ip address

var mongoose = require('mongoose'); //package handler for mongo
//Count Schema
var IpSchema = mongoose.Schema({
	ip: {
		type: String,
	},
	count: {
		type: Number,
	}
});
var Ip = module.exports = mongoose.model('Ip', IpSchema);

count.js will be called by our homepage to initiate the hit tracking. This is done so as below.

//Homepage<br>router.get('/', function(req, res){
	publicIp.v4().then(ip => {
		Public_ip = ip;
    	console.log("ipv4: "+ Public_ip);
    	
    	//=> '46.5.21.123' 
	});
	publicIp.v6().then(ip => {
    	console.log("ipv6" + ip);
    	Public_ip=ip;
    	//=> 'fe80::200:f8ff:fe21:67cf' 
	});	</p><p>	Count.getCount(collection, ipc, Public_ip, function(count){
    	
    });
    count = db.collection('count').findOne({id: "hit counter"}, function(err, count){
    	userCount = count.count;
    	res.render('homepage', {count: userCount});
    }); 
});

This happens every time someone goes to our homepage, in this case theinternet.onthewifi.com/homepage.

It checks the IP of the user, ip4 or ip6, and then stores that value where it sends it to count.get.collection which is a function stored in our count.js file.

After checking the uniqueness of the user it then returns and posts the count value to the homepage as a handlebars variable.

The count.js file is as follows.

<p>//count.js<br>var mongo = require('mongodb'); //supports database
var mongoose = require('mongoose'); //package handler for mongo
mongoose.connect('mongodb://localhost/loginapp');
var db = mongoose.connection;
var Ip = require('../models/ip');
 
//Count Schema
var CountSchema = mongoose.Schema({
	id: {
		type: String,
	},
	count: {
    		type: Number,
    }
});
 
var Count = module.exports = mongoose.model('Count', CountSchema);
 
module.exports.getCount = function(count, ipc, Public_ip, callback){
//count is test, callback isfunction
    ipc.findOne({ip : Public_ip}, function(err, iptest){
        if(!iptest)//add a new ip if not in the database, and update counter
        {
            var new_ip = new Ip({
                ip: Public_ip,
                count: 1
            });
            db.collection('ip').save(new_ip);//add new ip to database
            count.update(//update hit counter
                { id: "hit counter"},
                { $inc: {count: 1} }
            )
        }
        else//update specific ip counter, to see who visits the most
        {
            ipc.update(
                { ip: Public_ip },
                { $inc: {count: 1} }
            )
        }
    });
}

This creates the count schema, and our .getCount function. The .getCount function checks the DB for a the users ip and if it finds it, the function increments the count of that user, not the hit counter. However if the users ip is not found it will create a new collection object with the users ip and increment the hit counter by 1.

This is then returned and displayed to the webpage.

There you have it an ip tracking hit counter.

Step 6: Blog

I currently am trying to develop a blog centralized on my interests about software, smart homes, and Polaroids. So I created a blog section. The blog uses static html pages and the handlebars framework. After looking into better technologies to make blogging easier I have since redesigned my website using hugo. Hugo is a static html generator. I talk more about this in the below mentioned tutorial.

Step 7: Finished

There you go an in depth tutorial on my node.js website hosted locally on my raspberry pi. If you have questions or comments please leave them below.

I hope this helps others out there.

For a different approach to this site using hugo, a static web page generator see my other tutorial (coming soon).