Using pump instead of pipe

On thing that I quickly discovered whilst using Gulp to automate my development process, was the piping the results through from one plugin to another could often go wrong, and it was often hard to trap the reason and get proper error messaging out.

Then I discovered some example code which pointed me to pump.  Pump is...

a small node module that pipes streams together and destroys all of them if one of them closes

Essentially it allows you to restructure your Gulp task in a way that handles multiple plugins much better, and it fails more gracefully.

So I needed to include a copy of the new plugin at the top of my Gulp file...

var pump = require("pump");

Then I took the task which I wrote about in my last post...

gulp.task("js",function(cb) {

  browserify(jsFiles).bundle(),

    .pipe(source("script.js"))

    .pipe(gulp.dest("build/js"));

});

And re-wrote it to look like this...

gulp.task("js",function(cb) {

  pump([ 

    browserify(jsFiles).bundle(),

    source("script.js"),

    gulp.dest("build/js")

      ],cb); 
    });

As you can see, instead of using the "pipe" method to pass the files/content from one plugin to another, I am now passing an array of calls into pump and this is doing the hard work for me.

I have found that this makes the error messaging much more useful when I have syntax errors or plugins fail to work as expected.  It's certainly saving me a lot of time when it comes to debugging and fixing my Gulp file.

Adding Browserify into the mix

Having already written my first gulp task, I was ready to get a bit more advanced.  My first task was to concatenate javascript files, including putting my local version of jQuery in front of my own javascript, so there was a single file to download, with all the required plugins before.  However, I'd already heard of Browserify, and wanted to see if this would work with Gulp (short answer - yes!).

Browserify is a tool which lets you "require" plugins in your javascript, and then processes these at build time (for example, in a Gulp task) to automatically include the plugins as well.  This would achieve the same end result as my current concatenation task, but without me having to keep and manually update local copies of the plugin files, as I could get Browserify to do the job of finding and including them.

I went through a few different iterations, all of which worked but were a bit messy for one reason or another (including using the gulp-browserify plugin, which is deprecated!) but I'm going to skip over those and skip straight to my final implementation.

So I needed to include a copy of extra plugins at the top of my Gulp file...

var browserify = require("browserify");

var source = require("vinyl-source-stream");

Then I updated my task to look this this...

gulp.task("js",function(cb) {

  browserify(jsFiles).bundle(),

    .pipe(source("script.js"))

    .pipe(gulp.dest("build/js"));

});

Notice that I am no longer using Gulp.Src to pick up the javascript files from the array, Browserify is now handling this, and I'm using the "bundle" method to concatenate them, instead of using gulp-concat.  During this "bundle" method, the javascript is also being processed to find any "require" statements.

For me, I needed to require jQuery and a few other bits, so at the top of my project javsacript file, I added the following lines...

var jQuery = require("jquery");

window.$ = window.jQuery = jQuery;

require("jquery.easing");

require("bootstrap/js/scrollspy");

As you can see, you can simply use the "require" method by itself, or you can assign the exported method to a variable.  In the case of jQuery, it needs to be stored globally in order for Bootstrap to load properly.  

I also went from including all of the Bootstrap library from a CDN to just including the component that I required, which saves on filesize!

It's now really easy to include new libraries and plugins into my javascript, by simply installing them using NPM, and including them using Browserify, as part of my automated Gulp task.

First Gulp task - concatenation

I recently started using a new development process, using Gulp to automate development tasks.  I thought I'd start off easy, so I'd concatenate my existing javascript files into a single file.

You might ask, why would you want to do this?  And my answer would be, you want to reduce the number of items fetched when loading your page, for optimal performance.

You might ask, why don't you do this manually once and then always work on the concatenated file?  And my answer would be, some of the files are plugins or libraries, which I want to keep separate so that I can easily update them.  I only want them to be combined in the build version.

So, questions out of the way, here's how it went down.

Firstly, you need to make sure you include the relevant bits at the top of your Gulp file.  I included 3 to start with...

var gulp = require("gulp");

var util = require("gulp-util");

var concat = require("gulp-concat");

Gulp and gulp-util are essentially your basics, needed in every Gulp file.  I then found gulp-concat on NPM (Node Package Manager, regardless of what the top left corner of the site says!) and it looked like it would do the trick, so I included that too.

This is one of the big advantages to Gulp.  Because it runs in Node, you can use NPM to find and install packages, which allows you to maximise the code not written, but using some that someone else has written for you!

So now I create a simple Gulp task...

gulp.task("js",function(cb) {

  gulp.src(jsFiles)

    .pipe(concat("script.js"))

    .pipe(gulp.dest("build/js"));

});

I'd recommend reading up more on Gulp, but essentially this is taking an array of filenames stored in "jsFiles" and using it as the source, piping this through to the gulp-concat plugin which takes a parameter to name the resultant output file, and then setting the destination to be my build directory.  When I run this, it sucks in all the files, concatenates them, and then spits out a single file to my build directory.

I then modified the HTML so that instead of linking to lots of different javascript files in the <head> tag, I now reference just one javascript file, and I also do it at the end of the <body>.  I've also added a "defer" attribute, because it really doesn't need to be loaded upfront - this reduces the time till the First Meaningful Paint.

New development process with Git and Gulp

Whilst working on my new project, Privacy Tools, I decided that I really needed to sort out my development process.  Not that there was anything particularly wrong with it, it's just very old school and manual.

I literally write every line of code (HTML, PHP, CSS and Javascript) by hand, in Notepad++.  This doesn't even syntax highlight particularly well, let alone auto-complete.  And then the build process is also manual, going to websites like Closure Compiler to minify my javascript.  Rather labourious, especially for someone who doesn't get much time to code these days, and therefore needs every minute to go into writing code!

So I started by finally using Git!  I don't think anyone in the development community could have possibly not heard of Git, even if they haven't used it.  I'd played about with it, done some tutorials (courtesy of LinkedIn Learning), but never used it properly.  I figured now was the time, and I'm genuinely loving it.  It's great to know that everything is properly stored away, and I can easily track changes and get back to earlier versions of a file (or set of files) when needed.  It's also stopping me from wandering off and starting a new bit whilst I'm still in the middle of another, as this really confuses the commits - now I'm working on one distinct chunk, getting that working, committing it with a lovely comment, and then on to the next one.  Lovely!

However, that doesn't really improve my workflow or efficiency (unless I need to revert back to an earlier file version).  Where I've really had fun is with Gulp.  Gulp is...

a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something.

And it really does do that!  I did have a look at some others, such as the increasingly popular Webpack, but what I really like about Gulp is that the automation is written in javsacript, so can be completely customisable.  You can easily minify javascript files and stylesheets, concatenate files, and pretty much any else that you can think of.  

I'm not going to go into too much detail on this post, but my next one will show you the details of my first Gulp file.  I've already started adding more and more as I've gone along, so I'm sure there'll be many more posts to follow.

Project - Privacy Tools

One of the main reasons that I decided to have my own website, was to host some of the different projects that I've been working on. Some are in good working order (but never finished!) such as WCG Online, and others are in their infancy, like the one I'm talking about today.

Privacy Tools

Privacy matters - here's why!

A lot of internet users don't realise just how much websites can track them and invade their privacy. You can have security without privacy but you can't have privacy without security.  I have created the tools below to help show what websites can determine about you.

And that's the gist really, it is designed to be a set of tools to help educate users on internet privacy and how they can be more careful about the information that they give away.

It currently looks at things like your navigator details and history, canvas information, battery level, display and position information, and connection details.  It also looks at the major social networking sites to see which ones you're currently logged into.

As I say, this is very much a work in progress, and will continue to improve over the coming months.  

Check it out on my Projects page or directly at Privacy Tools.