Optimising javascript files

In this next great installment of my new development process I want to head back to looking at javascript files, as my last three posts have been about adding stylesheets in my process, lazy loading stylesheets that are non-critical and using resource hints (although technically that last one applies to any resources, not just stylesheets).

The last time I wrote about javascript files, I had them beautifully minified and ready to go.  My gulp task for javascript files currently looks like this...

gulp.task("js",function(cb) {
  pump([
    browserify(jsFiles).bundle(),
    source("script.js"),
    buffer(),
    uglify(),
    gulp.dest("build/js")
  ],cb);
});

If you have a lot of javascript, especially with immediately-invoked function expressions (IIFEs), it can be a good idea to optimise them.  This essentially works by indicating to the browser's javascript engine during pre-parsing that it can skip this function as it can be fully parsed later, and this stops it from being parsed twice.

For more details, I would definitely recommend checking out the Github page for the plugin I'm about to talk about, which is optimize-js.  Once again, there is a thin wrapper Gulp plugin, called gulp-optimize-js.  

Side note: The naming convention isn't a coincidence, it's actually a naming convention!

Anyway, so if you've been reading this series of posts, you've probably already picked up on the pattern here.  I installed the plugin from NPM, then require it at the top of my Gulp file...

var optimize = require("gulp-optimize-js");

And then I add the relevant call into my javascript Gulp task...

gulp.task("js",function(cb) {
  pump([
    browserify(jsFiles).bundle(),
    source("script.js"),
    buffer(),
    uglify(),
    optimize(),
    gulp.dest("build/js")
  ],cb);
});

It's very important to make sure that you call this plugin after you've called gulp-uglify, because minifying the code will remove the optimisation that has been added, as technically it's done by adding additional brackets into your code.  However, these extra characters should be worth it overall.

I've tried testing with and without this optimisation on my own website, and I can't tell the difference/  The reason for this is that I don't have enough javascript code, but maybe you do.  I like knowing that my development process is as good as it can be though, even if in this case, the results are not tangible.

This plugin is covered in my Skillshare course, Optimising your website: A development workflow with Git and Gulp. The relevant video is 12 - Optimising Javascript Files.

Lazy loading stylesheets using LoadCSS

In my last post, I talked about adding stylesheets into my Gulp file, part of my new development process.  The follow on to this for me was thinking about whether all of those stylesheets were really needed up front.  As I explained in that post, concatenating them and minifying them will certainly reduce the overall filesize, and the number of TCP connections (and therefore time), but what if some of this could be delayed until after the page had even loaded?

This is often referred to as lazy loading.  The Filament Group have created a great plugin for this called loadCSS, which can be found on NPM as fg-loadcss.  Their description of why you should be using it goes like this...

Referencing CSS stylesheets with link[rel=stylesheet] or @import causes browsers to delay page rendering while a stylesheet loads. When loading stylesheets that are not critical to the initial rendering of a page, this blocking behavior is undesirable. The new <link rel="preload">standard enables us to load stylesheets asynchronously, without blocking rendering, and loadCSS provides a JavaScript polyfill for that feature to allow it to work across browsers. Additionally, loadCSS offers a separate (and optional) JavaScript function for loading stylesheets dynamically.

The "preload" option is not well supported at all currently, so for the time being at least, a polyfill of this nature is definitely required.

As I'm already using Browserify in my javascript Gulp task, this is really easy to add into my javascript file.  Obviously I need to first install the fg-loadcss package, and then I can add the following lines of javascript...

  var loadcss = require("fg-loadcss");
  var reflink = $("head").children("link[rel=stylesheet]").get(-1);
  loadcss.loadCSS("https://fonts.googleapis.com/css?family=Indie+Flower",reflink);

This first requires the package (which Browserify will pull in), then finds a reference element (it will insert the <style> tag directly after this one) and then it calls the manual "loadCSS" function with the path to the stylesheet.  In my example, this is to a Google Font file.

This manual method is actually not part of Filament Groups recommended workflow, but I prefer it, as it keeps the code neat in my opinion, and runs it after my javascript is running, which I think is best for non-critical styles.  If you look on their Github repo, they do give other example usage though.

You can then remove the reference to the stylesheet from the <head> section, or better yet, move it to the very bottom of your page with a no-javascript fallback, like this...

<noscript><link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Indie+Flower"></noscript>

This means that even if javascript is disabled, your stylesheet (or font, in my case) will still load, which is great!

I didn't get the chance to cover this in my Optimising your website: A development workflow with Git and Gulp course on Skillshare, but I hope to add it into a future course.

Adding stylesheets into my Gulp file

I have been working on a series of blog post about my new development process, which so far has focused exclusively on javascript, including concatenation of javascript files, using Browserify to load jQuery and other javascript library files, and minifying (or uglifying) javascript files.  Next it's time to look at adding stylesheets.

Similar to javascript files, stylesheet files can be concatenated, in order to save round trips for individual files.  So the first place I started was copying the javascript task, rewriting it for stylesheet files, and stripping out everything but the call to gulp-concat - the same plugin can be used as it fill concatenate any files.  

gulp.task("css",function(cb) {
  pump([
    gulp.src(["css/*.css"]),
    concat("style.css"),
    gulp.dest("build/css")
  ]);
});

I think also creating a default Gulp task, in order to make it easy to call both my javascript and stylesheet tasks...

gulp.task("default",["js","css"]);

This means that when you run just "gulp" on the command line, it will automatically call the default task, which will then run through the array of related tasks - these are pre-requisites that are run first.  This list of pre-requisites can be set for any task, but is especially useful in this default mode.

Also similar to javascript files, stylesheet files can be minified (or uglified), in order to save bandwidth and download time.  However, this time a different plugin will be required, and the best I've found it called gulp-clean-css.  Like many Gulp plugins, this is actually a thin wrapper around another plugin, called clean-css.

First I added the new plugin to the top of my Gulp file...

var cleancss = require("gulp-clean-css");

And then I added the call to my stylesheet task...

gulp.task("css",function(cb) {
  pump([
    gulp.src(["css/*.css"]),
    concat("style.css"),
    cleancss(),
    gulp.dest("build/css")
  ]);
});

The script has two levels...

1. These operate on single properties only, and are mostly on by default. 
2. These operate on multiple properties at a time, including restructuring and reordering rules, but are off by default.

I have been using the default options for some time now, and these seems to serve me pretty well.  But if you want even more savings, you can play with the options, especially by activating the second level of optimisations.

These are also covered in my Skillshare course, Optimising your website: A development workflow with Git and Gulp.  The relevant videos are 15 - Concatenating Stylesheets and 16 - Minifying Stylesheets.

Minifying (or uglifying) javascript

Totally in keeping with my New Year's Resolution, here is a lovely new blog post!  And it's a continuation of my new development process.  The last post in the series detailed my switch to using pump instead of pipe in my Gulp task.

Today's post is about minifying javascript, which is sometimes called "uglifying".  The reason being that this beautifully crafted javascript snippet...

//jquery wrapped anonymous function
$(function() {
  //clickjacking protection
  if(self!==top) {
    top.location = self.location; //break out of frame
  }
});

...gets minified to this monstrosity...

$(function(){self!==top&&(top.location=self.location)});

As you can see, the comments and whitespace have all been stripped, as well as converting the if statement to the shorthand variant - a number of operations are performed as part of this process.

So why on earth would we want to do this, I hear you cry.  Well essentially because it makes the file size smaller (in some cases, considerably smaller).  In the example above, the character count has gone from 166 to 56, which is approximately a third of the size, but with exactly the same functionality.  This makes it both quicker and cheaper (if you're on any kind of data plan, such as a mobile phone) to download the file for the user, and also saves you bandwidth costs.  This can be better than gzip compression for some files, but even better yet, do both!

If you want to read further on why you should do this, you can read this blog post relating to Drupal.

The best way to achieve this is to use the gulp-uglify package. So I installed this plugin and then included it at the top of my Gulp file...

var uglify = require("gulp-uglify");

I then took the task that I wrote about in my last blog post in this series and added the highlighted line...

gulp.task("js",function(cb) {

  pump([ 

    browserify(jsFiles).bundle(),

    source("script.js"),

    uglify(), 

    gulp.dest("build/js")

      ],cb); 
    });

This doesn't actually work, so not quite that simple.  The problem is to do with the way that Gulp can handle files as streams or buffers.  In this case the browserify plugin outputs the files as streams, but the gulp-uglify package requires them as buffers.  

Luckily this is easily fixed using a plugin called vinyl-buffer, which once included like so...

var buffer = require("vinyl-buffer");

...can be added to the task so that it looks like this...

gulp.task("js",function(cb) {

  pump([ 

    browserify(jsFiles).bundle(),

    source("script.js"),

    buffer(),  

    uglify(), 

    gulp.dest("build/js")

      ],cb); 
    });

Now we have a process which minifies our files, improving the performance and user experience, as well as reducing bandwidth costs - win win!

This is also covered in my Skillshare course, Optimising your website: A development workflow with Git and Gulp.  The relevant video is 11 - Minifying Javascript Files.

New Year's resolution

I don't usually do New Year's resolutions, I figure that if you want to achieve something then you should set that goal straight away, rather than arbitrarily doing it once a year.  However, it's been almost 6 months since I put up a new blog post, and that's just not acceptable!

So, my New Year's resolution is... to update this blog at least once a week.  

I've been busy in the last 6 months, very busy!  Here are a few highlights...

I've created a new website for the lovely Emma Malik - world renowned comedian.  This was a really interesting project, working on someone else's personal site, to their requirements, but putting into practise the security and performance tricks I'd learnt working on my own site, and my many years of professional experience.

I've been freelancing in my spare time on PeoplePerHour.  This has been great fun, as I love taking on new projects and solving problems, and some of the projects have already pushed me outside of my comfort zone - I love learning new things!

I've been getting into Wordpress development a lot, largely as part of the freelancing.  As an application, it's come a long way from when I'd previously looked at it, many years ago.  I'm liking it a lot, and may look at migrating this blog over, when I get more time to think through the logistics of it.

I've been working on and published my first course on Skillshare.  This is entitled Optimising your website: A development workflow with Git and Gulp, and it's based on the series of blog posts that I started to write, about my New development process with Git and Gulp and First Gulp task - concatenation.  It goes through many more steps through the process that I've built, but also, doesn't include everything.  I will be looking to create more courses to go into further detail with optimisation, and also a security-focused workflow course.  I plan to retroactively blog about each part as well, so look out for those posts over the next few weeks.  Should help me keep my resolution for January at least!

Looking forward to a productive and eventful 2018!  I hope you are too.

Using pump instead of pipe

On thing that I quickly discovered whilst using Gulp to automate my development process, was the piping the results through from one plugin to another could often go wrong, and it was often hard to trap the reason and get proper error messaging out.

Then I discovered some example code which pointed me to pump.  Pump is...

a small node module that pipes streams together and destroys all of them if one of them closes

Essentially it allows you to restructure your Gulp task in a way that handles multiple plugins much better, and it fails more gracefully.

So I needed to include a copy of the new plugin at the top of my Gulp file...

var pump = require("pump");

Then I took the task which I wrote about in my last post...

gulp.task("js",function(cb) {

  browserify(jsFiles).bundle(),

    .pipe(source("script.js"))

    .pipe(gulp.dest("build/js"));

});

And re-wrote it to look like this...

gulp.task("js",function(cb) {

  pump([ 

    browserify(jsFiles).bundle(),

    source("script.js"),

    gulp.dest("build/js")

      ],cb); 
    });

As you can see, instead of using the "pipe" method to pass the files/content from one plugin to another, I am now passing an array of calls into pump and this is doing the hard work for me.

I have found that this makes the error messaging much more useful when I have syntax errors or plugins fail to work as expected.  It's certainly saving me a lot of time when it comes to debugging and fixing my Gulp file.

Adding Browserify into the mix

Having already written my first gulp task, I was ready to get a bit more advanced.  My first task was to concatenate javascript files, including putting my local version of jQuery in front of my own javascript, so there was a single file to download, with all the required plugins before.  However, I'd already heard of Browserify, and wanted to see if this would work with Gulp (short answer - yes!).

Browserify is a tool which lets you "require" plugins in your javascript, and then processes these at build time (for example, in a Gulp task) to automatically include the plugins as well.  This would achieve the same end result as my current concatenation task, but without me having to keep and manually update local copies of the plugin files, as I could get Browserify to do the job of finding and including them.

I went through a few different iterations, all of which worked but were a bit messy for one reason or another (including using the gulp-browserify plugin, which is deprecated!) but I'm going to skip over those and skip straight to my final implementation.

So I needed to include a copy of extra plugins at the top of my Gulp file...

var browserify = require("browserify");

var source = require("vinyl-source-stream");

Then I updated my task to look this this...

gulp.task("js",function(cb) {

  browserify(jsFiles).bundle(),

    .pipe(source("script.js"))

    .pipe(gulp.dest("build/js"));

});

Notice that I am no longer using Gulp.Src to pick up the javascript files from the array, Browserify is now handling this, and I'm using the "bundle" method to concatenate them, instead of using gulp-concat.  During this "bundle" method, the javascript is also being processed to find any "require" statements.

For me, I needed to require jQuery and a few other bits, so at the top of my project javsacript file, I added the following lines...

var jQuery = require("jquery");

window.$ = window.jQuery = jQuery;

require("jquery.easing");

require("bootstrap/js/scrollspy");

As you can see, you can simply use the "require" method by itself, or you can assign the exported method to a variable.  In the case of jQuery, it needs to be stored globally in order for Bootstrap to load properly.  

I also went from including all of the Bootstrap library from a CDN to just including the component that I required, which saves on filesize!

It's now really easy to include new libraries and plugins into my javascript, by simply installing them using NPM, and including them using Browserify, as part of my automated Gulp task.

First Gulp task - concatenation

I recently started using a new development process, using Gulp to automate development tasks.  I thought I'd start off easy, so I'd concatenate my existing javascript files into a single file.

You might ask, why would you want to do this?  And my answer would be, you want to reduce the number of items fetched when loading your page, for optimal performance.

You might ask, why don't you do this manually once and then always work on the concatenated file?  And my answer would be, some of the files are plugins or libraries, which I want to keep separate so that I can easily update them.  I only want them to be combined in the build version.

So, questions out of the way, here's how it went down.

Firstly, you need to make sure you include the relevant bits at the top of your Gulp file.  I included 3 to start with...

var gulp = require("gulp");

var util = require("gulp-util");

var concat = require("gulp-concat");

Gulp and gulp-util are essentially your basics, needed in every Gulp file.  I then found gulp-concat on NPM (Node Package Manager, regardless of what the top left corner of the site says!) and it looked like it would do the trick, so I included that too.

This is one of the big advantages to Gulp.  Because it runs in Node, you can use NPM to find and install packages, which allows you to maximise the code not written, but using some that someone else has written for you!

So now I create a simple Gulp task...

gulp.task("js",function(cb) {

  gulp.src(jsFiles)

    .pipe(concat("script.js"))

    .pipe(gulp.dest("build/js"));

});

I'd recommend reading up more on Gulp, but essentially this is taking an array of filenames stored in "jsFiles" and using it as the source, piping this through to the gulp-concat plugin which takes a parameter to name the resultant output file, and then setting the destination to be my build directory.  When I run this, it sucks in all the files, concatenates them, and then spits out a single file to my build directory.

I then modified the HTML so that instead of linking to lots of different javascript files in the <head> tag, I now reference just one javascript file, and I also do it at the end of the <body>.  I've also added a "defer" attribute, because it really doesn't need to be loaded upfront - this reduces the time till the First Meaningful Paint.

New development process with Git and Gulp

Whilst working on my new project, Privacy Tools, I decided that I really needed to sort out my development process.  Not that there was anything particularly wrong with it, it's just very old school and manual.

I literally write every line of code (HTML, PHP, CSS and Javascript) by hand, in Notepad++.  This doesn't even syntax highlight particularly well, let alone auto-complete.  And then the build process is also manual, going to websites like Closure Compiler to minify my javascript.  Rather labourious, especially for someone who doesn't get much time to code these days, and therefore needs every minute to go into writing code!

So I started by finally using Git!  I don't think anyone in the development community could have possibly not heard of Git, even if they haven't used it.  I'd played about with it, done some tutorials (courtesy of LinkedIn Learning), but never used it properly.  I figured now was the time, and I'm genuinely loving it.  It's great to know that everything is properly stored away, and I can easily track changes and get back to earlier versions of a file (or set of files) when needed.  It's also stopping me from wandering off and starting a new bit whilst I'm still in the middle of another, as this really confuses the commits - now I'm working on one distinct chunk, getting that working, committing it with a lovely comment, and then on to the next one.  Lovely!

However, that doesn't really improve my workflow or efficiency (unless I need to revert back to an earlier file version).  Where I've really had fun is with Gulp.  Gulp is...

a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something.

And it really does do that!  I did have a look at some others, such as the increasingly popular Webpack, but what I really like about Gulp is that the automation is written in javsacript, so can be completely customisable.  You can easily minify javascript files and stylesheets, concatenate files, and pretty much any else that you can think of.  

I'm not going to go into too much detail on this post, but my next one will show you the details of my first Gulp file.  I've already started adding more and more as I've gone along, so I'm sure there'll be many more posts to follow.