Adding Browserify into the mix

Having already written my first gulp task, I was ready to get a bit more advanced.  My first task was to concatenate javascript files, including putting my local version of jQuery in front of my own javascript, so there was a single file to download, with all the required plugins before.  However, I'd already heard of Browserify, and wanted to see if this would work with Gulp (short answer - yes!).

Browserify is a tool which lets you "require" plugins in your javascript, and then processes these at build time (for example, in a Gulp task) to automatically include the plugins as well.  This would achieve the same end result as my current concatenation task, but without me having to keep and manually update local copies of the plugin files, as I could get Browserify to do the job of finding and including them.

I went through a few different iterations, all of which worked but were a bit messy for one reason or another (including using the gulp-browserify plugin, which is deprecated!) but I'm going to skip over those and skip straight to my final implementation.

So I needed to include a copy of extra plugins at the top of my Gulp file...

var browserify = require("browserify");

var source = require("vinyl-source-stream");

Then I updated my task to look this this...

gulp.task("js",function(cb) {

  browserify(jsFiles).bundle(),

    .pipe(source("script.js"))

    .pipe(gulp.dest("build/js"));

});

Notice that I am no longer using Gulp.Src to pick up the javascript files from the array, Browserify is now handling this, and I'm using the "bundle" method to concatenate them, instead of using gulp-concat.  During this "bundle" method, the javascript is also being processed to find any "require" statements.

For me, I needed to require jQuery and a few other bits, so at the top of my project javsacript file, I added the following lines...

var jQuery = require("jquery");

window.$ = window.jQuery = jQuery;

require("jquery.easing");

require("bootstrap/js/scrollspy");

As you can see, you can simply use the "require" method by itself, or you can assign the exported method to a variable.  In the case of jQuery, it needs to be stored globally in order for Bootstrap to load properly.  

I also went from including all of the Bootstrap library from a CDN to just including the component that I required, which saves on filesize!

It's now really easy to include new libraries and plugins into my javascript, by simply installing them using NPM, and including them using Browserify, as part of my automated Gulp task.

First Gulp task - concatenation

I recently started using a new development process, using Gulp to automate development tasks.  I thought I'd start off easy, so I'd concatenate my existing javascript files into a single file.

You might ask, why would you want to do this?  And my answer would be, you want to reduce the number of items fetched when loading your page, for optimal performance.

You might ask, why don't you do this manually once and then always work on the concatenated file?  And my answer would be, some of the files are plugins or libraries, which I want to keep separate so that I can easily update them.  I only want them to be combined in the build version.

So, questions out of the way, here's how it went down.

Firstly, you need to make sure you include the relevant bits at the top of your Gulp file.  I included 3 to start with...

var gulp = require("gulp");

var util = require("gulp-util");

var concat = require("gulp-concat");

Gulp and gulp-util are essentially your basics, needed in every Gulp file.  I then found gulp-concat on NPM (Node Package Manager, regardless of what the top left corner of the site says!) and it looked like it would do the trick, so I included that too.

This is one of the big advantages to Gulp.  Because it runs in Node, you can use NPM to find and install packages, which allows you to maximise the code not written, but using some that someone else has written for you!

So now I create a simple Gulp task...

gulp.task("js",function(cb) {

  gulp.src(jsFiles)

    .pipe(concat("script.js"))

    .pipe(gulp.dest("build/js"));

});

I'd recommend reading up more on Gulp, but essentially this is taking an array of filenames stored in "jsFiles" and using it as the source, piping this through to the gulp-concat plugin which takes a parameter to name the resultant output file, and then setting the destination to be my build directory.  When I run this, it sucks in all the files, concatenates them, and then spits out a single file to my build directory.

I then modified the HTML so that instead of linking to lots of different javascript files in the <head> tag, I now reference just one javascript file, and I also do it at the end of the <body>.  I've also added a "defer" attribute, because it really doesn't need to be loaded upfront - this reduces the time till the First Meaningful Paint.

New development process with Git and Gulp

Whilst working on my new project, Privacy Tools, I decided that I really needed to sort out my development process.  Not that there was anything particularly wrong with it, it's just very old school and manual.

I literally write every line of code (HTML, PHP, CSS and Javascript) by hand, in Notepad++.  This doesn't even syntax highlight particularly well, let alone auto-complete.  And then the build process is also manual, going to websites like Closure Compiler to minify my javascript.  Rather labourious, especially for someone who doesn't get much time to code these days, and therefore needs every minute to go into writing code!

So I started by finally using Git!  I don't think anyone in the development community could have possibly not heard of Git, even if they haven't used it.  I'd played about with it, done some tutorials (courtesy of LinkedIn Learning), but never used it properly.  I figured now was the time, and I'm genuinely loving it.  It's great to know that everything is properly stored away, and I can easily track changes and get back to earlier versions of a file (or set of files) when needed.  It's also stopping me from wandering off and starting a new bit whilst I'm still in the middle of another, as this really confuses the commits - now I'm working on one distinct chunk, getting that working, committing it with a lovely comment, and then on to the next one.  Lovely!

However, that doesn't really improve my workflow or efficiency (unless I need to revert back to an earlier file version).  Where I've really had fun is with Gulp.  Gulp is...

a toolkit for automating painful or time-consuming tasks in your development workflow, so you can stop messing around and build something.

And it really does do that!  I did have a look at some others, such as the increasingly popular Webpack, but what I really like about Gulp is that the automation is written in javsacript, so can be completely customisable.  You can easily minify javascript files and stylesheets, concatenate files, and pretty much any else that you can think of.  

I'm not going to go into too much detail on this post, but my next one will show you the details of my first Gulp file.  I've already started adding more and more as I've gone along, so I'm sure there'll be many more posts to follow.

Project - Privacy Tools

One of the main reasons that I decided to have my own website, was to host some of the different projects that I've been working on. Some are in good working order (but never finished!) such as WCG Online, and others are in their infancy, like the one I'm talking about today.

Privacy Tools

Privacy matters - here's why!

A lot of internet users don't realise just how much websites can track them and invade their privacy. You can have security without privacy but you can't have privacy without security.  I have created the tools below to help show what websites can determine about you.

And that's the gist really, it is designed to be a set of tools to help educate users on internet privacy and how they can be more careful about the information that they give away.

It currently looks at things like your navigator details and history, canvas information, battery level, display and position information, and connection details.  It also looks at the major social networking sites to see which ones you're currently logged into.

As I say, this is very much a work in progress, and will continue to improve over the coming months.  

Check it out on my Projects page or directly at Privacy Tools.

Going HTTPS with Cloudflare

Having already made my WCG Online project secure (as detailed in a previous post), it's been on my list for a while to do the same with my main website.  But I'd heard good things about Cloudflare and wanted to give them a try.

Well I'm glad I did, it was so easy and straight forward.  Here's how it went...

1. Go to Cloudflare to sign up
2. Enter your chosen email address and a new password
3. Add your website domain, so for me this was www.rik.onl
4. Click "Scan for DNS" and you'll get a little video to watch whilst it's processing
5. Click "Continue" once it's down
6. You'll then see a list of your DNS records, and which ones Cloudflare are planning to redirect.  Make sure this list is correct! 
7. Click "Continue" once you're happy
8. You then need to choose a plan - I was just testing so I went with Free website, but this will depend on what your website is used for and what level of service you require
9. Click "Continue" when you've selected
10. You'll then see a couple of new name servers - you'll need to go to your current web hosting platform and change the name servers over to the Cloudflare ones you've been given.
11. Click "Continue" once you've done this

That's it, you're done.  Give it some time for everything to update, and you'll have an HTTPS version of your site ready to go!

You'll see a number of different sections within Cloudflare that are available to configure.

Overview

That's where you should arrive by default.  This gives you a summary of the status of your account.

Analytics

Initially there won't be anything in here, but over time it will start to show the traffic that Cloudflare has handled and how much of your own web hosting bandwidth has been saved by their caching.

DNS

This shows you the DNS records again, and the nameservers, and from here you can make changes if needed, depending on your plan level.

Crypto

This is where the HTTPS magic happens!  By default it is set to "Full", which is good, and will give you the nice green "Secure" message in the address bar (on Chrome, at least).  You can also enable HTTP Strict Transport Security (HSTS) and Automatic HTTPS Rewrites, both of which I would definitely recommend to help ensure users find and continue to use the HTTPS version of your website.  For HSTS you get some sub-options, including the Maximum Age (max-age) which they recommend 6 months, which sounds about right to me.  You can also apply the setting to subdomains, preload (if you wish, but I haven't) and and the "no-sniff" header (X-Content-Type-Options: nosniff), which I would recommend.

Firewall

This section can be used to try and filter out "bad" traffic.  You can do things like rate limit per IP address, and change the challenge page settings, which is where a user gets sent if they are determined as "bad", so they get a chance to prove themselves to actually be "good".  I left all of this as default.

Speed

Here you get the option to do things to speed up your website, such as minifying (I selected Javascript, CSS and HTML).  A lot of these options require a paid plan, but it's worth having a look and seeing what works for you.

Caching

You can set a few different cache options in here, but there are also a couple of really important buttons in here.

1. Development Mode - if you are in the process of making and testing changes to your website, you're going to want to enable this, so that the cache is bypassed.
2. Purge Cache - once you've made the change to your website, you're going to want to then purge the Cloudflare cache, to ensure the changes come through, before you disable the Development Mode.

Page Rules

This allows you to change certain settings for different pages, so you can make exceptions.  I've not set any of these, but I think this could prove really useful moving forwards!

Network

Here you can make certain network decisions, such as whether web sockets should be able to link to your original server, and IPv4 vs. IPv6, etc..

Traffic

Here you can control and manage your traffic, as well as seeing any firewall events that have been triggered.

Customize

This allows you to change certain error messages (such as HTTP 429 - Too Many Requests), but none of the options here are available on the free plan.

Apps

You can enable extra functionality here, including...

• A Better Browser - warns users if their browser is old
• Google Analytics
• Google Webmaster Tools
• Infolinks - monetize your site with ads
• New Relic Browser - provides insight into the experience your site users are having
• OpenDyslexic - overrides site fonts with OpenDyslexic
• Trumpet - lets you put a dismissable message at the top of your site
• VigLink - helps you earn money from your site's outbound links automatically

Scrape Shield

Cloudflare can scrape your content in order to make certain changes, to help protect you.  For example, it can obfuscate email address, and prevent hotlinking.  

And that's it, all done!  You're ready to go, and they've done all the hard work for you.

There's also an API for all of these settings, so you can make changes automatically if you want to.  However, I've not looked at this side of things yet. 

So now I can run my newly secure website through securityheaders.io and see how I'm doing.  And I get an A!

The only reason that I don't get an A+ is that I'm missing is HTTP Public Key Pinning (HPKP).  This is generally a good thing to do, but can cause problems when using Cloudflare as you're never sure when they're going to update the certificates.  They've suggested waiting until they implement this natively, so I'm going to go with that advice.  To be honest, I don't have a login page or any ecommerce, so no one's going to gain anything by doing a rather complicated attack on my certificates, so there's no risk here for me.

From my experience, I can certainly recommend Cloudflare to anyone, especially if they're looking to make their website secure.

Response headers - adding Content-Security-Policy

I recently wrote an update as I continue to work on my response headers, in which I said that I was working on adding content-security-policy, with the help of Scott Helme, who has written a great blog post on this.  He has also created an excellent site called report-uri.io which has a number of tools, including one to help you build your CSP.

As a refresher, this is used to control what access different content on your site has.  For example, you can control what javascript and stylesheets can be included, inline or from different domains, etc.  This can help to limit the attack surface for things like cross-site scripting attacks.

So there are lots of parts, but here's how I went about it...


Default Source (default-src)

This is the default setting, so to be really secure, I went with "none".  This essentially means don't allow any content at all.  It is then overridden for each content type explicitly.

Script Source (script-src)

This is applied to all the javascript files and script tags on the page.  I started with "self", as I wanted everything from my site ("www.rik.onl") to work.  Then I looked through my code and added in the CDN domains that I was using to load the libraries I'm using... "https://code.jquery.com https://maxcdn.bootstrapcdn.com https://cdnjs.cloudflare.com" (space delimited).

Style Source (style-src)

This is applied to all stylesheet files and style tags on the page.  Again, I started with "self", and then needed to add "https://maxcdn.bootstrapcdn.com".

Image Source (image-src)

You guessed it, this is applied to all the images on the page.  This time I needed "self" and also "https://www.gravatar.com".

Font Source (font-src)

I don't load any fonts from my own site here, so I just need to add "https://maxcdn.bootstrapcdn.com".

Connect Source (connect-src)

This is applied to things like AJAX requests, web sockets and event sources.  I don't have any of these currently, so I've left this out, which means it will default to the Default Source, which is "none".  I could also explicitly set this to "none".

Media Source (media-src)

This is applied to audio and video tags.  I don't have any, so I've left this out.

Object Source (object-src)

This is applied to object, applet and embed tags.  I don't have any, so I've left this out.

Child Source (child-src) and Frame Source (frame-src - deprecated)

These are applied to frame and iframe tags.  I don't have any, so I've left these both out.  You should use Child Source instead of Frame Source though, as the later is deprecated.

Worked Source (worker-src)

This is applied to things like service workers, which I don't have, so I've left this out.

Frame Ancesters (frame-ancesters)

This is applied to frame and iframe tags, stating which parents may embed a page.  It replaces the "X-Frame-Options" header, but again, I've left it out.

Form Action (form-action)

This is applied to form tags, stating which location they can post to.  I don't have a form, so I've left it out.  In case you've forgotten, this means it defaults to the "Default Source", which is "none".

Upgrade Insecure Requests (upgrade-insecure-requests) and Block All Mixed Content (block-all-mixed-content)

My site currently runs on HTTP (non-secure), so I've left these out, but I'll definitely be looking to add them in moving forwards.

Reflected Cross-Site Scripting (reflected-xss)

Reflected cross-site scripting is bad!  I've set this to "block".  You can set this to "filter", which means the browser will try to cleanse the script, but I think it's safer to block it completely, if XSS is detected.

Manifest Source (manifest-src)

This is applied to manifest files, so I've set this to "self", as I have a manifest for icons.

Plugin Types (plugin-type)

This tells the browser which plugins they can invoke, such as the Adobe PDF Viewer.  As I don't think my site needs any, I've left this out.

Referrer (referrer)

This directive tells the browser when (and when not) to send the referer [sic] header when fetching content from another domain.  In my last blog post I decided to set the "Referrer-Policy" header to be "no-referrer-when-downgrade", and so I've set this directive to match that.

Report URI (report-uri)

This is where the browser will post violation reports to, so you can keep an eye on what content is breaking.  This could be because you've mis-configured your policy, or it could be because someone is trying to hack your site!  Scott Helme's excellent site report-uri.io will allow you to configure a URI which you can set here.


So now it's ready for testing!  

To test, instead of setting "Content-Security-Policy", set "Content-Security-Policy-Report-Only".  This means that the browser will report all of the violations, but will continue to load the content anyway, which is perfect until you're sure it's right.  

This is what I came up with...

default-src 'none'; script-src 'self' https://code.jquery.com https://maxcdn.bootstrapcdn.com https://cdnjs.cloudflare.com; style-src 'self' https://maxcdn.bootstrapcdn.com; img-src 'self' https://www.gravatar.com; font-src https://maxcdn.bootstrapcdn.com; upgrade-insecure-requests; block-all-mixed-content; reflected-xss block; manifest-src 'self'; referrer no-referrer-when-downgrade; report-uri https://rik.report-uri.io/r/default/csp/reportOnly;

I then reloaded my site, checked the console, and saw a number of violations!

This is because I'd forgotten a couple of included scripts that are loaded by Google Analytics.  In fact, they also use an inline script tag, but I really don't want to add "unsafe-inline" to the Script Source, otherwise all inline scripts could be run.  

Luckily you can fix inline script and style tags that you want to allow by calculating a SHA-256 hash of the contents and including this in the Script Source.  And Scott's got a hash generator tool as well, so that's easy.

Having fixed these problems, and reloading my site with no violations reported, I'm now ready to start enforcing.  This means that I need to set the "Content-Security-Policy" header, and if you're using report-uri.io then you need to update your Report URI directive as well.

Note that if you want to support IE11, you also need to set "X-Content-Security-Policy" to the same value.  It's not supported in older versions of IE at all.

Now that I've added these extra headers, you can check out my live report.  Last I checked, I got an A+ 😃

Response headers - an update

I previously wrote about what response headers I was sending back from my website - now I have an update.

Part of the problem was that extra headers were being sent, which I didn't particularly want to be sent.  So I've been working on getting rid of them.

X-Hostname

I believe this is added by my web host, but I managed to remove it by modifying my .htaccess file with the following...

  Header unset X-Hostname

X-Powered-By

This is added by PHP, but it was easily removed by modifying my php.ini file with the following...

  expose_php = Off

Server

I could not change this, unfortunately, due to the fact that I'm currently using shared hosting, and therefore don't have access.  But for others, whilst it can't be removed, it can be changed to minimise it's output, by adding the apache directives...

  ServerTokens ProductOnly
  ServerSignature Off

So now I've tidied that up a bit, I wanted to look at what else I should be adding.  I found an excellent site for this by Scott Helme called securityheaders.io.  You simply scan your site, and follow the advice it gives you.  

It warned me about the "Server" header, but I've already worked out I'm going to have to live with that.  Other headers it suggested that I add included...

Referrer-Policy

This is used to define what referrer information gets sent when someone clicks on a link on your site that goes to another site, or even a page within your own site.  Scott Helme has written a great blog post on this, in which he recommends going with "no-referrer-when-downgrade", which sounds good enough for me.  My site is currently shipped over HTTP, but when I move it to HTTPS (yes, this is the plan!) then it will ensure referrer information isn't passed on to HTTP sites.

Content-Security-Policy

This is used to control what access different content on your site has.  For example, you can control what javascript and stylesheets can be included, inline or from different domains, etc.  Again, Scott Helme has written another great blog post on this.  He has also created an excellent site called report-uri.io which has a number of tools, including one to help you build your CSP.  This is a bit more involved though, so I think I'll cover this in a separate post.