CC Helen

Dynamic Vendor Bundling in Webpack

If you aren’t already, there’s very little reason not to take advantage of code-splitting (bundle chunking) in Webpack. Especially for single-page apps, there’s no reason why the code that comprises the admin panel should load when a user is on the publicly viewable landing page, for example.

TL;DR

1. Use CommonChunksPlugin minChunks callback to dynamically chunk 3rd party dependencies into a vendor bundle.
2. Be sure to use the defer attribute on all entry chunk script tags.

Bundles vs Chunks

First, a very brief intro to code-splitting just so we’re all on the same page:

You may have heard these two terms used interchangeably, but they’re not quite the same. So let’s just quickly understand the difference between a “bundle” and a “chunk” before we get into the meat of this post.

In Webpack, there are actually three different types of chunks that can be emitted when compiling your code, but there’s only two that we’re concerned with:

1. Entry chunk: Contains runtime necessary to run your app. This is commonly referred to as a “bundle”. These need to be included on the page via <script /> tags.
2. Normal chunk: Does not contain any runtime code, instead it represents the modules generated per code-split points in your app. You do not need to reference these via <script /> tags, Webpack will dynamically load them for you as needed.

How to code-split

There’s been countless articles written about how to code-split, but the gist of it is like so (in Webpack 2, with react-router v3):

NOTE: This syntax is only possible with Webpack 2 (for System.import) and react-router v3 (so that the route can understand a Promise).

UPDATE: If you’re now using Webpack 3, the same pattern still works, just use import('./Foo') instead of System.import('./Foo').

And let’s say our Webpack entry looks like this:

{
  entry: { main: ‘./app.js’ },
  output: { 
    filename: ‘[name].js',
    chunkFilename: '[name]-[chunkhash].js', 
  }
}

With that, you’ll get 1 bundle, and 2 chunks:

1. Main app.js bundle composed of all the synchronously loaded dependencies (e.g. App, via static require() or import statements). Remember, you’ll need to include this in the HTML template of your app. This can be done for you if you use the HTMLWebpackPlugin.
2. A “Foo” chunk. Webpack automatically loads this when the user navigates to /foo.
3. A “Bar” chunk. Webpack automatically loads this when the user navigates to /bar.

This is great, because if Foo and Bar pages are not within a common user flow, we can avoid loading the extraneous page.

Taking it further with a vendor bundle

One common optimization people do with code-splitting is to define a Vendor bundle to include all the 3rd party libraries in a separate bundle that can be cached much longer than your main App bundle (because, presumably, you wouldn’t update your 3rd party dependencies nearly as much as you would update your app code).

However, I’ve often seen something like this prescribed:

{
  entry: { 
    main: ‘./app.js’,
    vendor: [‘react’, ‘react-router’, ...],
  },
  output: { 
    filename: '[name].js',
    chunkFilename: '[name]-[chunkhash].js', 
  }
}

While this works, it isn’t great because you’d need to manually keep this list of 3rd party dependencies up to date.

Another approach is to pull them from package.json directly:

const pkg = require('./package.json');
{
  entry: { 
    main: './app.js',
    vendor: Object.keys(pkg.dependencies),
  },
  output: { 
    filename: '[name].js',
    chunkFilename: '[name]-[chunkhash].js', 
  }
}

In one way this is slightly better because as we pull in new dependencies, they’ll automatically be added to our vendor bundle and we don’t have to manually keep this list up to date. However, this is arguably even worse because if we forget to remove a dependency that we’re no longer using in our app, it’ll still be included in our vendor bundle! This method also breaks down when writing isomorphic apps because then you’d have to manually supply an exclusion list to leave off things like express and so on.

Dynamically generating a vendor bundle

I recently came across this comment by Tobias Koppers (author of webpack) that shows an even better way:

It turns out, the CommonsChunkPlugin minChunks property accepts a callback that allows you to return a boolean for whether or not to include each module it comes across in the bundle. In the above code we’re chunking into a “vendor” bundle only when the path of the bundle is in node_modules. This way, we only include 3rd party libraries that we actually reference in our app into the vendor bundle.

This also has the added benefit of a more optimized bundle, since if you import like this:

import thing from 'lodash/thing';

You would only get thing and not the entire lodash library.

Aside: You probably don’t need lodash.

Ensuring consistently hashed bundle names

The next step is to ensure that we get the same hashed bundle name every time we build our app. This is so that we can cache the bundles on our server effectively. We should only get a new hash if the contents of the vendor bundle changes (e.g. if we added a new vendor lib, or upgraded an existing one).

The reason this doesn’t happen out of the box is because each bundle contains a mapping to the other chunks required to load the rest of the app. The key is to externalize this mapping into a “manifest” bundle.

There are apparently many ways to do this, but unfortunately getting it just right has proven difficult. There is currently a long-running “P1” issue in the webpack repo about this. Hopefully this will be made easier soon!

If you’ve found a working solution to this problem, please chime in on the issue in webpack! https://github.com/webpack/webpack/issues/1315

Update 10/4/2016

A few people have asked me what configuration worked for me, so here it is:

This seems to work, although it’s highly dependent on what your app’s entry looks like, but I hope this is helpful to some!

I also recommend that you inline the manifest bundle as it will likely be very small (~1KB, since it’s just a mapping of all the bundles) and you can save on the roundtrip. You can inline it by just reading it in on the server fs.readFileSync()and inserting it in a script tag before any of the entry bundles. You get the generated filenames from the JSON that assets-webpack-plugin creates.

Use defer for the entry bundles

One last word of advice: in your HTML template, be sure to use the defer attribute on the script tags for your bundles (as opposed to async or no attribute at all). This ensures that the browser will load the scripts in parallel, but still execute them in the order they appear in the document. Order matters for these bundles.

<script defer src="vendor.js"></script>
<script defer src="main.js"></script>

For more on async vs defer check out this great article: Growing with the web — async vs defer attributes.