fix: only have the stream close after writing Webpack filesystem caches

[pxpeterxu]

Motivation

I wanted to use webpack-stream with Webpack 5's filesystem caches so that we can improve incremental build performance when starting Webpack again. I used a Webpack configuration similar to:

const stream = webpack({
  config: {
    mode: 'development',
    output: {filename: 'bundle.js'},
    cache: {
      type: 'filesystem',
      idleTimeoutForInitialStore: 0,
    },
  }
});

I noticed that:

• When running npm run webpack or yarn webpack with this configuration, a cache directory would be created (in node_modules/.cache/webpack
• When running with webpack-stream (used in a Gulp script), the Gulp task would exit without creating a cache directory

After digging into error messages a bit, I realized that:

• webpack-stream calls const compiler = webpack(...) and compiler.run to start a compilation task

• Webpack's docs recommend that on finishing compilation, we run compiler.close to make sure that cache-writing happens

  

Changes

• When we're not watching, add a call to compiler.close and move the stream closing/callback calls to only after compiler.close finishes
  • This makes sure that we write any caches
• Add a test to make sure cache directory gets created
• Ensure Webpack 4 backwards compatibility by checking for compiler.close() presence

Testing

Added an automated test. Also, ran this with my (much larger) project and verified that this worked as expected