Task to create webcontainer snapshots for project embeds #20
Description
playfulprogramming/playfulprogramming#1334 adds code embeds that build nested projects using the WebContainers API
In order to speed these up dramatically, we can preinstall any npm dependencies (with --ignore-scripts!) and create a binary snapshot of each container with @webcontainer/snapshot.
As another optimization, we should generate one combined webcontainer snapshot per post, with each project in a subdirectory. npm install would still need to be run separately for each subdirectory, but this means that browser installs/cache can be reused when switching embeds.
Additionally, once the snapshot is generated, try compressing it with gzip. I have no idea if @webcontainer/snapshot does this already - if not it would likely reduce the download size massively.
On the browser, the script can then re-run npm install --prefer-offline to skip a lot of downloads.
To summarize, this issue involves:
- Creating a worker task to generate webcontainer snapshots
- Given a post id, this should find its location on GitHub and fetch the project sources
- It then needs to place those contents in a temp dir, run
npm install, and generate the snapshot file
(make sure to clean up these dirs & npm cache afterwards!) - Upload the snapshot file to S3
- Store the S3 file ID in the database, alongside a hash of the project sources, keyed by the post id.
- Connecting the new worker task to the GitHub sync task (Webhook to import/sync content from GitHub with the db #7) to re-generate projects when their content is changed