tar fails extracting slug with resource limits set

[Cryptophobia]

From @drock on January 24, 2017 15:11

I am seeing a strange problem when trying to deploy my app with resource limits set. With no resource limits set, everything works fine. However, when I set the resource limits of the app below a certain value, it fails to launch

deis limits:set web=512M

After doing so kubectl will show the new pod in a CrashLoopBackoff and the logs of the pod show

tar: ./.heroku/node/bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge: Directory renamed before its status could be extracted
tar: ./client/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules: Directory renamed before its status could be extracted
tar: ./client: Directory renamed before its status could be extracted
tar: ./bin: Directory renamed before its status could be extracted
tar: Exiting with failure status due to previous errors

This is all strange as the error message seem to be unrelated to memory limits. The compressed slug size is ~150M and the uncompressed app folder tree is:

deis run 'du -sh'
Running 'du -sh'...
747M	.

Bumping the memory limit up to only 600M solves the problem though.

Related, but probably separate is that if I drop the memory limit even lower to 400M then the failure happens when trying to merely download the slug with only this in the logs

/bin/get_object: line 13:     9 Killed                  objstorage --storage-type="$BUILDER_STORAGE" download "$SLUG_URL" "$GET_PATH"

My app only needs ~300M to run so I was hoping to not set the kubernetes limits so high.

Copied from original issue: deis/slugrunner#61

[Cryptophobia]

From @bacongobbler on January 24, 2017 16:52

I wonder if deis/controller#1179 might resolve this by proxy as it will be using a separate init container to pull down the slug.

[Cryptophobia]

From @drock on January 24, 2017 17:40

That sounds plausible since only the memory limit seems to trigger this issue and that would download the slug and extract it before those limits are enforced?

[Cryptophobia]

From @mboersma on January 25, 2017 15:19

This seems similar to docker/hub-feedback#727.