Le Nguyen The Dat bio photo

Le Nguyen The Dat

Data Science and Engineering at Uber. Kaggle Master.

Email Twitter Facebook LinkedIn Github

image

If you ever run into such error below when building your Docker image…

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/l/linux/linux-libc-dev
_3.13.0-55.94_amd64.deb  404  Not Found [IP: 91.189.91.14 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get install -y s3cmd postgresql wget build-essential unzip gawk'
returned a non-zero code: 100

It’s likely you had these as separated lines in your Dockerfile (before apt-get install commands):

RUN apt-get update
RUN apt-get install -y s3cmd postgresql wget

You should instead combine them in a single line, for example:

RUN apt-get update && apt-get install -y s3cmd postgresql wget

Here is why:

  • By default, Docker cache your commands to reduce time spent building images. Unless there was any change before such commands (or at the same line).
  • Linux distros are updated regularly, and it requires you to run apt-get update frequently otherwise apt-get install might not works as intended.

So, if you have built the image earlier (with a linux distro that recently had some updates), and you want to add some more libraries, for example:

RUN apt-get update
RUN apt-get install -y s3cmd postgresql wget build-essential unzip gawk

The apt-get update command will get bypassed since it’s already get cached, your package management (i.e apt-get) would not be up-to-date while you thought it should always be.

Note: After a few search, I found this issue on Docker repo: https://github.com/docker/docker/issues/1996 too bad it’s closed because there were not many real world use cases - they might be right though. Of course, we can always use --no-cache=true as an option for our docker build command, but it will build everything all over again everytime - you probably don’t want that.

Anyway, below are some few links that I find useful for my Docker use:

And that should be it.