Fix syslog-ng.

The system() source in the default syslog-ng config file does
not work in Docker because it tries to access /proc/kmsg,
which is not allowed due to Docker's privileges. We replace
it by a source that only reads from /dev/log.

Changelog.md

@@ -1,3 +1,29 @@
+## 0.9.10 (release date: 2014-05-12)
+
+ * Upgraded to Ubuntu 14.04 (Trusty). We will no longer release images based on 12.04.
+   Thanks to contributions by mpeterson, Paul Jimenez, Santiago M. Mola and Kingdon Barrett.
+ * Fixed a problem with my_init not correctly passing child processes' exit status. Fixes GH-45.
+ * When reading environment variables from /etc/container_environment, the trailing newline (if any) is ignored. This makes commands like this work, without unintentially adding a newline to the environment variable value:
+
+        echo my_value > /etc/container_environment/FOO
+
+   If you intended on adding a newline to the value, ensure you have *two* trailing newlines:
+
+        echo -e "my_value\n" > /etc/container_environment/FOO
+ * It was not possible to use `docker run -e` to override environment variables defined in /etc/container_environment. This has been fixed (GH-52). Thanks to Stuart Campbell for reporting this bug.
+
+## 0.9.9 (release date: 2014-03-25)
+
+ * Fixed a problem with rssh. (Slawomir Chodnicki)
+ * The `INITRD` environment variable is now set in the container by default. This prevents updates to the `initramfs` from running grub or lilo.
+ * The `ischroot` tool in Ubuntu has been modified to always return true. This prevents updates to the `initscripts` package from breaking /dev/shm.
+ * Various minor bug fixes, improvements and typo corrections. (Felix Hummel, Laurent Sarrazin, Dung Quang, Amir Gur)
+
+## 0.9.8 (release date: 2014-02-26)
+
+ * Fixed a regression in `my_init` which causes it to delete environment variables passed from Docker.
+ * Fixed `my_init` not properly forcing Runit to shut down if Runit appears to refuse to respond to SIGTERM.
+
 ## 0.9.7 (release date: 2014-02-25)
 
  * Improved and fixed bugs in `my_init` (Thomas LÉVEIL):

Makefile

@@ -1,12 +1,12 @@
 NAME = phusion/baseimage
-VERSION = 0.9.7
+VERSION = 0.9.10
 
 .PHONY: all build test tag_latest release ssh
 
 all: build
 
 build:
-	docker build -t $(NAME):$(VERSION) -rm image
+	docker build -t $(NAME):$(VERSION) --rm image
 
 test:
 	env NAME=$(NAME) VERSION=$(VERSION) ./test/runner.sh
@@ -15,7 +15,7 @@ tag_latest:
 	docker tag $(NAME):$(VERSION) $(NAME):latest
 
 release: test tag_latest
-	@if ! docker images phusion/baseimage | awk '{ print $$2 }' | grep -q -F $(VERSION); then echo "$(NAME) version $(VERSION) is not yet built. Please run 'make build'"; false; fi
+	@if ! docker images $(NAME) | awk '{ print $$2 }' | grep -q -F $(VERSION); then echo "$(NAME) version $(VERSION) is not yet built. Please run 'make build'"; false; fi
 	docker push $(NAME)
 	@echo "*** Don't forget to create a tag. git tag rel-$(VERSION) && git push origin rel-$(VERSION)"
 

README.md

@@ -66,7 +66,7 @@ You can configure the stock `ubuntu` image yourself from your Dockerfile, so why
 
 | Component        | Why is it included? / Remarks |
 | ---------------- | ------------------- |
-| Ubuntu 12.04 LTS | The base system. |
+| Ubuntu 14.04 LTS | The base system. |
 | A **correct** init process | According to the Unix process model, [the init process](https://en.wikipedia.org/wiki/Init) -- PID 1 -- inherits all [orphaned child processes](https://en.wikipedia.org/wiki/Orphan_process) and must [reap them](https://en.wikipedia.org/wiki/Wait_(system_call)). Most Docker containers do not have an init process that does this correctly, and as a result their containers become filled with [zombie processes](https://en.wikipedia.org/wiki/Zombie_process) over time. <br><br>Furthermore, `docker stop` sends SIGTERM to the init process, which is then supposed to stop all services. Unfortunately most init systems don't do this correctly within Docker since they're built for hardware shutdowns instead. This causes processes to be hard killed with SIGKILL, which doesn't give them a chance to correctly deinitialize things. This can cause file corruption. <br><br>Baseimage-docker comes with an init process `/sbin/my_init` that performs both of these tasks correctly. |
 | Fixes APT incompatibilities with Docker | See https://github.com/dotcloud/docker/issues/1024. |
 | syslog-ng | A syslog daemon is necessary so that many services - including the kernel itself - can correctly log to /var/log/syslog. If no syslog daemon is running, a lot of important messages are silently swallowed. <br><br>Only listens locally. |
@@ -90,7 +90,7 @@ Baseimage-docker *encourages* multiple processes through the use of runit.
 
 To look around in the image, run:
 
-    docker run -rm -t -i phusion/baseimage /sbin/my_init -- bash -l
+    docker run --rm -t -i phusion/baseimage /sbin/my_init -- bash -l
 
 You don't have to download anything manually. The above command will automatically pull the baseimage-docker image from the Docker registry.
 
@@ -131,7 +131,7 @@ You can add additional daemons (e.g. your own app) to the image by creating runi
 
 The shell script must be called `run`, must be executable, and is to be placed in the directory `/etc/service/<NAME>`.
 
-Here's an example showing you how to a memached server runit entry can be made.
+Here's an example showing you how a memached server runit entry can be made.
 
     ### In memcached.sh (make sure this file is chmod +x):
     #!/bin/sh
@@ -227,7 +227,7 @@ During startup, before running any [startup scripts](#running_startup_scripts),
 
 For example, here's how you can define an environment variable from your Dockerfile:
 
-    RUN echo -n Apachai Hopachai > /etc/container_environment/MY_NAME
+    RUN echo Apachai Hopachai > /etc/container_environment/MY_NAME
 
 You can verify that it works, as follows:
 
@@ -237,6 +237,12 @@ You can verify that it works, as follows:
     # echo $MY_NAME
     Apachai Hopachai
 
+**Handling newlines**
+
+If you've looked carefully, you'll notice that the 'echo' command actually prints a newline. Why does $MY_NAME not contain a newline then? It's because `my_init` strips the trailing newline, if any. If you intended on the value having a newline, you should add *another* newline, like this:
+
+    RUN echo -e "Apachai Hopachai\n" > /etc/container_environment/MY_NAME
+
 <a name="envvar_dumps"></a>
 #### Environment variable dumps
 
@@ -313,7 +319,7 @@ Once you have the ID, look for its IP address with:
 Now SSH into the container as follows:
 
     curl -o insecure_key -fSL https://github.com/phusion/baseimage-docker/raw/master/image/insecure_key
-    chmod 700 insecure_key
+    chmod 600 insecure_key
     ssh -i insecure_key root@<IP address>
 
 <a name="enabling_the_insecure_key_permanently"></a>

Vagrantfile

@@ -1,31 +1,54 @@
 # -*- mode: ruby -*-
 # vi: set ft=ruby :
-ROOT = File.dirname(File.expand_path(__FILE__))
+ROOT = File.dirname(File.absolute_path(__FILE__))
 
 # Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
-VAGRANTFILE_API_VERSION = "2"
+VAGRANTFILE_API_VERSION = '2'
+
+# Default env properties which can be overridden
+# Example overrides:
+#   echo "ENV['PASSENGER_DOCKER_PATH'] ||= '../../phusion/passenger-docker'   " >> ~/.vagrant.d/Vagrantfile
+#   echo "ENV['BASE_BOX_URL']          ||= 'd\:/dev/vm/vagrant/boxes/phusion/'" >> ~/.vagrant.d/Vagrantfile
+BASE_BOX_URL          = ENV['BASE_BOX_URL']    || 'https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/'
+VAGRANT_BOX_URL       = ENV['VAGRANT_BOX_URL'] || BASE_BOX_URL + 'ubuntu-14.04-amd64-vbox.box'
+VMWARE_BOX_URL        = ENV['VMWARE_BOX_URL']  || BASE_BOX_URL + 'ubuntu-14.04-amd64-vmwarefusion.box'
+BASEIMAGE_PATH        = ENV['BASEIMAGE_PATH' ] || '.'
+PASSENGER_DOCKER_PATH = ENV['PASSENGER_PATH' ] || '../passenger-docker'
+DOCKERIZER_PATH       = ENV['DOCKERIZER_PATH'] || '../dockerizer'
+
+$script = <<SCRIPT
+wget -q -O - https://get.docker.io/gpg | apt-key add -
+echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list
+apt-get update -qq
+apt-get install -q -y --force-yes lxc-docker
+usermod -a -G docker vagrant
+docker version
+su - vagrant -c 'echo alias d=docker >> ~/.bash_aliases'
+SCRIPT
 
 Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
-  config.vm.box = "phusion-open-ubuntu-12.04-amd64"
+  config.vm.box = 'phusion-open-ubuntu-14.04-amd64'
-  config.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/ubuntu-12.04.3-amd64-vbox.box"
+  config.vm.box_url = VAGRANT_BOX_URL
   config.ssh.forward_agent = true
-  if File.directory?("#{ROOT}/../passenger-docker")
+  passenger_docker_path = File.absolute_path(PASSENGER_DOCKER_PATH, ROOT)
-    config.vm.synced_folder File.expand_path("#{ROOT}/../passenger-docker"),
+  if File.directory?(passenger_docker_path)
-      "/vagrant/passenger-docker"
+    config.vm.synced_folder passenger_docker_path, '/vagrant/passenger-docker'
+  end
+  baseimage_path = File.absolute_path(BASEIMAGE_PATH, ROOT)
+  if File.directory?(baseimage_path)
+    config.vm.synced_folder baseimage_path, "/vagrant/baseimage-docker"
+  end
+  dockerizer_path = File.absolute_path(DOCKERIZER_PATH, ROOT)
+  if File.directory?(dockerizer_path)
+    config.vm.synced_folder dockerizer_path, '/vagrant/dockerizer'
   end
 
   config.vm.provider :vmware_fusion do |f, override|
-    override.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/ubuntu-12.04.3-amd64-vmwarefusion.box"
+    override.vm.box_url = VMWARE_BOX_URL
-    f.vmx["displayName"] = "baseimage-docker"
+    f.vmx['displayName'] = 'baseimage-docker'
   end
 
   if Dir.glob("#{File.dirname(__FILE__)}/.vagrant/machines/default/*/id").empty?
-    # Add lxc-docker package
+    config.vm.provision :shell, :inline => $script
-    pkg_cmd = "wget -q -O - https://get.docker.io/gpg | apt-key add -;" \
-      "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list;" \
-      "apt-get update -qq; apt-get install -q -y --force-yes lxc-docker; "
-    # Add vagrant user to the docker group
-    pkg_cmd << "usermod -a -G docker vagrant; "
-    config.vm.provision :shell, :inline => pkg_cmd
   end
 end

image/Dockerfile

@@ -1,4 +1,4 @@
-FROM ubuntu:12.04
+FROM ubuntu:14.04
 MAINTAINER Phusion <info@phusion.nl>
 
 ENV HOME /root

image/cleanup.sh

@@ -6,5 +6,7 @@ set -x
 apt-get clean
 rm -rf /build
 rm -rf /tmp/* /var/tmp/*
+rm -rf /var/lib/apt/lists/*
+rm -f /etc/dpkg/dpkg.cfg.d/02apt-speedup
 
 rm -f /etc/ssh/ssh_host_*

image/config/sshd_config

@@ -123,7 +123,7 @@ ChallengeResponseAuthentication no
 #Banner none
 
 # override default of no subsystems
-Subsystem	sftp	/usr/lib/sftp-server
+Subsystem	sftp	/usr/lib/openssh/sftp-server
 
 # Example of overriding settings on a per-user basis
 #Match User anoncvs

image/config/syslog_ng_default

@@ -9,5 +9,4 @@
 #CONSOLE_LOG_LEVEL=1
 
 # Command line options to syslog-ng
-# We set --default-modules because of https://github.com/phusion/baseimage-docker/pull/7.
+SYSLOGNG_OPTS="--no-caps"
-SYSLOGNG_OPTS="--no-caps --default-modules=affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat"

image/enable_insecure_key

@@ -21,7 +21,7 @@ else
 		|                                                                              |
 		| DO NOT expose port 22 on the Internet unless you know what you are doing!    |
 		|                                                                              |
-		| Use the private key bellow to connect with user root                         |
+		| Use the private key below to connect with user root                         |
 		+------------------------------------------------------------------------------+
 
 	EOF

image/my_init

@@ -1,5 +1,5 @@
-#!/usr/bin/python2 -u
+#!/usr/bin/python3 -u
-import os, os.path, sys, stat, signal, errno, argparse, time, json, re, posixfile
+import os, os.path, sys, stat, signal, errno, argparse, time, json, re
 
 KILL_PROCESS_TIMEOUT = 5
 KILL_ALL_PROCESSES_TIMEOUT = 5
@@ -54,16 +54,21 @@ def is_exe(path):
 	except OSError:
 		return False
 
-def import_envvars():
+def import_envvars(clear_existing_environment = True, override_existing_environment = True):
 	new_env = {}
 	for envfile in listdir("/etc/container_environment"):
 		name = os.path.basename(envfile)
 		with open("/etc/container_environment/" + envfile, "r") as f:
-			value = f.read()
+			# Text files often end with a trailing newline, which we
+			# don't want to include in the env variable value. See
+			# https://github.com/phusion/baseimage-docker/pull/49
+			value = re.sub('\n\Z', '', f.read())
 		new_env[name] = value
-	os.environ.clear()
+	if clear_existing_environment:
+		os.environ.clear()
 	for name, value in new_env.items():
-		os.environ[name] = value
+		if override_existing_environment or not name in os.environ:
+			os.environ[name] = value
 
 def export_envvars(to_dir = True):
 	shell_dump = ""
@@ -93,6 +98,13 @@ def shquote(s):
 def waitpid_reap_other_children(pid):
 	done = False
 	status = None
+	try:
+		this_pid, status = os.waitpid(pid, os.WNOHANG)
+	except OSError as e:
+		if e.errno == errno.ECHILD or e.errno == errno.ESRCH:
+			return None
+		else:
+			raise
 	while not done:
 		this_pid, status = os.waitpid(-1, 0)
 		done = this_pid == pid
@@ -134,7 +146,10 @@ def run_command_killable(*argv):
 		stop_child_process(filename, pid)
 		raise
 	if status != 0:
-		error("%s failed with exit code %d\n" % (filename, status))
+		if status is None:
+			error("%s exited with unknown status\n" % filename)
+		else:
+			error("%s failed with status %d\n" % (filename, os.WEXITSTATUS(status)))
 		sys.exit(1)
 
 def run_command_killable_and_import_envvars(*argv):
@@ -213,7 +228,7 @@ def install_insecure_key():
 	run_command_killable("/usr/sbin/enable_insecure_key")
 
 def main(args):
-	import_envvars()
+	import_envvars(False, False)
 	export_envvars()
 
 	if args.enable_insecure_key:
@@ -228,23 +243,34 @@ def main(args):
 	if not args.skip_runit:
 		runit_pid = start_runit()
 	try:
+		exit_status = None
 		if len(args.main_command) == 0:
 			runit_exited, exit_code = wait_for_runit_or_interrupt(runit_pid)
 			if runit_exited:
-				info("Runit exited with code %d" % exit_code)
+				if exit_code is None:
+					info("Runit exited with unknown status")
+					exit_status = 1
+				else:
+					exit_status = os.WEXITSTATUS(exit_code)
+					info("Runit exited with status %d" % exit_status)
 		else:
 			info("Running %s..." % " ".join(args.main_command))
 			pid = os.spawnvp(os.P_NOWAIT, args.main_command[0], args.main_command)
 			try:
 				exit_code = waitpid_reap_other_children(pid)
-				info("%s exited with exit code %d." % (args.main_command[0], exit_code))
+				if exit_code is None:
+					info("%s exited with unknown status." % args.main_command[0])
+					exit_status = 1
+				else:
+					exit_status = os.WEXITSTATUS(exit_code)
+					info("%s exited with status %d." % (args.main_command[0], exit_status))
 			except KeyboardInterrupt:
 				stop_child_process(args.main_command[0], pid)
 			except BaseException as s:
 				warn("An error occurred. Aborting.")
 				stop_child_process(args.main_command[0], pid)
 				raise
-		sys.exit(exit_code)
+		sys.exit(exit_status)
 	finally:
 		if not args.skip_runit:
 			shutdown_runit_services()

image/prepare.sh

@@ -3,22 +3,38 @@ set -e
 source /build/buildconfig
 set -x
 
-## Enable Ubuntu Universe.
+## Temporarily disable dpkg fsync to make building faster.
-echo deb http://archive.ubuntu.com/ubuntu precise main universe > /etc/apt/sources.list
+echo force-unsafe-io > /etc/dpkg/dpkg.cfg.d/02apt-speedup
-echo deb http://archive.ubuntu.com/ubuntu precise-updates main universe >> /etc/apt/sources.list
-apt-get update
 
-## Install HTTPS support for APT.
+## Prevent initramfs updates from trying to run grub and lilo.
-$minimal_apt_get_install apt-transport-https
+## https://journal.paul.querna.org/articles/2013/10/15/docker-ubuntu-on-rackspace/
+## http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=594189
+export INITRD=no
+mkdir -p /etc/container_environment
+echo -n no > /etc/container_environment/INITRD
+
+## Enable Ubuntu Universe and Multiverse.
+sed -i 's/^#\s*\(deb.*universe\)$/\1/g' /etc/apt/sources.list
+sed -i 's/^#\s*\(deb.*multiverse\)$/\1/g' /etc/apt/sources.list
+apt-get update
 
 ## Fix some issues with APT packages.
 ## See https://github.com/dotcloud/docker/issues/1024
 dpkg-divert --local --rename --add /sbin/initctl
 ln -sf /bin/true /sbin/initctl
 
+## Replace the 'ischroot' tool to make it always return true.
+## Prevent initscripts updates from breaking /dev/shm.
+## https://journal.paul.querna.org/articles/2013/10/15/docker-ubuntu-on-rackspace/
+## https://bugs.launchpad.net/launchpad/+bug/974584
+dpkg-divert --local --rename --add /usr/bin/ischroot
+ln -sf /bin/true /usr/bin/ischroot
+
+## Install HTTPS support for APT.
+$minimal_apt_get_install apt-transport-https
+
 ## Upgrade all packages.
-echo "initscripts hold" | dpkg --set-selections
+apt-get dist-upgrade -y --no-install-recommends
-apt-get upgrade -y --no-install-recommends
 
 ## Fix locale.
 $minimal_apt_get_install language-pack-en

image/runit/syslog-ng

@@ -1,6 +1,11 @@
 #!/bin/sh
 set -e
 
+# If /dev/log is either a named pipe or it was placed there accidentally,
+# e.g. because of the issue documented at https://github.com/phusion/baseimage-docker/pull/25,
+# then we remove it.
+if [ ! -S /dev/log ]; then rm -f /dev/log; fi
+
 SYSLOGNG_OPTS=""
 
 [ -r /etc/default/syslog-ng ] && . /etc/default/syslog-ng
@@ -19,6 +24,8 @@ esac
 if [ ! -e /dev/xconsole ]
 then
   mknod -m 640 /dev/xconsole p
+  chown root:adm
+  [ -x /sbin/restorecon ] && /sbin/restorecon $XCONSOLE
 fi
 
 exec syslog-ng -F -p /var/run/syslog-ng.pid $SYSLOGNG_OPTS

image/setuser

@@ -1,4 +1,4 @@
-#!/usr/bin/python2
+#!/usr/bin/python3
 import sys, os, pwd
 
 if len(sys.argv) < 3:

image/system_services.sh

@@ -21,6 +21,9 @@ mkdir /etc/service/syslog-ng
 cp /build/runit/syslog-ng /etc/service/syslog-ng/run
 mkdir -p /var/lib/syslog-ng
 cp /build/config/syslog_ng_default /etc/default/syslog-ng
+# Replace the system() source because inside Docker we
+# can't access /proc/kmsg.
+sed -i -E 's/^(\s*)system\(\);/\1unix-stream("\/dev\/log");/' /etc/syslog-ng/syslog-ng.conf
 
 ## Install logrotate.
 $minimal_apt_get_install logrotate