Comment 4 for bug 1346734

I created a per-user container "t1", and confirm that it does start under upstart/cgmanger and doesn't under systemd. I now have a preliminary patch for putting the user slices into all cgroup controllers, plus some hand-crafted "chown ubuntu" for all the user-1000.slice cgroup directories so that they become writable (this part still needs to be added to the patch). I understand that this should now be sufficient:

ubuntu@ulxc$ cat /proc/$$/cgroup
10:devices:/user.slice/user-1000.slice
9:memory:/user.slice/user-1000.slice
8:cpuset:/
7:hugetlb:/user.slice/user-1000.slice
6:blkio:/user.slice/user-1000.slice
5:cpu,cpuacct:/user.slice/user-1000.slice
4:freezer:/user.slice/user-1000.slice
3:perf_event:/user.slice/user-1000.slice
2:net_cls,net_prio:/user.slice/user-1000.slice
1:name=systemd:/user.slice/user-1000.slice/session-1.scope

ubuntu@ulxc:~$ ls -ld /sys/fs/cgroup/*/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/blkio/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/cpuacct/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/cpuset/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/cpu/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/devices/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/freezer/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/hugetlb/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/memory/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/net_cls,net_prio/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/net_cls/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/net_prio/user.slice/user-1000.slice/
drwxr-xr-x 2 ubuntu root 0 Nov 26 10:41 /sys/fs/cgroup/perf_event/user.slice/user-1000.slice/
drwxr-xr-x 4 root root 0 Nov 26 10:33 /sys/fs/cgroup/systemd/user.slice/user-1000.slice/

I'm not sure why my login shell isn't in "cpuset", I'll debug that still. But I chown'ed /sys/fs/cgroup/cpuset/ to "ubuntu" as well.

But still lxc-start fails:

$ lxc-start -n t1 -F
lxc-start: cgfs.c: lxc_cgroupfs_create: 849 Could not set clone_children to 1 for cpuset hierarchy in parent cgroup.
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/devices/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/memory/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset//user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset//user.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Read-only file system - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpuset/
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/hugetlb/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/blkio/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/freezer/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/perf_event/user.slice/user-1000.slice
lxc-start: cgfs.c: cgroup_rmdir: 207 Permission denied - cgroup_rmdir: failed to delete /sys/fs/cgroup/net_cls,net_prio/user.slice/user-1000.slice
lxc-start: start.c: lxc_spawn: 864 failed creating cgroups

Questions:

 - Why is it trying to *remove* the existing cgroups? It sounds wrong to fuzz around with those, I thought it would merely want and need to create new cgroups below those? And the ubuntu user can definitively do that:

ubuntu@ulxc:~$ mkdir /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/mygroup
ubuntu@ulxc:~$ ls -ld /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/mygroup
drwxrwxr-x 2 ubuntu ubuntu 0 Nov 26 10:50 /sys/fs/cgroup/cpu,cpuacct/user.slice/user-1000.slice/mygroup

--logpriority debug --logfile /tmp/d doesn't really give much information either. stracing lxc-start only shows rmdir() whose errors are shown above, it doesn't have any mkdir() or similar call which would show an attempt to create new cgroups?