Debian 12 + LXD/LXC security.idmap.isolated fails - eviltoast

Debian 12.1 (6.1.0-11-amd64) running LXD/LXC and on an unprivileged container setting security.idmap.isolated=true seems to fail to update the owner/group of the container’s files.

Here is an example:

# lxc launch images:debian/12 debian
(...)

# lxc config get debian volatile.idmap.base
296608

# lxc stop debian
Error: The instance is already stopped

# lxc config set debian security.idmap.isolated true

# lxc config get debian security.idmap.isolated
true

# lxc start debian

Now if I list the files on the container volume I’ll get they’re all owned by the host root user:

# ls -la /mnt/NVME1/lxd/containers/debian/rootfs/
total 24
drwxr-xr-x 1 root   root  154 Sep  5 06:28 .
d--x------ 1 296608 root   78 Sep  5 15:59 ..
lrwxrwxrwx 1 root   root    7 Sep  5 06:25 bin -> usr/bin
drwxr-xr-x 1 root   root    0 Jul 14 17:00 boot
drwxr-xr-x 1 root   root    0 Sep  5 06:28 dev
drwxr-xr-x 1 root   root 1570 Sep  5 06:28 etc

I tried multiple versions of LXD/LXC. This happens with both 5.0.2 from apt as well with 4.0 and 5.17 (latest) from snap.

Interestingly enough I have another Debian 10 (4.19.0-25-amd64) running and older LXD 4 from snap and on that one things work as expected:

# ls -la /mnt/NVME1/lxd/containers/debian/rootfs/
total 0
drwxr-xr-x 1 1065536 1065536  138 Oct 29  2020 .
d--x------ 1 1065536 root      78 Oct 14  2020 ..
drwxr-xr-x 1 1065536 1065536 1328 Jul 24 19:07 bin
drwxr-xr-x 1 1065536 1065536    0 Sep 19  2020 boot
drwxr-xr-x 1 1065536 1065536    0 Oct 14  2020 dev
drwxr-xr-x 1 1065536 1065536 1716 Jul 24 19:08 etc

As you can see on this systems all the files are owned by 1065536:1065536.


Update:

I tried to probe around the maps with lxc config show debian in both machines and I saw this:

Machine running Debian 10:

security.idmap.isolated: "true"
(...)
volatile.idmap.base: "1065536"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'

Machine running Debian 12:

security.idmap.isolated: "true"
(...)
volatile.idmap.base: "231072"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":231072,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":231072,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":231072,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":231072,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[]'

Why didn’t it populate volatile.last_state.idmap: '[]'?

How can I fix it? Thank you.