1 year ago

#347561

test-img

Jeffery Don

Attribute of Workdir of Docker container turned to "c(character device file)" from "d(directory)" after running for a while

The question is found from a docker command error when I tried to execute exec command to a container:

OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: chdir to cwd ("/home/admin") set in config.json failed: no such file or directory: unknown command terminated with exit code 126

so I use docker inspect to determine whether the directory exists, and what I found is: file attribute of directory /home/admin turned to a character device file (also known as "c" in linux system). Pictures of normal situation and abnormal situation is below:

Normal: normal file attribute before glitch

Abnormal: abnormal file attribute after glitch

What I can google is this article: Every podman run invocation generates two "Couldn't stat device /dev/char/10:200: No such file or directory" lines in the journal, as i found the same message from systemd about /dev/char/10:200. The solution provided by this article is to check docker daemon.json file, but I didn't check it, as I use this file in plenty of projects and no glitch ever happened. I can show my daemon.json below:

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ],
  "registry-mirrors": ["https://registry.docker-cn.com"],
  "graph": "/services/docker"
}

After I gave up on solving this problem from docker, I discussed with the developer of that glitch application. And now the situation is: We deployed the application in a Kubernetes cluster, other applications never had this kind of problem but just his application. And this application will create some subprocess to execute some linux command, and those commands will download some file to container's workdir.

Can anyone help me with this problem? I really need some help.

BTW, environment where container runs is down below:

Linux Kernel: 3.10.0-957.el7.x86_64
Linux OS: CentOS Linux 7 (Core)
CPU Architecture: AMD x86_64
Docker Version: 19.03.12
Kubernetes Version: 1.18.8

As for Rico's question, the container is created by kubernetes, and there is no mount of any filesystem, all the write and read operation is in container, temporarily。

linux

docker

kubernetes

file-attributes

0 Answers

Your Answer

Accepted video resources