support disk usage metrics for containerd factory #3502

chengjoey · 2024-03-19T15:07:56Z

query rootfs dir from contianerd status and get disk usage stat by rootfs dir /run/containerd/io.containerd.runtime.v2.task/${contianerid}/rootfs

set hasFileSystem to true for containerd when included disk usage metrics
added common fs handler for container rootfs

k8s-ci-robot · 2024-03-19T15:08:07Z

Hi @chengjoey. Thanks for your PR.

I'm waiting for a google member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

chengjoey · 2024-03-19T15:09:17Z

container_fs_usage_bytes metrics miss pod/container label when useing containerd

iwankgb · 2024-03-31T17:31:40Z

container/docker/handler_test.go

@@ -169,7 +169,7 @@ func TestAddDiskStatsCheck(t *testing.T) {
 	fileSystems := []fs.Fs{fileSystem}

 	var fsStats info.FsStats
-	addDiskStats(fileSystems, nil, &fsStats)
+	fs.AddDiskStats(fileSystems, nil, &fsStats)


This test should be moved to fs module.

iwankgb · 2024-03-31T17:31:46Z

container/docker/handler_test.go

@@ -203,7 +203,7 @@ func TestAddDiskStats(t *testing.T) {
 	fileSystems := []fs.Fs{fileSystem}

 	// Act
-	addDiskStats(fileSystems, &fsInfo, &fsStats)
+	fs.AddDiskStats(fileSystems, &fsInfo, &fsStats)


iwankgb · 2024-03-31T17:34:40Z

container/containerd/client.go

+	}
+	configStr := r.Info["config"]
+	config := make(map[string]interface{})
+	if err := json.Unmarshal([]byte(configStr), &config); err != nil {


Could you unmarshal config into a struct? It will be easier to grasp what values are interesting.

The original definition of config is here

I extracted the struct in config.go to container/containerd/config

iwankgb · 2024-03-31T17:35:34Z

/ok-to-test

chengjoey · 2024-04-02T14:59:58Z

/retest

silence8013 · 2024-05-06T02:47:06Z

Will this commit continue? Looking forward to it~

duanhongyi · 2024-05-11T10:27:38Z

I am looking forward to this feature merging.

jianxiaoguo · 2024-05-11T13:51:29Z

I eagerly anticipate the completion of this feature.

EamonZhang · 2024-05-13T01:32:20Z

This feature is particularly important and I'm looking forward to it

wsszh · 2024-05-24T11:07:07Z

Hi, I think the disk usage of a container should include the usage of log files in /var/log/pods.

chengjoey · 2024-05-25T06:28:12Z

Hi, I think the disk usage of a container should include the usage of log files in /var/log/pods.

hi @wsszh , The disk usage calculation of contianerd is similar to that of docker and podman. I think that several container factories currently do not include /var/log/pods in the calculation. This may be because cadvisor is more about container-level calculations rather than pods. It is also difficult to know which directory should be calculated because it is not fixed in /var/log/pods. although I think it should be calculated, the specific directory of the log cannot be obtained from the info interface of cri.

chengjoey · 2024-05-27T13:24:15Z

Hi, I think the disk usage of a container should include the usage of log files in /var/log/pods.

if a container should include the usage of log files, I would like to submit another PR to modify the container runtimes such as podman, docker, containerd, etc. This PR only does usage metric for containerd

query rootfs dir from contianerd status and get disk usage stat by rootfs dir `/run/containerd/io.containerd.runtime.v2.task/${contianerid}/rootfs` Signed-off-by: joey <[email protected]>

kxbin · 2024-12-19T07:33:06Z

great work，Hope to speed up to merge in master

melikeiremguler · 2024-12-30T10:27:42Z

Hi, will this pull request solve this problem ?

chengjoey · 2024-12-30T15:09:54Z

Hi, will this pull request solve this problem ?

@melikeiremguler , Yes, this PR is to solve the issue of 2785.

gigilee · 2025-03-24T20:21:36Z

Is this PR going to be merged?

Imlosing · 2025-04-24T08:09:50Z

Looking forward to the early merge.

Imlosing · 2025-04-24T08:10:50Z

Looking forward to the early merge.

Or is there any alternative solution now?

kam1kaze · 2025-05-05T07:25:22Z

@iwankgb How could we speed up merge process of this code? We are using it on production and it works perfect.

Imlosing · 2025-05-13T06:34:30Z

@iwankgb How could we speed up merge process of this code? We are using it on production and it works perfect.

@kam1kaze I also merged this PR and tried to use it, but I found that the actual value collected was greater than the writable layer capacity. Have you encountered this problem?

kamikaze · 2025-07-29T14:58:35Z

please

k8s-ci-robot added the needs-ok-to-test label Mar 19, 2024

This was referenced Mar 19, 2024

[containerd-cri]Set HasFilesystem to true #3253

Open

container_fs_usage_bytes metrics miss pod/container label #3495

Open

iwankgb requested changes Mar 31, 2024

View reviewed changes

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Mar 31, 2024

chengjoey force-pushed the feat/containerd-disk-usage branch from 68c7bbf to 5f1986c Compare April 1, 2024 02:37

chengjoey force-pushed the feat/containerd-disk-usage branch 6 times, most recently from 347eed9 to 1b85ff8 Compare August 14, 2024 15:04

support disk usage metrics for containerd factory

9e7a939

query rootfs dir from contianerd status and get disk usage stat by rootfs dir `/run/containerd/io.containerd.runtime.v2.task/${contianerid}/rootfs` Signed-off-by: joey <[email protected]>

chengjoey force-pushed the feat/containerd-disk-usage branch from 1b85ff8 to 9e7a939 Compare August 14, 2024 15:13

Imlosing mentioned this pull request May 28, 2025

[BUG] #3502: The container_fs_usage_bytes metric is too large, including the size of the container's read-only layer #3691

Open

support disk usage metrics for containerd factory #3502

Are you sure you want to change the base?

support disk usage metrics for containerd factory #3502

Uh oh!

Conversation

chengjoey commented Mar 19, 2024

Uh oh!

k8s-ci-robot commented Mar 19, 2024

Uh oh!

chengjoey commented Mar 19, 2024

Uh oh!

iwankgb Mar 31, 2024

Choose a reason for hiding this comment

Uh oh!

chengjoey Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

iwankgb Mar 31, 2024

Choose a reason for hiding this comment

Uh oh!

chengjoey Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

iwankgb Mar 31, 2024

Choose a reason for hiding this comment

Uh oh!

chengjoey Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

iwankgb commented Mar 31, 2024

Uh oh!

chengjoey commented Apr 2, 2024

Uh oh!

silence8013 commented May 6, 2024

Uh oh!

duanhongyi commented May 11, 2024

Uh oh!

jianxiaoguo commented May 11, 2024

Uh oh!

EamonZhang commented May 13, 2024

Uh oh!

wsszh commented May 24, 2024

Uh oh!

chengjoey commented May 25, 2024

Uh oh!

chengjoey commented May 27, 2024

Uh oh!

kxbin commented Dec 19, 2024

Uh oh!

melikeiremguler commented Dec 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chengjoey commented Dec 30, 2024

Uh oh!

gigilee commented Mar 24, 2025

Uh oh!

Imlosing commented Apr 24, 2025

Uh oh!

Imlosing commented Apr 24, 2025

Uh oh!

kam1kaze commented May 5, 2025

Uh oh!

Imlosing commented May 13, 2025

Uh oh!

kamikaze commented Jul 29, 2025

Uh oh!

Uh oh!

melikeiremguler commented Dec 30, 2024 •

edited

Loading