refactor: prepare for stable metrics static analysis by defining metric labels with same format

[CatherineF-dev]

…ls with same format

What this PR does / why we need it:

KSM metrics are defined freely, which increases the difficulties of static analysis.

Will add a static analysis to make sure each metric defines label in this way:

1. SetLabelKeys appears once
2. The values is []string{*}

labelKeys := []string{"container"}

metric.SetLabelKeys(ms, labelKeys)

How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality) N/A

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1833

[x] certificatesigningrequest.go
[x] clusterrolebinding.go
[x] clusterrole.go
[x] configmap.go
[x] cronjob.go
[x] daemonset.go
[x] deployment.go
[x] endpoint.go
[x] endpointslice.go
[x] horizontalpodautoscaler.go
[x] ingressclass.go
[x] ingress.go
[x] job.go
[x] lease.go
[x] limitrange.go
[x] mutatingwebhookconfiguration.go
[x] namespace.go
[x] networkpolicy.go
[x] node.go
[x] persistentvolumeclaim.go
[x] persistentvolume.go
[x] poddisruptionbudget.go
[x] pod.go
[x] replicaset.go
[x] replicationcontroller.go
[x] resourcequota.go
[x] rolebinding.go
[x] role.go
[x] secret.go
[x] serviceaccount.go
[x] service.go
[x] statefulset.go
[x] storageclass.go
[x] validatingwebhookconfiguration.go
[x] volumeattachment.go

[CatherineF-dev]

cc @dgrisonnet to review

[dgrisonnet]

Forcing a particular coding style will make it a bit harder for contributors, but if we really have to do that, I am fine with it.
Could you perhaps explain what is your current approach that requires this change so that I can better understand your reasoning?

Ideally if we go that route, we should reflect it in https://github.com/kubernetes/kube-state-metrics/blob/main/docs/developer/guide.md, to documented the new coding style.

Also, there are a bunch more areas of the code that has this kind of definition, so it would be better to do it in one swoop: https://github.com/search?q=repo%3Akubernetes%2Fkube-state-metrics+LabelKeys%3A+%5B%5Dstring&type=code

[CatherineF-dev]

Could you perhaps explain what is your current approach that requires this change so that I can better understand your reasoning?

Similar to static analysis in k/k repo, it extracts all metric labels for each STABLE metric and generate a metric file containing metric name and metric labels. During presubmit, it regenerates this file and compare with previously generated file.

Since the way of defining metric labels are constrained a little, it's easier to extract metric labels.

For example, it will get label "container" for this metric.

labelKeys := []string{"container"}

metric.SetLabelKeys(ms, labelKeys)

[logicalhan]

/lgtm

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[CatherineF-dev]

Also, there are a bunch more areas of the code that has this kind of definition, so it would be better to do it in one swoop: https://github.com/search?q=repo%3Akubernetes%2Fkube-state-metrics+LabelKeys%3A+%5B%5Dstring&type=code

Okay, will update other files:

certificatesigningrequest.go
clusterrolebinding.go
clusterrole.go
configmap.go
cronjob.go
daemonset.go
deployment.go
endpoint.go
endpointslice.go
horizontalpodautoscaler.go
ingressclass.go
ingress.go
job.go
lease.go
limitrange.go
mutatingwebhookconfiguration.go
namespace.go
networkpolicy.go
node.go
persistentvolumeclaim.go
persistentvolume.go
poddisruptionbudget.go
pod.go
replicaset.go
replicationcontroller.go
resourcequota.go
rolebinding.go
role.go
secret.go
serviceaccount.go
service.go
statefulset.go
storageclass.go
validatingwebhookconfiguration.go
volumeattachment.go

[CatherineF-dev]

cc @mrueg for review.

[dgrisonnet]

Similar to static analysis in k/k repo, it extracts all metric labels for each STABLE metric and generate a metric file containing metric name and metric labels. During presubmit, it regenerates this file and compare with previously generated file.

Yeah I know this is the general goal, but could you explain how this refactor makes it easier to parse?

[CatherineF-dev]

A good question.

Before a:

kube-state-metrics/internal/store/pod.go

Lines 136 to 164 in 14e935a

	func createPodContainerInfoFamilyGenerator() generator.FamilyGenerator {
	return *generator.NewFamilyGeneratorWithStability(
	"kube_pod_container_info",
	"Information about a container in a pod.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapPodFunc(func(p *v1.Pod) *metric.Family {
	ms := []*metric.Metric{}
	labelKeys := []string{"container", "image_spec", "image", "image_id", "container_id"}
	for _, c := range p.Spec.Containers {
	for _, cs := range p.Status.ContainerStatuses {
	if cs.Name != c.Name {
	continue
	}
	ms = append(ms, &metric.Metric{
	LabelKeys: labelKeys,
	LabelValues: []string{cs.Name, c.Image, cs.Image, cs.ImageID, cs.ContainerID},
	Value: 1,
	})
	}
	}
	return &metric.Family{
	Metrics: ms,
	}
	}),
	)
	}

Now b:

kube-state-metrics/internal/store/pod.go

Lines 137 to 167 in 5ff07b5

	func createPodContainerInfoFamilyGenerator() generator.FamilyGenerator {
	return *generator.NewFamilyGeneratorWithStability(
	"kube_pod_container_info",
	"Information about a container in a pod.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapPodFunc(func(p *v1.Pod) *metric.Family {
	ms := []*metric.Metric{}
	labelKeys := []string{"container", "image_spec", "image", "image_id", "container_id"}
	for _, c := range p.Spec.Containers {
	for _, cs := range p.Status.ContainerStatuses {
	if cs.Name != c.Name {
	continue
	}
	ms = append(ms, &metric.Metric{
	LabelValues: []string{cs.Name, c.Image, cs.Image, cs.ImageID, cs.ContainerID},
	Value: 1,
	})
	}
	}
	metric.SetLabelKeys(ms, labelKeys)
	return &metric.Family{
	Metrics: ms,
	}
	}),
	)
	}

• It reduces depth from root node to metric label node. For example, example a needs following this path to find labels createPodContainerInfoFamilyGenerator > return > wrapPodFunc > for > for > ms > LabelKeys. Example b only needs to follow this path createPodContainerInfoFamilyGenerator > return > wrapPodFunc > LabelKeys

• Unify different ways on defining labels.
  

  kube-state-metrics/internal/store/job.go

  Lines 292 to 316 in 14e935a

  	*generator.NewFamilyGeneratorWithStability(
  	"kube_job_failed",
  	"The job has failed its execution.",
  	metric.Gauge,
  	basemetrics.STABLE,
  	"",
  	wrapJobFunc(func(j *v1batch.Job) *metric.Family {
  	ms := []*metric.Metric{}
  	for _, c := range j.Status.Conditions {
  	if c.Type == v1batch.JobFailed {
  	metrics := addConditionMetrics(c.Status)
  	for _, m := range metrics {
  	metric := m
  	metric.LabelKeys = []string{"condition"}
  	ms = append(ms, metric)
  	}
  	}
  	}
  	return &metric.Family{
  	Metrics: ms,
  	}
  	}),
  	),

kube-state-metrics/internal/store/pod.go

Lines 136 to 164 in 14e935a

	func createPodContainerInfoFamilyGenerator() generator.FamilyGenerator {
	return *generator.NewFamilyGeneratorWithStability(
	"kube_pod_container_info",
	"Information about a container in a pod.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapPodFunc(func(p *v1.Pod) *metric.Family {
	ms := []*metric.Metric{}
	labelKeys := []string{"container", "image_spec", "image", "image_id", "container_id"}
	for _, c := range p.Spec.Containers {
	for _, cs := range p.Status.ContainerStatuses {
	if cs.Name != c.Name {
	continue
	}
	ms = append(ms, &metric.Metric{
	LabelKeys: labelKeys,
	LabelValues: []string{cs.Name, c.Image, cs.Image, cs.ImageID, cs.ContainerID},
	Value: 1,
	})
	}
	}
	return &metric.Family{
	Metrics: ms,
	}
	}),
	)
	}

kube-state-metrics/internal/store/endpoint.go

Lines 167 to 195 in 14e935a

	*generator.NewFamilyGeneratorWithStability(
	"kube_endpoint_address",
	"Information about Endpoint available and non available addresses.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapEndpointFunc(func(e *v1.Endpoints) *metric.Family {
	ms := []*metric.Metric{}
	for _, s := range e.Subsets {
	for _, available := range s.Addresses {
	ms = append(ms, &metric.Metric{
	LabelValues: []string{available.IP, "true"},
	LabelKeys: []string{"ip", "ready"},
	Value: 1,
	})
	}
	for _, notReadyAddresses := range s.NotReadyAddresses {
	ms = append(ms, &metric.Metric{
	LabelValues: []string{notReadyAddresses.IP, "false"},
	LabelKeys: []string{"ip", "ready"},
	Value: 1,
	})
	}
	}
	return &metric.Family{
	Metrics: ms,
	}
	}),
	),

kube-state-metrics/internal/store/replicaset.go

Lines 167 to 211 in 14e935a

	*generator.NewFamilyGeneratorWithStability(
	"kube_replicaset_owner",
	"Information about the ReplicaSet's owner.",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapReplicaSetFunc(func(r *v1.ReplicaSet) *metric.Family {
	owners := r.GetOwnerReferences()
	if len(owners) == 0 {
	return &metric.Family{
	Metrics: []*metric.Metric{
	{
	LabelKeys: []string{"owner_kind", "owner_name", "owner_is_controller"},
	LabelValues: []string{"", "", ""},
	Value: 1,
	},
	},
	}
	}
	ms := make([]*metric.Metric, len(owners))
	for i, owner := range owners {
	if owner.Controller != nil {
	ms[i] = &metric.Metric{
	LabelValues: []string{owner.Kind, owner.Name, strconv.FormatBool(*owner.Controller)},
	}
	} else {
	ms[i] = &metric.Metric{
	LabelValues: []string{owner.Kind, owner.Name, "false"},
	}
	}
	}
	for _, m := range ms {
	m.LabelKeys = []string{"owner_kind", "owner_name", "owner_is_controller"}
	m.Value = 1
	}
	return &metric.Family{
	Metrics: ms,
	}
	}),
	),

kube-state-metrics/internal/store/service.go

Lines 59 to 76 in 14e935a

	*generator.NewFamilyGeneratorWithStability(
	"kube_service_created",
	"Unix creation timestamp",
	metric.Gauge,
	basemetrics.STABLE,
	"",
	wrapSvcFunc(func(s *v1.Service) *metric.Family {
	if !s.CreationTimestamp.IsZero() {
	m := metric.Metric{
	LabelKeys: nil,
	LabelValues: nil,
	Value: float64(s.CreationTimestamp.Unix()),
	}
	return &metric.Family{Metrics: []*metric.Metric{&m}}
	}
	return &metric.Family{Metrics: []*metric.Metric{}}
	}),
	),

• It defines all labels in the beginning. In example a, more labelKeys can be added in the same function later.

• Extract shared labels easier by defining all shared labels in the beginning with type SharedLabelKey

• Also improve readability that readers can find metric labels easily.

cc @dgrisonnet

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CatherineF-dev, logicalhan
Once this PR has been reviewed and has the lgtm label, please ask for approval from rexagod. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[CatherineF-dev]

Will use #2294 instead. It's an end-to-end example on verifying stable metrics.