Cloud integration tests are failing

[imilchev]

The cloud integration tests have been failing for a while now. We should investigate the root cause and fix them https://github.com/mondoohq/mondoo-operator/actions/runs/3406390108/jobs/5665044460

[czunker]

From the output, two possible problems:

Messages:   	Couldn't find expected version in MondooAuditConfig.Status.ReconciledByOperatorVersion

or

integration/audit_config_base_suite.go:801	Webhook is not available yet: Post "https://127.0.0.1:18443/validate-k8s-mondoo-com": dial tcp 127.0.0.1:18443: connect: connection refused

Bot were already addressed in previous PRs: #648 and #660

[czunker]

The VERSION is missing for the integration tests:

That's because the docker step to fetch the version failed:

The docker step is missing ${{ env.MONDOO_OPERATOR_IMAGE_TAG }}:

[czunker]

They still fail, but with a different error: https://github.com/mondoohq/mondoo-operator/actions/runs/3510168153

[czunker]

Different errors:

In the webhook:

2022-11-20T23:18:56Z	ERROR	webhook-validator	failed to set labels for incoming request	{"error": "object does not implement the Object interfaces"}
go.mondoo.com/mondoo-operator/pkg/webhooks/handler.(*webhookValidator).Handle
	/home/runner/work/mondoo-operator/mondoo-operator/pkg/webhooks/handler/webhook.go:113
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle
	/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/webhook.go:169
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP
	/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/webhook/admission/http.go:98
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1
	/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:40
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1
	/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:117
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2
	/home/runner/go/pkg/mod/github.com/prometheus/[email protected]/prometheus/promhttp/instrument_server.go:84
net/http.HandlerFunc.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2109
net/http.(*ServeMux).ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2487
net/http.serverHandler.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:2947
net/http.initALPNRequest.ServeHTTP
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/server.go:3556
net/http.(*http2serverConn).runHandler
	/opt/hostedtoolcache/go/1.19.1/x64/src/net/http/h2_bundle.go:5911

And:

2022-11-20T23:18:56Z	INFO	webhook-validator	Webhook triggered	{"kind": "Deployment", "resource": "mondoo-operator/passing-deployment"}
2022-11-20T23:18:56Z	INFO	webhook-validator	Scan result	{"shouldAdmit": false, "kind": "Deployment", "resource": "kube-system/coredns-autoscaler", "worstscore": {"value":100}}
2022-11-20T23:18:56Z	DEBUG	controller-runtime.webhook.webhooks	wrote response	{"webhook": "/validate-k8s-mondoo-com", "code": 403, "reason": "FAILED MONDOO SCAN", "UID": "1e7b33f6-7702-4966-944e-b1a2ab94b009", "allowed": false}

From the scan API logs:

�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m discover related assets for 1 asset(s)
�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m resolved assets �[36mresolved-assets=�[0m0
�[90m2022-11-21T10:21:59Z�[0m �[32mINF�[0m scan took 5.8µs

This seems fine.

[czunker]

This time it worked: https://github.com/mondoohq/mondoo-operator/actions/runs/3615483827

But we might still see this one error case from time to time: https://github.com/mondoohq/mondoo-operator/actions/runs/3572876685/jobs/6006274533

[czunker]

This time they failed because of missing protc: https://github.com/mondoohq/mondoo-operator/actions/runs/3671233346/jobs/6206299007

Fix PR is up: #723