Zilla / ArgoCD / Kustomize #969

vordimous · 2024-04-19T23:42:32Z

vordimous
Apr 19, 2024
Collaborator

I am trying to install zilla on kubernetes and manage it with ArgoCD. Having issues with how to specify the zilla.yaml file with kustomize. The helm install cant find the zilla.yaml file based on how I have configured it. The issue is how to translate the --set-file zilla\\.yaml=zilla.yaml zilla.yaml, values.yaml and the kustomize files are all in the same directory.

Here is what I have now:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - namespace.yaml
  - zilla.yaml

helmCharts:
  - name: zilla
    repo: ghcr.io/aklivity/charts/zilla
    version: 0.9.78
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

Answered by vordimous

Apr 23, 2024

Adding the answer here for future reference.

The problem is oci:// isn't supported fully yet in Kustomize: kubernetes-sigs/kustomize#5134 (comment)

The solution is to manually pull the chart version you want and untar it into a local folder:

helm pull oci://ghcr.io/aklivity/charts/zilla --version <zilla_version> --untar --untardir <folder_name>

Now you can remove the repo attribute from your yaml and add the helm-charts path to helmGlobals

helmGlobals:
  chartHome: <folder_name>

helmCharts:
  - name: zilla
    version: <zilla_version>
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

View full answer

vordimous · 2024-04-19T23:53:02Z

vordimous
Apr 19, 2024
Collaborator Author

In your helm values.yaml you can specify a zilla.yaml property and directly include the raw string value of your zilla.yaml file. This will forgo the need to use the --set-file zilla\\.yaml=zilla.yaml Helm feature that doesn't look like it is supported in kustomize

values.yaml:

...
zilla.yaml: |
  name: example
  bindings:
    my_binding:
      type: ...
      kind: ...

...

1 reply

vordimous Apr 19, 2024
Collaborator Author

Note that the kustomize team has an open ticket to improve their helm support. This implementaion may not be needed in the future.

vacquah · 2024-04-20T13:43:42Z

vacquah
Apr 20, 2024

@vordimous thanks for the reply. It seems there is also an issue with the zilla oci registry? When I do a helm pull:

helm pull oci://ghcr.io/aklivity/zilla --version 0.9.78 --debug , I get this :

DEBU[0000] resolving                                     host=ghcr.io
DEBU[0000] do request                                    host=ghcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=HEAD url="https://ghcr.io/v2/aklivity/zilla/manifests/0.9.78"
DEBU[0000] fetch response received                       host=ghcr.io response.header.content-length=73 response.header.content-type=application/json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.www-authenticate="Bearer realm=\"https://ghcr.io/token\",service=\"ghcr.io\",scope=\"repository:aklivity/zilla:pull\"" response.header.x-github-request-id="A26B:2A7B5C:287CE35:2F9E006:6623C629" response.status="401 Unauthorized" url="https://ghcr.io/v2/aklivity/zilla/manifests/0.9.78"
DEBU[0000] Unauthorized                                  header="Bearer realm=\"https://ghcr.io/token\",service=\"ghcr.io\",scope=\"repository:aklivity/zilla:pull\"" host=ghcr.io
DEBU[0000] do request                                    host=ghcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=HEAD url="https://ghcr.io/v2/aklivity/zilla/manifests/0.9.78"
DEBU[0000] fetch response received                       host=ghcr.io response.header.content-length=1609 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49\"" response.header.x-github-request-id="A26B:2A7B5C:287CE38:2F9E00C:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/0.9.78"
DEBU[0000] resolved                                      desc.digest="sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49" host=ghcr.io
DEBU[0000] do request                                    digest="sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49" request.header.accept="application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=GET url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49"
DEBU[0000] fetch response received                       digest="sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49" response.header.content-length=1609 response.header.content-type=application/vnd.oci.image.index.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49\"" response.header.x-github-request-id="A26B:2A7B5C:287CE3B:2F9E00F:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:e97603ec9254884231eea9e5dbe3619754d17deaac13dcfd268ae5a9b2132d49"
DEBU[0000] do request                                    digest="sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac" request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=GET url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac"
DEBU[0000] fetch response received                       digest="sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac" response.header.content-length=566 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac\"" response.header.x-github-request-id="A26B:2A7B5C:287CE41:2F9E014:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:699adafdf84dde3a20b6e98be0c5e89fdc719694968e71f036c62c9655713bac"
DEBU[0000] do request                                    digest="sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593" request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=GET url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593"
WARN[0000] unknown type: application/vnd.in-toto+json
WARN[0000] unknown type: application/vnd.oci.image.config.v1+json
DEBU[0000] fetch response received                       digest="sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593" response.header.content-length=1051 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593\"" response.header.x-github-request-id="A26B:2A7B5C:287CE43:2F9E016:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:bb8470a470c39cd2d20ea3e5b655b3624f8fa25175fa14261de07d3890327593"
DEBU[0000] do request                                    digest="sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1" request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=GET url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1"
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
WARN[0000] unknown type: application/vnd.oci.image.config.v1+json
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
DEBU[0000] fetch response received                       digest="sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1" response.header.content-length=1051 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1\"" response.header.x-github-request-id="A26B:2A7B5C:287CE45:2F9E01E:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:b94fa48ae555381c4438eec2cfdb2d0b244c554591db8599e2f8086be42862d1"
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
WARN[0000] unknown type: application/vnd.oci.image.config.v1+json
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
DEBU[0000] do request                                    digest="sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5" request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=Helm/3.14.4 request.method=GET url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5"
WARN[0000] unknown type: application/vnd.oci.image.layer.v1.tar+gzip
DEBU[0000] fetch response received                       digest="sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5" response.header.content-length=566 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Sat, 20 Apr 2024 13:42:01 GMT" response.header.docker-content-digest="sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5\"" response.header.x-github-request-id="A26B:2A7B5C:287CE4B:2F9E025:6623C629" response.status="200 OK" url="https://ghcr.io/v2/aklivity/zilla/manifests/sha256:09972526d38d8db468940b48593a4b4f928aece4ed00e4aca3cbdde414acb4c5"
WARN[0000] unknown type: application/vnd.in-toto+json
WARN[0000] unknown type: application/vnd.oci.image.config.v1+json
Error: manifest does not contain minimum number of descriptors (2), descriptors found: 1
helm.go:84: [debug] manifest does not contain minimum number of descriptors (2), descriptors found: 1

1 reply

vordimous Apr 20, 2024
Collaborator Author

That oci address is for the docker container. The helm chart is at oci://ghcr.io/aklivity/charts/zilla so the command would be:

helm pull oci://ghcr.io/aklivity/charts/zilla --version 0.9.78

vacquah · 2024-04-20T14:20:07Z

vacquah
Apr 20, 2024

Using this kustomize file with your suggested changes in the values.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - namespace.yaml

helmCharts:
  - name: zilla
    repo: ghcr.io/aklivity/charts/zilla
    version: 0.9.78
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

I get this:

Error: could not find protocol handler for: 
: unable to run: 'helm pull --untar --untardir /dev/argocd/apps/zilla/charts/zilla-0.9.78 --repo ghcr.io/aklivity/charts/zilla zilla --version 0.9.78' with env=[HELM_CONFIG_HOME=/var/folders/gg/n643b5ws74jbvbgb7smh49sw0000gn/T/kustomize-helm-3327849985/helm HELM_CACHE_HOME=/var/folders/gg/n643b5ws74jbvbgb7smh49sw0000gn/T/kustomize-helm-3327849985/helm/.cache HELM_DATA_HOME=/var/folders/gg/n643b5ws74jbvbgb7smh49sw0000gn/T/kustomize-helm-3327849985/helm/.data] (is 'helm' installed?): exit status 1

helm is installed and I have used it to install a few apps in argocd.

1 reply

vordimous Apr 20, 2024
Collaborator Author

Error: could not find protocol handler for:

This error is because of a missing oci:// protocol when referring to the chart

vordimous · 2024-04-20T14:50:25Z

vordimous
Apr 20, 2024
Collaborator Author

@vacquah I think specifying the repo is causing the issue. In our helm install example we use the fully qualified oci chart name and don't need to first add a repo:

helm install zilla oci://ghcr.io/aklivity/charts/zilla --namespace zilla --create-namespace --wait \
    --values values.yaml \
    --set-file zilla\\.yaml=zilla.yaml

I don't think the github package registry ghcr.io/aklivity/charts works like a normal chart repo.

% helm repo add aklivity oci://ghcr.io/aklivity/charts
Error: looks like "oci://ghcr.io/aklivity/charts" is not a valid chart repository or cannot be reached: object required

Here is what should solve your problem:

oci://ghcr.io/aklivity/charts/zilla is the full name of the chart. What might work for customize helmCharts is this:

helmCharts:
  - name: oci://ghcr.io/aklivity/charts/zilla
    version: 0.9.78
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

10 replies

vordimous Apr 23, 2024
Collaborator Author

@vacquah did this work for you inside your ArgoCD pipeline?

vacquah Apr 23, 2024

Yes it works perfectly. But I assume it implies I will have to manually update the charts? So reverting back to the original because the code below seems to work. Just needed to change the chart url slightly:

helmCharts:
  - name: zilla
    repo: oci://ghcr.io/aklivity/charts
    version: 0.9.78
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

vordimous Apr 23, 2024
Collaborator Author

But I assume it implies I will have to manually update the charts?

It is best to use a static version variable in your CD pipeline. The helm pull command will fetch new versions if you make the zilla version part of your build variables, you would be able to update the Zilla version in one place.

vordimous Apr 23, 2024
Collaborator Author

helmCharts:
 - name: zilla
   repo: oci://ghcr.io/aklivity/charts
   version: 0.9.78
   releaseName: zilla
   namespace: zilla
   valuesFile: values.yaml

This config works in ArgoCD? It is still failing for me when I use the kubectl kustomize --enable-helm . | kubectl apply -f - command.

vacquah Apr 24, 2024

Hi - yes its working for me. I had to setup kustomize helm plugin in argocd. Perhaps your way is simpler.

vordimous · 2024-04-23T16:10:26Z

vordimous
Apr 23, 2024
Collaborator Author

Adding the answer here for future reference.

The problem is oci:// isn't supported fully yet in Kustomize: kubernetes-sigs/kustomize#5134 (comment)

The solution is to manually pull the chart version you want and untar it into a local folder:

helm pull oci://ghcr.io/aklivity/charts/zilla --version <zilla_version> --untar --untardir <folder_name>

Now you can remove the repo attribute from your yaml and add the helm-charts path to helmGlobals

helmGlobals:
  chartHome: <folder_name>

helmCharts:
  - name: zilla
    version: <zilla_version>
    releaseName: zilla
    namespace: zilla
    valuesFile: values.yaml

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Zilla / ArgoCD / Kustomize #969

{{title}}

Replies: 5 comments 13 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Zilla / ArgoCD / Kustomize #969

vordimous Apr 19, 2024 Collaborator

Replies: 5 comments · 13 replies

vordimous Apr 19, 2024 Collaborator Author

vordimous Apr 19, 2024 Collaborator Author

vacquah Apr 20, 2024

vordimous Apr 20, 2024 Collaborator Author

vacquah Apr 20, 2024

vordimous Apr 20, 2024 Collaborator Author

vordimous Apr 20, 2024 Collaborator Author

vordimous Apr 23, 2024 Collaborator Author

vacquah Apr 23, 2024

vordimous Apr 23, 2024 Collaborator Author

vordimous Apr 23, 2024 Collaborator Author

vacquah Apr 24, 2024

vordimous Apr 23, 2024 Collaborator Author

vordimous
Apr 19, 2024
Collaborator

Replies: 5 comments 13 replies

vordimous
Apr 19, 2024
Collaborator Author

vordimous Apr 19, 2024
Collaborator Author

vacquah
Apr 20, 2024

vordimous Apr 20, 2024
Collaborator Author

vacquah
Apr 20, 2024

vordimous Apr 20, 2024
Collaborator Author

vordimous
Apr 20, 2024
Collaborator Author

vordimous Apr 23, 2024
Collaborator Author

vordimous Apr 23, 2024
Collaborator Author

vordimous Apr 23, 2024
Collaborator Author

vordimous
Apr 23, 2024
Collaborator Author