Can't do any operations on collection with expired targets cert/metadata

[patoarvizu]

(Creating this issue as per @justincormack on Slack, and cc'ing @jonnystoten as requested. It's a duplicate of #1648, but I though a new issue with more detailed description might help.)

I have a Notary collection at docker.io/patoarvizu/kms-vault-operator that I created a little over 3 years ago, so the targets key is now expired. Now, most commands I run on that collection from the CLI are throwing the following error:

ERRO[0001] Metadata for targets expired
ERRO[0001] Metadata for targets expired
ERRO[0001] Metadata for targets expired
ERRO[0001] Metadata for targets expired

* fatal: targets expired at Thu Aug 18 12:21:15 EDT 2022

Including commands that do not require encryption (I believe), like notary list docker.io/patoarvizu/kms-vault-operator.

I don't know where to go from here. I have the root key in a Yubikey and there's also one delegation role that was created around the same time as the repository was initialized so I assume it's also expired, but I can't know for sure because when I do notary delegation list docker.io/patoarvizu/kms-vault-operator I get an error similar to the above.

This is with Notary 0.6.1 (and I believe the collection was initialized with the same version, if that matters). I still have access to all my private cert material, including the root key.

Can I get some assistance?

[jonnystoten]

Hey, sorry for the delay, I've been off work sick 🤕

The (hacky) fix for this is basically to set the clock back on the client so that it no longer considers the metadata expired. Then you can re-sign the metadata, set the clock back to normal, and re-sign again. It's important to note that it is the metdata files that expire, not the keys, so you should be able to use your existing keys for this.

If you're using Docker's public notary instance to host your metadata, you might run into a problem where there is no overlap between the time your metadata files are valid and the time the server's TLS certificates are valid, meaning that if you set the clock back you won't be able to make any TLS requests to the server. If you run into this problem, let me know and I can walk through a solution with you in the new year when I'm back from PTO (from the 9th Jan).

Hope this helps!

[patoarvizu]

Thanks for the response @jonnystoten!

Can you let me know when you're back from PTO to see if we can schedule a call or some other form of communication to walk through your solution?

Thanks and happy new year!

[patoarvizu]

Hi @jonnystoten! Following up on this. Would you have any availability soon to give me a hand with this? Thanks!

[jonnystoten]

Apologies for the delay on this @patoarvizu! Yes, let's set up some time. I'm available Wed, Thu and Fri next week (12th, 13th, 14th April), and I'm based in the UK so I'm available until around 12:30 PM your time (I see you're based in New York). Would any of those days work for you?

[patoarvizu]

Yes! I think Wednesday morning about 10:30-11am eastern (yes, I'm based in New York) works best. Are you on the CNCF Slack workspace? We can connect directly there (or any other Slack workspace) to coordinate.

[HoyluBert]

I am facing the same issue on an Azure Container Registry with enabled Content Trust. Did you ever resolve this issue?

[patoarvizu]

No, I haven't heard back, unfortunately.

Pinging @jonnystoten one more time to see if we can get some traction on this.

[Csahu1997]

+1 on this. I'm also encountering this issue "Metadata for targets expired". As suggested by @jonnystoten I tried resigning the metadata by setting the clock back on client but I was not able to make any TLS requests to the server. Error :
Unable to connect to the server: tls: failed to verify certificate: x509: certificate has expired or is not yet valid: current time 2024-04-16T01:18:23Z is before 2024-05-21T15:01:57Z
@jonnystoten Could I get any help on this ? TIA !

[williamdes]

Here is the commands to run to fix this: https://github.com/sudo-bot/action-docker-sign/?tab=readme-ov-file#renewingre-building-the-repository-metadata

I got my repo back in working state after having Metadata for targets expired