Extend Prometheus Labels to include tags (requires restart for NEW labels on the monitor to be visible)

[spali]

⚠️⚠️⚠️ Since we do not accept all types of pull requests and do not want to waste your time. Please be sure that you have read pull request rules:
https://github.com/louislam/uptime-kuma/blob/master/CONTRIBUTING.md#can-i-create-a-pull-request-for-uptime-kuma

Tick the checkbox if you understand [x]:

• I have read and understand the pull request rules.

Description

adds dynamic prometheus labels
fixes #680
replaces #898

Type of change

Please delete any options that are not relevant.

• New feature (non-breaking change which adds functionality)

Checklist

• My code follows the style guidelines of this project
• I ran ESLint and other linters for modified files
• I have performed a self-review of my own code and tested it
• I have commented my code, particularly in hard-to-understand areas (including JSDoc for methods)
• My changes generates no new warnings
• My code needed automated testing. I have added them (this is optional task)

Screenshots (if any)

Please do not use any external image service. Instead, just paste in or drag and drop the image here, and it will be uploaded automatically.

[spali]

I didn't had time to retest after fixing the merge conflicts...
wrote and tested the code over a year ago...
I would appreciate if someone could test a bit my PR.

Edit(CommanderStorm):
If somebody is curious how to test this, we have this docker image:

docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -e 'UPTIME_KUMA_GH_REPO=spali:dynamic_prometheus_labels' louislam/uptime-kuma:pr-test2

[spali]

did some basic tests and works for me as expected. After saving, the tags are immediately visible on the /metrics endpoint.

[CommanderStorm]

I also just did, but I cannot reproduce this working. I am missing some part.
Could you share your testing mechanism?

Mine is:

• docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -e 'UPTIME_KUMA_GH_REPO=spali:dynamic_prometheus_labels' louislam/uptime-kuma:pr-test2

  setup mariadb, log in
• create a monitor
• request /metrics => see monitor without tag
• edit monitor and add a tag
• request /metrics => see monitor without tag

[spali]

Did the same, but didn't use docker. In the repo and my PR branch npm install and npm run start.
I noticed you used pr-test2... is master based on V2?

[CommanderStorm]

Yes, master and all further feature development is v2.0 only.
Given that this PR adds a feature, this is the correct branch imo.
See #4500 for the status of said release and the 1.23.0-release notes:

🐻 It should be the last minor version of Uptime Kuma v1. I will focus on the development of v2. Stay tuned!
Update: The last minor version (1.23.x) means that there won't be any new features introduced in v1. However, bug fixes will still be provided. You can expect to see 1.23.1, 1.23.2 and so on.

[spali]

Then I can't explain the difference between our both test setup's. 🤔

[spali]

Could reproduce it after fresh config and no data.

It's a feature 😉 Needs a server restart for new tag's.
See explanation in #898 (comment) which I also forgot about after this long time 😉
Question is, if solution 3.5 is fine to start with or if we need to put more work into it from start.
As it's a new feature, existing users won't miss anything. And it could be optimized for the user experience later. I think some here wait for it and don't mind a restart after introducing new tags.
Never got an answer from @louislam or any other Collaborator here.

Edit:
I suggest to stay with this solution (3.5) as it is the most less invasive one. And later optimizing is always possible if someone finds the time.

[CommanderStorm]

3.5 sounds fine by me..

[spali]

Tested it again with the premise of restarting to see the monitor tags:

• create ./data directory with 777 permission

•  docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -v ./data:/app/data -e 'UPTIME_KUMA_GH_REPO=spali:dynamic_prometheus_labels' louislam/uptime-kuma:pr-test2
  

• setup sqlite db
• login
• disable auth
• create monitor with tag test
• kill container and restart with same docker command as above
• check /metrics for tag to be visible
• edit monitor and remove existing tag test and re-add tag test with new value
• check /metrics for tag with new value to be visible immediately
• edit monitor and remove tag test
• check /metrics for tag to be removed immediately

Looks good to me and works as intended.

[spali]

renamed PR in case title finds it's way into the changelogs, so the behaviour is noted there:

• dynamic prometheus labels (requires restart for new labels on the monitor to be visible)

[xlr-8]

Hey! Thanks for the MR, that's exactly what I wished to have! 🙏

Is there any chance for this MR to get reviewed? I don't know anything about JS 😓 but I think this MR could be a great addition to the project, since that would help using it better with prometheus/grafana and associated dashboards.

Looking forward to it whenever someone have time!

[CommanderStorm]

Thanks for the bugfix/new feature! 🎉

Note

This PR is part of the v2.0 merge window => see #4500 for the bugs that need to be addressed before we can ship this release ^^

_{All changes in this PR are small and uncontroversial ⇒ merging with junior maintainer approval}

[CommanderStorm]

@xlr-8
FYI: See #4704 (comment) for steps how you can review changes on a functional level without needing to get into the code

[xlr-8]

@xlr-8 FYI: See #4704 (comment) for steps how you can review changes on a functional level without needing to get into the code

Thanks for the heads up, you're right.

I've tried now but can't seem to make it work based on your insight and README.

I've just:

> git clone https://github.com/louislam/uptime-kuma/ && cd uptime-kuma
> mkdir data && chmod 777 data
>  docker run --rm -it -p 3000:3000 -p 3001:3001 --pull always -v ./data:/app/data -e 'UPTIME_KUMA_GH_REPO=spali:dynamic_prometheus_labels' louislam/uptime-kuma:pr-test2

Whenever going onto the http://0.0.0.0:3001/setup-database, I hit a blank page no matter the browser / private window or not. I don't see much on the network tab either; is there something I'm missing?

Perhaps I could help by providing a docker-compose, Makefile or something like this if you judge it relevant ?

And thank you for the quick review/merge 🙏

[CommanderStorm]

Skip the first two lines (the docker image already does that)
The web client is available on port 3000 => change that port in the browsers location bar and everything should work.

Thanks for testing this PR. Much appreciated 👍

PS: yea, I know. We should likely redirect users in dev mode when they connect to the API server on the most common route that is only used by the webclient

[louislam]

It could possibly throws an exception:

Trace: Error: Invalid label name
    at async Proxy.start (C:\Users\louis\Dropbox\My Projects\uptime-kuma\server\model\monitor.js:348:27)
    at async startMonitors (C:\Users\louis\Dropbox\My Projects\uptime-kuma\server\server.js:1801:9)
    at process.unexpectedErrorHandler (C:\Users\louis\Dropbox\My Projects\uptime-kuma\server\server.js:1855:13)
    at process.emit (node:events:520:28)
    at emitUnhandledRejection (node:internal/process/promises:250:13)
    at throwUnhandledRejectionsMode (node:internal/process/promises:385:19)
    at processPromiseRejections (node:internal/process/promises:470:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)
If you keep encountering errors, please report to https://github.com/louislam/uptime-kuma/issues

[louislam]

Actually, I have no idea how to fix #5096, if it is not fixed before 2.0.0-beta.0 release, it will be reverted and postpone to a later version.

[CommanderStorm]

Since Prometheus does only allow ASCII label names (thus handling of the client library is correct) we need to trim these chars out of the labels and only add a label if one is left afterwards.

Alternatively, we can just limit ourselves to labels which match the regex [a-zA-Z_][a-zA-Z0-9_]*.