Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uptime Kuma is very slow when deployed on NFS #4659

Closed
1 task done
sudoexec opened this issue Apr 7, 2024 · 4 comments
Closed
1 task done

Uptime Kuma is very slow when deployed on NFS #4659

sudoexec opened this issue Apr 7, 2024 · 4 comments
Labels
area:deployment related to how uptime kuma can be deployed help question Further information is requested

Comments

@sudoexec
Copy link

sudoexec commented Apr 7, 2024

πŸ“‘ I have found these related issues/pull requests

No related issues.

πŸ›‘οΈ Security Policy

Description

Deployed in k8s.
I have 54 monitors and make them all in groups.
The homepage is always return blank pages without any monitors(or waiting for a long time there will show monitors), and the settings page is very slowly.
In the devtools,websocket sometimes is pending, sometimes return very slowly(getTags request).

πŸ‘Ÿ Reproduction steps

At first, there're a few monitors, it works fine.
But now I have 54, it doesn't work well.
In my situation, when you add more and more monitors, this will reproduce.

πŸ‘€ Expected behavior

When opening the homepage, show all monitors immediately.

πŸ˜“ Actual Behavior

The websocket communication is very slowly and sometimes failed.

🐻 Uptime-Kuma Version

1.23.11

πŸ’» Operating System and Arch

k8s

🌐 Browser

Chromium 123.0.6312.105/FireFox 124.0.2

πŸ–₯️ Deployment Environment

  • Runtime: k8s
  • Database: sqlite
  • Filesystem used to store the database on: nfs
  • number of monitors: 54

πŸ“ Relevant log output

Monitor #23 'Group': Failing: Child inaccessible | Interval: 60 seconds | Type: group | Down Count: 0 | Resend Interval: 0



Trace: KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
    at Client_SQLite3.acquireConnection (/app/node_modules/knex/lib/client.js:312:26)
    at async Runner.ensureConnection (/app/node_modules/knex/lib/execution/runner.js:287:28)
    at async Runner.run (/app/node_modules/knex/lib/execution/runner.js:30:19)
    at async RedBeanNode.normalizeRaw (/app/node_modules/redbean-node/dist/redbean-node.js:572:22)
    at async RedBeanNode.getRow (/app/node_modules/redbean-node/dist/redbean-node.js:558:22)
    at async Monitor.calcUptime (/app/server/model/monitor.js:1255:22)
    at async Monitor.sendUptime (/app/server/model/monitor.js:1321:24)
    at async Monitor.sendStats (/app/server/model/monitor.js:1189:13) {
  sql: '\n' +
    '            SELECT\n' +
    '               -- SUM all duration, also trim off the beat out of time window\n' +
    '                SUM(\n' +
    '                    CASE\n' +
    '                        WHEN (JULIANDAY(`time`) - JULIANDAY(?)) * 86400 < duration\n' +
    '                        THEN (JULIANDAY(`time`) - JULIANDAY(?)) * 86400\n' +
    '                        ELSE duration\n' +
    '                    END\n' +
    '                ) AS total_duration,\n' +
    '\n' +
    '               -- SUM all uptime duration, also trim off the beat out of time window\n' +
    '                SUM(\n' +
    '                    CASE\n' +
    '                        WHEN (status = 1 OR status = 3)\n' +
    '                        THEN\n' +
    '                            CASE\n' +
    '                                WHEN (JULIANDAY(`time`) - JULIANDAY(?)) * 86400 < duration\n' +
    '                                    THEN (JULIANDAY(`time`) - JULIANDAY(?)) * 86400\n' +
    '                                ELSE duration\n' +
    '                            END\n' +
    '                        END\n' +
    '                ) AS uptime_duration\n' +
    '            FROM heartbeat\n' +
    '            WHERE time > ?\n' +
    '            AND monitor_id = ?\n' +
    '        ',
  bindings: [
    '2024-03-08 09:49:42',
    '2024-03-08 09:49:42',
    '2024-03-08 09:49:42',
    '2024-03-08 09:49:42',
    '2024-03-08 09:49:42',
    48
  ]
}
@sudoexec sudoexec added the bug Something isn't working label Apr 7, 2024
@louislam louislam added help and removed bug Something isn't working labels Apr 7, 2024
@CommanderStorm
Copy link
Collaborator

CommanderStorm commented Apr 7, 2024

Filesystem used to store the database on: nfs

Please refer to the wiki why using NFS is not a good idea for an database both to prevent db-corruption and performance.

This issue might also be reated to the performance problems of V1 (having to read the entire table) resolved in the upcoming V2.0 release. Please see #4500
=> Have you checked your retention? What is the size of the database?

@CommanderStorm CommanderStorm added the area:deployment related to how uptime kuma can be deployed label Apr 7, 2024
@CommanderStorm CommanderStorm changed the title Uptime Kuma is very slowly Uptime Kuma is very slow when deployed on NFS Apr 7, 2024
@CommanderStorm CommanderStorm added the question Further information is requested label Apr 7, 2024
@sudoexec
Copy link
Author

sudoexec commented Apr 7, 2024

The database size is 228M and there're 1531440 records in heartbeat table.

@CommanderStorm
Copy link
Collaborator

CommanderStorm commented Apr 7, 2024

Okay, so just NFS being unsuitable for running a database.
Please migrate to local storage instead as suggested in the installation guide.

@sudoexec
Copy link
Author

sudoexec commented Apr 8, 2024

Thanks for your help. I migrate the storage to hostpath, and it worked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:deployment related to how uptime kuma can be deployed help question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants