Skip to content

Commit

Permalink
feat: update rtu dashboard (#1660)
Browse files Browse the repository at this point in the history
* feat: update rtu dashboard

* fix: old links

* feat: add redirect

---------

Co-authored-by: Ian <[email protected]>
  • Loading branch information
ianFar96 and Ian authored Sep 23, 2024
1 parent cb11fa9 commit 7950a85
Show file tree
Hide file tree
Showing 9 changed files with 1,280 additions and 1,109 deletions.
4 changes: 4 additions & 0 deletions 301redirects.json
Original file line number Diff line number Diff line change
Expand Up @@ -1790,5 +1790,9 @@
"/docs/runtime_suite_tools/mlp/deploy": {
"destination": "/docs/runtime_suite_tools/mlp/overview",
"addedOn": "2024-07-31"
},
"/docs/fast_data/monitoring/dashboards/real_time_updater": {
"destination": "/docs/fast_data/monitoring/dashboards/ingestion_services",
"addedOn": "2024-09-20"
}
}
19 changes: 19 additions & 0 deletions docs/fast_data/monitoring/dashboards/ingestion_services.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
id: ingestion_services
title: Ingestion services
sidebar_label: Ingestion services
---

This dashboard focuses solely on the Ingestion services of the Fast Data ([Real-Time Updater](/fast_data/realtime_updater.md) and [Projection Storer](/fast_data/projection_storer.md)) and displays information related to the ingestion process (the saving, updating or deleting of projection records coming from the CDC). Among the features you'll see:

- I/O message rate (ingestion messages process and pr-updates published)
- Consumer lag per topic
- Mongo commands execution time
- CPU/Memory usage of the service
- ...

To discover all the features you can download the dashboard's json <a download target="_blank" href="/docs_files_to_download/dashboards/ingestion-services.json">**here**</a> and import it on your Grafana portal.

:::note
Just like with the other dashboards, be sure to check out the [Fast Data Services metrics](/fast_data/monitoring/overview.md#fast-data-services-metrics) section to make sure the metrics are exposed correctly.
:::
19 changes: 0 additions & 19 deletions docs/fast_data/monitoring/dashboards/real_time_updater.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/fast_data/monitoring/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,6 @@ You can create any custom dashboards using the metrics provided by the Fast Data
- [Debezium CDC Dashboard](/fast_data/monitoring/dashboards/debezium_cdc.md)
- [Kafka Messages Dashboard](/fast_data/monitoring/dashboards/kafka_messages.md)
- [Projection Changes Dashboard](/fast_data/monitoring/dashboards/projection_changes.md)
- [Real Time Updater Dashboard](/fast_data/monitoring/dashboards/real_time_updater.md)
- [Ingestion services Dashboard](/fast_data/monitoring/dashboards/ingestion_services.md)
- [Single Views Dashboard](/fast_data/monitoring/dashboards/single_views.md)
- [Unmapped fields Dashboard](/fast_data/monitoring/dashboards/unmapped_fields.md)
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ To stop running the strategies you must disable the projections changes generati

### 2. Consume all the ingestion messages in queue

Wait for the Real-Time Updaters to consume all the ingestion messages in queue (if possible, scale up the Real-Time Updaters to do it faster). Once the consumer lag is at 0, make sure no more messages are consumed while performing the Single View regeneration. To do so, scale the Real-Time Updaters replicas down to 0. This will accumulate the incoming ingestion messages in their topics, so you can resume the consumption later. To verify the consumers' lag we recommend to use our Grafana dashboards for [Kafka Messages](/fast_data/monitoring/dashboards/kafka_messages.md) or [Real-Time Updaters](/fast_data/monitoring/dashboards/real_time_updater.md) (v7.5.5+).
Wait for the Real-Time Updaters to consume all the ingestion messages in queue (if possible, scale up the Real-Time Updaters to do it faster). Once the consumer lag is at 0, make sure no more messages are consumed while performing the Single View regeneration. To do so, scale the Real-Time Updaters replicas down to 0. This will accumulate the incoming ingestion messages in their topics, so you can resume the consumption later. To verify the consumers' lag we recommend to use our Grafana dashboards for [Kafka Messages](/fast_data/monitoring/dashboards/kafka_messages.md) or [Ingestion services](/fast_data/monitoring/dashboards/ingestion_services.md).

### 3. Manually generate the Projection Changes to trigger all the Single Views' regeneration

Expand Down
2 changes: 1 addition & 1 deletion sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -1334,7 +1334,7 @@
"type": "doc"
},
{
"id": "fast_data/monitoring/dashboards/real_time_updater",
"id": "fast_data/monitoring/dashboards/ingestion_services",
"type": "doc"
},
{
Expand Down
Loading

0 comments on commit 7950a85

Please sign in to comment.