You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When the dead_letter_input plugin is used to investigate failing log events in the dead letter queue, I normally use the commit_offsets option set to false and I start an additional instance of Logstash with a minimal config to make the content of the DLQ visible as suggested in https://www.elastic.co/guide/en/logstash/7.0/dead-letter-queues.html#processing-dlq-events.
The problem I have is, that with this config Logstash will never complete the task because the default behaviour of the dead_letter_input plugin is to poll the DLQ infinitly for new arriving events. Due to this I have to kill Logstash with Ctrl+c after some (hopefully long enough) period of time.
During the investigation phase it would be nice to have an option, which instructs the dead_letter_input plugin to stop processing when it has catched up to the end of the DLQ in a similar way as the stdin input plugin does it, when stdin is closed.
A possible implementation of this feature could be based on a duration, during which no new events are seen in the DLQ. This also would make it possible to post-process the output of Logstash with other tools (in our case with a pipe to jq ).
The text was updated successfully, but these errors were encountered:
When the dead_letter_input plugin is used to investigate failing log events in the dead letter queue, I normally use the
commit_offsets
option set tofalse
and I start an additional instance of Logstash with a minimal config to make the content of the DLQ visible as suggested in https://www.elastic.co/guide/en/logstash/7.0/dead-letter-queues.html#processing-dlq-events.The problem I have is, that with this config Logstash will never complete the task because the default behaviour of the dead_letter_input plugin is to poll the DLQ infinitly for new arriving events. Due to this I have to kill Logstash with
Ctrl+c
after some (hopefully long enough) period of time.During the investigation phase it would be nice to have an option, which instructs the dead_letter_input plugin to stop processing when it has catched up to the end of the DLQ in a similar way as the stdin input plugin does it, when stdin is closed.
A possible implementation of this feature could be based on a duration, during which no new events are seen in the DLQ. This also would make it possible to post-process the output of Logstash with other tools (in our case with a pipe to
jq
).The text was updated successfully, but these errors were encountered: