Skip to content

Latest commit

 

History

History
337 lines (215 loc) · 16.3 KB

CHANGELOG.next.asciidoc

File metadata and controls

337 lines (215 loc) · 16.3 KB
Raw

Beats version HEAD

Breaking changes

Affecting all Beats - queue.mem.events is changing from 4096 to 3200. - queue.mem.flush.min_events is changing from 2048 to 1600. - queue.mem.flush.timeout is changing from 1s to 10s. - output.elasticsearch.bulk_max_size is changing from 50 to 1600. - output.elasticsearch.idle_connection_timeout is changing from 60s to 3s.

Auditbeat

Filebeat

Heartbeat - Decreases the ES default timeout to 10 for the load monitor state requests

Metricbeat

Osquerybeat

  • Upgrade to osquery 5.10.2. 37115

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats - Support for multiline zookeeper logs 2496 - Add checks to ensure reloading of units if the configuration actually changed. 34346 - Fix namespacing on self-monitoring 32336 - Fix namespacing on self-monitoring 32336 - Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964 - Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031 - 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider - 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field - Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640 - Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820 - Support build of projects outside of beats directory 36126

Auditbeat

Filebeat

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

  • Fix panic when sqs input metrics getter is invoked 36101 36077

  • Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

  • Fix Filebeat Cisco module with missing escape character 36325 36326

  • Fix panic when redact option is not provided to CEL input. 36387 36388

  • Remove 'onFilteredOut' and 'onDroppedOnPublish' callback logs 36299 36399

  • Added a fix for Crowdstrike pipeline handling process arrays 36496

  • Ensure winlog input retains metric collection when handling recoverable errors. 36479 36483

  • Revert error introduced in 35734 when symlinks can’t be resolved in filestream. 36557

  • Fix ignoring external input configuration in take_over: true mode 36378 36395

  • Add validation to http_endpoint config for empty URL 36816 36772

  • Fix merging of array fields(processors, paths, parsers) in configurations generated from hints and default config. 36838 36857

  • Fix handling of response errors in HTTPJSON and CEL request trace logging. 36956

  • Do not error when Okta API returns no data. 37092

  • Fix request body close behaviour in HTTP_Endpoint when handling GZIP compressed content. 37091

  • Make CEL input now global evaluate to a time in UTC. 37159

Heartbeat

  • Fix panics when parsing dereferencing invalid parsed url. 34702

Metricbeat

  • in module/windows/perfmon, changed collection method of the second counter value required to create a displayable value 32305

  • Fix and improve AWS metric period calculation to avoid zero-length intervals 32724

  • Add missing cluster metadata to k8s module metricsets 32979 33032

  • Add GCP CloudSQL region filter 32943

  • Fix logstash cgroup mappings 33131

  • Remove unused elasticsearch.node_stats.indices.bulk.avg_time.bytes mapping 33263

  • Make generic SQL GA 34637

  • Collect missing remote_cluster in elasticsearch ccr metricset 34957

  • Add context with timeout in AWS API calls 35425

  • Fix EC2 host.cpu.usage 35717

  • Add option in SQL module to execute queries for all dbs. 35688

  • Add remaining dimensions for azure storage account to make them available for tsdb enablement. 36331

  • Add missing 'TransactionType' dimension for Azure Storage Account. 36413

  • Add log error when statsd server fails to start 36477

  • Fix CassandraConnectionClosures metric configuration 34742

  • Fix event mapping implementation for statsd module 36925

  • The region and availability_zone ecs fields nested within the cloud field. 37015

  • Fix CPU and memory metrics collection from privileged process on Windows 17314https://github.com/elastic/beats/pull/37027[37027]

  • Enhanced Azure Metrics metricset with refined grouping logic and resolved duplication issues for TSDB compatibility 36823

  • Fix memory leak on Windows 37142 37171

  • Fix unintended skip in metric collection on Azure Monitor 37204 37203

  • Fix the "api-version query parameter (?api-version=) is required for all requests" error in Azure Billing. 37158

  • Add memory hard limit from container metadata and remove usage percentage in AWS Fargate. 37194

Osquerybeat

Packetbeat

Winlogbeat

Elastic Logging Plugin

Added

Affecting all Beats

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • When running under Elastic-Agent the status is now reported per Unit instead of the whole Beat 35874 36183

  • Add warning message to SysV init scripts for RPM-based systems that lack /etc/rc.d/init.d/functions. 35708 36188

  • Mark translate_sid processor is GA. 36279 36280

  • dns processor: Add support for forward lookups (A, AAAA, and TXT). 11416 36394

  • Mark syslog processor as GA, improve docs about how processor handles syslog messages. 36416 36417

  • Add support for AWS external IDs. 36321 36322

  • [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor 36506 Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

  • allow queue configuration settings to be set under the output. 35615 36788

  • Beats will now connect to older Elasticsearch instances by default 36884

  • Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments

  • elasticsearch output now supports idle_connection_timeout. 35615 36843

  • Upgrade golang/x/net to v0.17.0. Updates the publicsuffix table used by the registered_domain processor. 36969 Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

  • Upgrade to Go 1.20.11. 37123

  • The Elasticsearch output can now configure performance presets with the preset configuration field. 37259

Auditbeat

  • Add ignore_errors option to audit module. 15768 36851

  • Fix copy arguments for strict aligned architectures. 36976

Filebeat

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Add oracle authentication messages parsing 35127

  • Add clean_session configuration setting for MQTT input. 16204

  • Add fingerprint mode for the filestream scanner and new file identity based on it 34419 35734

  • Add file system metadata to events ingested via filestream 35801 36065

  • Add support for localstack based input integration testing 35727

  • Allow parsing bytes in and bytes out as long integer in CEF processor. 36100 36108

  • Add support for registered owners and users to AzureAD entity analytics provider. 36092

  • Add support for endpoint resolver in AWS config 36208

  • Added support for Okta OAuth2 provider in the httpjson input. 36273

  • Add support of the interval parameter in Salesforce setupaudittrail-rest fileset. 35917 35938

  • Add device handling to Okta input package for entity analytics. 36049

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30916 36286

  • [Azure] Add input metrics to the azure-eventhub input. 35739

  • Reduce HTTPJSON metrics allocations. 36282

  • Add support for a simplified input configuraton when running under Elastic-Agent 36390

  • Make HTTPJSON response body decoding errors more informative. 36481

  • Allow fine-grained control of entity analytics API requests for Okta provider. 36440 36492

  • Add support for expanding journald.process.capabilities into the human-readable effective capabilities in the ECS process.thread.capabilities.effective field. 36454 36470

  • Allow fine-grained control of entity analytics API requests for AzureAD provider. 36440 36441

  • For request tracer logging in CEL and httpjson the request and response body are no longer included in event.original. The body is still present in http.{request,response}.body.content. 36531

  • Added support for Okta OAuth2 provider in the CEL input. 36336 36521

  • Improve error logging in HTTPJSON input. 36529

  • Disable warning message about ingest pipeline loading when running under Elastic Agent. 36659

  • Add input metrics to http_endpoint input. 36402 36427

  • Remove Event Normalization from GCP PubSub Input. 36716

  • Update mito CEL extension library to v1.6.0. 36651

  • Added support for new features & removed partial save mechanism in the Azure Blob Storage input. 35126 36690

  • Improve template evaluation logging for HTTPJSON input. 36668

  • Add CEL partial value debug function. 36652

  • Added support for new features and removed partial save mechanism in the GCS input. 35847 36713

  • Re-use buffers to optimise memory allocation in fingerprint mode of filestream 36736

  • Allow http_endpoint input to receive PUT and PATCH requests. 36734

  • Add cache processor. 36786

  • Avoid unwanted publication of Azure entity records. 36753

  • Avoid unwanted publication of Okta entity records. 36770

  • Add support for Digest Authentication to CEL input. 35514 36932

  • Use filestream input with file_identity.fingerprint as default for hints autodiscover. 35984 36950

  • Add network processor in addition to interface based direction resolution. 37023

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

  • Make CEL input log current transaction ID when request tracing is turned on. 37065

  • Made Azure Blob Storage input GA and updated docs accordingly. 37128

  • Add request trace logging to http_endpoint input. 36951 36957

  • Made GCS input GA and updated docs accordingly. 37127

  • Suppress and log max HTTP request retry errors in CEL input. 37160

  • Prevent CEL input from re-entering the eval loop when an evaluation failed. 37161

  • Update CEL extensions library to v1.7.0. 37172

Auditbeat

  • Upgrade go-libaudit to v2.4.0. 36776 36964

  • Add a /inputs/ route to the HTTP monitoring endpoint that exposes metrics for each dataset instance. 36971

Libbeat

Heartbeat - Added status to monitor run log report. - Capture and log the individual connection metrics for all the lightweight monitors

Metricbeat

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Add metrics grouping by dimensions and time to Azure app insights 36634

  • Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms 36647

  • Enhance GCP billing with detailed tables identification, additional fields, and optimized data handling. 36902

  • Add a /inputs/ route to the HTTP monitoring endpoint that exposes metrics for each metricset instance. 36971

  • Add linux IO metrics to system/process 37213

  • Add new memory/cgroup metrics to Kibana module 37232

Osquerybeat

Packetbeat

Packetbeat

Winlogbeat

  • Make ingest pipeline routing robust to letter case of channel names for forwarded events. 36670 36899

  • Document minimum permissions required for local user account. 15773 37176

  • Bump Windows Npcap version to v1.78. 37300 37370

Functionbeat

Winlogbeat

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues