v203.0.0

New Feature

• Add ability to limit the number of messages in the queue. Will stop ingesting new messages when the limit is reached. - 524b3a3
  • This will prevent the redis.aof file to become larger than the available memory if the queue dies.

properties:
  redis:
    congestion_interval: 10
    congestion_threshold: 1000000

Improvements

• Remove monit network check on the Ingestor - 468015a

  • Sometimes, monit keeps restarting the process without waiting enough time.

• Better readability on filter_pre.conf - 3e9026a

• Try to extract the deployment name after all rules - baefb33 ef5f130

• Compile python3 and HAProxy in parallel - 414349b

• Move blobstore to AWS EU - adc5706

• Don't use static IPs for the Parsers by default - 5e64ef1

• Document ARP issue - 2c26f8e

• Calculate Redis max memory as a percentage of system memory - d07ec70

  • Defaults to 90% of VM RAM. Configurable by defining redis.maxmemory property:

  properties:
    redis:
      maxmemory: 90

• Bump logstash version to 2.3.1 - 07dc0f7

Bug fixes:

• Templating of curator specs - 3a83f83

v202.0.0

RELEASE NOTES

New features

• Added support for routing in the ElasticSearch job via the logstash_parser.elasticsearch_routing spec. Use only if you know what you are doing.

Bug fixes

• nats-to-syslog now writes its logs into a file under /var/vcap/sys/log
• Default index template doesn't make unnecessary string fields get analysed by elasticsearch
• Various fixes to parsing rules
• Fixed race condition that caused the logstash-* index template to be uploaded to ElasticSearch after the first index was created
• the haproxy VM was renamed to ls-router
• Updated support scripts for better process management
• haproxy on ls-router now sends its logs to the cluster monitor by default
• ElasticSearch log level is now INFO by default

v201.0.16

Bug fixes:

• Limit memory usage of kibana process to support VMs with lower amount of system memory.

properties:
  kibana:
    memory_limit: 80 # trigger GC at 80% of system memory

• Minor changes to NATS heartbeat log parsing logic

New features:
The cluster monitor has a different set of credentials from now on (used to be the same basic auth as the kibana user)

properties:
  haproxy:
    cluster_monitor:
      auth:
        username: username
        password: password

v201.0.15

Improvement

• parse timestamps with nanosecond precision if data is present

Bug fixes

Parser

• temporary fix for bug in logstash-2.2.3 redis plugin that caused the parser process to fill up the disk with error logs

syslog ingestor

• Issue #221: ignore logs that have an empty payload (eg: monit healthcheck)

v201.0.14

• Fix issue with HAProxy job:
  • Look for certificate in the correct path when ssl enabled.
• Fix issue in the manifest stub:
  • Get haproxy ip from first network
• Upgrade to manually packaged logstash-2.2.3 with all plugins
• #220 Fix - remove temp fix for RabbitMQ plugin issue

v201.0.12

• Merge pull request #217 from onemedical/elasticsearch-timeout
• Allow if_looks_like_json filter to be enabled
  • This filter is disabled by default. Set enable_json_filter to true in your manifest to enable it:

properties:
  logstash_parser:
    enable_json_filter: true

• if_looks_like_json should parse string timestamps
  • @timestamp holds the timestamp with millisecond precision
  • @timestamp_ns holds the nanosecond of the @timestamp time.

v201.0.11

ADDS an end to end smoke test errand that sends a syslog log file to the haproxy; and then polls until the log file is available in elasticsearch. Use this to check that your ingestion pipeline is working after a deploy; or run it continuously to monitor that your cluster is working.

bosh run errand smoke-tests

v201.0.10

This patch release makes some minor changes to the if_it_looks_like_json parsing rules to:

• Add @source.program to the list of fields to extract a JSON key name from
• When JSON contains UNIX timestamps with nanosecond precision; extract to @timestamp with millisecond precision and store the original ns value @timestamp_ns

v201.0.9

Route traffic to cluster monitor UI via haproxy

Authentication: The same password as configured for the main kibana UI is used.

properties:
  haproxy:
    cluster_monitor:
      inbound_port: 8080
      backend_port: 5601
      backend_servers: [cluster_monitor IP]

v201.0.7

Test release for automated CI.