Fields not supported for ASN database

[pemontto]

Logstash crashes when supplying valid fields for the ASN database.

@filter=<LogStash::Filters::GeoIP id=>\"enrich: src asn\", cache_size=>10000, source=>\"src_ip\", target=>\"src_asn\", database=>\"GeoLite2-ASN.mmdb\", fields=>[\"as_org\", \"asn\"], enable_metric=>true, periodic_flush=>false, lru_cache_size=>1000>>", :error=>"illegal field value as_org. valid values are [AUTONOMOUS_SYSTEM_NUMBER, AUTONOMOUS_SYSTEM_ORGANIZATION, CITY_NAME, COUNTRY_NAME, CONTINENT_CODE, CONTINENT_NAME, COUNTRY_CODE2, COUNTRY_CODE3, IP, ISP, POSTAL_CODE, DMA_CODE, REGION_NAME, REGION_CODE, TIMEZONE, LOCATION, LATITUDE, LONGITUDE, ORGANIZATION]"}

• Version: Logstash 5.4.3, logstash-filter-geoip 4.2.1
• Operating System: Centos7
• Config File (if you have sensitive info, please remove it):

    geoip {
      id => "enrich: dest asn"
      cache_size => 10000
      source => "dest_ip"
      target => "dest_asn"
      database => "GeoLite2-ASN.mmdb"
      fields => [ "as_org", "asn"]
       tag_on_failure => []
    }

• Sample Data:
  {"src_ip": "2.8.8.8"}

I was expecting the code to allow any number of fields and do a best effort to extract them rather than have a static list which can get outdated.

[omnidepp]

I can confirm, that it behaves strangely with asn/isp database (getting the same error).

This worked for me, though: Specify the fields you want like this:

geoip {
      source => "id.resp_h"
      target => "resp_isp"
      database => "/opt/geoip/GeoIP2-ISP.mmdb"
      fields => ["ISP", "AUTONOMOUS_SYSTEM_NUMBER", "AUTONOMOUS_SYSTEM_ORGANIZATION", "ORGANIZATION"]
    }

[sliddjur]

Can confirm this happens for me too on logstash-filter-geoip (6.0.3)

I am using elasticsearch 7.9.3, and I have a new (not legacy) index template called logstash-*.

If I comment out this line: #fields => [ as_org, asn ] in my geoip filter, the error goes away.

Nov 17 21: 58: 34 elastic-srv logstash[1442798]: [2020 - 11 - 17 T21: 58: 34, 475][ERROR][logstash.agent] Failed to execute action {
    : id =>: main,: action_type => LogStash::ConvergeResult::FailedAction,: message => "Could not execute action: PipelineAction::Reload<main>, action_result: false",: backtrace => nil
}
Nov 17 21: 58: 34 elastic-srv logstash[1442798]: [2020 - 11 - 17 T21: 58: 34, 501][INFO][logstash.outputs.elasticsearch][main] Attempting to install template {
    : manage_template => {
        "index_patterns" => "logstash-*", "version" => 60001, "settings" => {
            "index.refresh_interval" => "5s", "number_of_shards" => 1
        }, "mappings" => {
            "dynamic_templates" => [{
                "message_field" => {
                    "path_match" => "message", "match_mapping_type" => "string", "mapping" => {
                        "type" => "text", "norms" => false
                    }
                }
            }, {
                "string_fields" => {
                    "match" => "*", "match_mapping_type" => "string", "mapping" => {
                        "type" => "text", "norms" => false, "fields" => {
                            "keyword" => {
                                "type" => "keyword", "ignore_above" => 256
                            }
                        }
                    }
                }
            }], "properties" => {
                "@timestamp" => {
                    "type" => "date"
                }, "@version" => {
                    "type" => "keyword"
                }, "geoip" => {
                    "dynamic" => true, "properties" => {
                        "ip" => {
                            "type" => "ip"
                        }, "location" => {
                            "type" => "geo_point"
                        }, "latitude" => {
                            "type" => "half_float"
                        }, "longitude" => {
                            "type" => "half_float"
                        }
                    }
                }
            }
        }
    }
}