Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing aggregates => LogStash::ConfigurationError: Aggregate plugin: more than one filter which defines timeout options. But only defining once. #119

Open
apple-corps opened this issue Nov 18, 2021 · 3 comments
Open

testing aggregates => LogStash::ConfigurationError: Aggregate plugin: more than one filter which defines timeout options. But only defining once. #119

apple-corps opened this issue Nov 18, 2021 · 3 comments

Comments

@apple-corps
Copy link

apple-corps commented Nov 18, 2021 •
edited
Loading

I'm trying to test my logstash configurations that use the aggregate plugin. I have not had any issues running them locally against the binary as I write them. But now I want to test the behavior expectations of the aggregations. Then I found a blog post and docker container that utilize rspec to test the filters. However I was trying to write a test to check the value of eventId and this ConfigurationError pops up. I'm surprised because it seems to me I'm only defining the timeout option in one aggregate block. But perhaps I'm doing something terribly wrong.

filter.conf

 fillter{
  dissect {
    mapping => {
      "message" => "%{emTimestamp}|%{}%{}| [%{eventId}] %{messageType}: %{msg}"
    }
  }

  if [messageType] == "GET Request" {
    aggregate {
      task_id => "%{eventId}"
      code => "
        map['eventId'] ||= event.get('eventId')
        map['emTimestamp'] ||= event.get('emTimestamp')
        map['actionType'] ||= 'aggregation'
        map['request'] = {'clientSource' => event.get('clientSource'), 'method' => event.get('method'), 'urlPath'=> event.get('urlPath')}"
    }
  }

  if [messageType] == "Response" {
    mutate { gsub => [ "msg","[\\]",""] }
    kv {
      source => "msg"
      value_split => "="
      field_split => "&?&"
      trim_key => "\s"
      trim_value => "\r"
      remove_field => ["msg", "message"]
    }

  json {
    source => ['body']
    target => "parsed"
  }

  ruby {
    code => "mapped = event.get('parsed').map {
      |h|
        {
          'obTimestamp' => h['item']['assets']['timestamp'],
          'fieldId'     => h['item']['assets']['reference']['fieldId'],
          'eventId'     => h['item']['assets']['reference']['eventId'],
          'userId'     => h['item']['assets']['reference']['userId'],
          'startTs'     => h['item']['attributes']['startTs'],
          'endTs'     => h['item']['attributes']['endTs']
        }
    }; event.set('mapped',mapped)"
  }

  aggregate {
    task_id => "%{eventId}"
    code => "
      map['eventId'] ||= event.get('eventId')
      map['emTimestamp'] ||= event.get('emTimestamp')
      map['actionType'] ||= 'aggregration'
      map['response'] = {'body' => event.get('mapped')}"
    }
  }

  if [messageType] == "ACK Response" {
    kv {
      source => "msg"
      value_split => "="
      field_split => "&,&"
      trim_key => "\s"
      trim_value => "\r"
      remove_field => ["msg"]
    }

    aggregate {
      task_id => "{eventId}"
      code => "
        map['eventId'] ||= event.get('eventId')
        map['emTimestamp'] ||= event.get('emTimestamp')
        map['actionType'] ||= 'aggregration'
      "
      push_map_as_event_on_timeout => true
      timeout_task_id_field => "eventId"
      timeout => 60
    }
  }
}

test_spec.rb

require "logstash/devutils/rspec/spec_helper"

# Load the configuration file
@@configuration = String.new
@@configuration << File.read("conf/filter.conf")

describe "Test ACK response type" do

  config(@@configuration)

  # Inject input event/message into the pipeline
  message = File.read("/opt/logstash/logs/tc.log")
  sample("message" => message, "type" => "") do
    # Check the ouput event/message properties
    insist { subject.get("transactionId")  == 104}
    reject { subject.get("tags").include?("_grokparsefailure") }
    reject { subject.get("tags").include?("_dateparsefailure") }
  end
end

error message and test failure

Failures:

  1) Test ACK response type "{"message":"10/23/2021 02:23:23 PM|DEBUG|HttpUtils ..." when processed
     Failure/Error: insist { subject.get("eventId")==104 } 

     LogStash::ConfigurationError:
       Aggregate plugin: For task_id pattern '%{eventId}', there are more than one filter which defines timeout options. All timeout options have to be defined in only one aggregate filter per task_id pattern. Timeout options are : timeout, inactivity_timeout, timeout_code, push_map_as_event_on_timeout, push_previous_map_as_event, timeout_timestamp_field, timeout_task_id_field, timeout_tags
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-filter-aggregate-2.9.1/lib/logstash/filters/aggregate.rb:103:in `block in register'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-filter-aggregate-2.9.1/lib/logstash/filters/aggregate.rb:97:in `register'
     # org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'
     # ./logstash-core/lib/logstash/pipeline.rb:288:in `register_plugin'
     # ./logstash-core/lib/logstash/pipeline.rb:299:in `block in register_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:299:in `register_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:660:in `maybe_setup_out_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:549:in `filter'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:54:in `block in results'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:52:in `block in results'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:68:in `block in subject'
     # ./rspec-tests/test_spec.rb:15:in `block in <main>'
     # ./vendor/bundle/jruby/2.5.0/gems/insist-1.0.0/lib/insist.rb:47:in `value'
     # ./vendor/bundle/jruby/2.5.0/gems/insist-1.0.0/lib/insist/comparators.rb:13:in `=='
     # ./rspec-tests/test_spec.rb:15:in `block in <main>'
     # ./vendor/bundle/jruby/2.5.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
     # ./lib/bootstrap/rspec.rb:31:in `<main>'

Finished in 0.76796 seconds (files took 1.54 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./rspec-tests/test_spec.rb:13 # Test ACK response type "{"message":"10/23/2021 02:23:23 PM|DEBUG|HttpUtils ..." when processed

tc.log

10/23/2021 02:23:23 PM|DEBUG|HttpUtils                              | [104] Response: body =[{},"assets":{},{}],},"part":0}]

To reproduce launch this docker container and follow instructions to volume mount the filter, spec, and tc.log so the text executes. https://github.com/iteratec/logstash-rspec

docker command, populate paths.

    docker run --rm --name logstash-rspecs \
        -v /<path to filter dir>:/opt/logstash/filters-under-test \
        -v /<path to spec dir>:/opt/logstash/rspec-tests  \
        -v /<path to log dir>:/opt/logstash/logs \
        iteratec/logstash-rspec:7.3.0
@apple-corps apple-corps mentioned this issue Nov 28, 2021
Open
@apple-corps
Copy link
Author

I set workers to 1 in a configuration and enabled debugging and still hung up on this. Not sure about the scripting support message. Still failing due to the aggregate ConfigurationError.

Using system java: /usr/bin/java
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
2021-11-29 07:51:02,139 main DEBUG Apache Log4j Core 2.14.0 initializing configuration org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@7d1cfe97
2021-11-29 07:51:02,145 main INFO Cannot initialize scripting support because this JRE does not support it. java.lang.NoClassDefFoundError: javax/script/ScriptEngineFactory
	at java.base/java.lang.ClassLoader.findBootstrapClass(Native Method)
	at java.base/java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1258)
	at java.base/java.lang.System$2.findBootstrapClassOrNull(System.java:2134)
	at java.base/jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull(ClassLoaders.java:118)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:616)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:640)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:616)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:576)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1210)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1221)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1265)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1300)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1385)
	at java.scripting/javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:123)
	at java.scripting/javax.script.ScriptEngineManager.init(ScriptEngineManager.java:87)
	at java.scripting/javax.script.ScriptEngineManager.<init>(ScriptEngineManager.java:62)
	at org.apache.logging.log4j.core.script.ScriptManager.<init>(ScriptManager.java:70)
	at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:219)
	at org.logstash.log.LogstashConfigurationFactory.getConfiguration(LogstashConfigurationFactory.java:68)
	at org.logstash.log.LogstashConfigurationFactory.getConfiguration(LogstashConfigurationFactory.java:40)
	at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:559)
	at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:483)
	at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:323)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:691)
	at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:712)
	at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:267)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:155)
	at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
	at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
	at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:597)
	at org.logstash.secret.store.SecretStoreFactory.<clinit>(SecretStoreFactory.java:75)
	at org.logstash.secret.store.SecretStoreExt.<clinit>(SecretStoreExt.java:32)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at org.jruby.javasupport.JavaSupportImpl.loadJavaClass(JavaSupportImpl.java:157)
	at org.jruby.javasupport.JavaSupportImpl.loadJavaClassVerbose(JavaSupportImpl.java:166)
	at org.jruby.javasupport.JavaClass.forNameVerbose(JavaClass.java:273)
	at org.jruby.javasupport.JavaClass.for_name(JavaClass.java:292)
	at org.jruby.javasupport.JavaClass.for_name(JavaClass.java:288)
	at org.jruby.javasupport.Java.get_proxy_class(Java.java:396)
	at org.jruby.javasupport.JavaUtilities.get_proxy_class(JavaUtilities.java:39)
	at org.jruby.javasupport.JavaUtilities$INVOKER$s$1$0$get_proxy_class.call(JavaUtilities$INVOKER$s$1$0$get_proxy_class.gen)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:172)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:160)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:116)
	at org.jruby.runtime.InterpretedIRBlockBody.yieldDirect(InterpretedIRBlockBody.java:107)
	at org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
	at org.jruby.runtime.Block.yield(Block.java:184)
	at org.jruby.RubyArray.collect(RubyArray.java:2563)
	at org.jruby.RubyArray.map19(RubyArray.java:2577)
	at org.jruby.RubyArray$INVOKER$i$0$0$map19.call(RubyArray$INVOKER$i$0$0$map19.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
	at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:93)
	at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:546)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:160)
	at org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:86)
	at org.jruby.internal.runtime.methods.InterpretedIRMethod.INTERPRET_METHOD(InterpretedIRMethod.java:155)
	at org.jruby.internal.runtime.methods.InterpretedIRMethod.call(InterpretedIRMethod.java:146)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:96)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:81)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:1218)
	at org.jruby.Ruby.loadFile(Ruby.java:2785)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:234)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:887)
	at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:535)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:402)
	at org.jruby.RubyKernel.requireCommon(RubyKernel.java:981)
	at org.jruby.RubyKernel.require(RubyKernel.java:974)
	at org.jruby.RubyKernel$INVOKER$s$1$0$require.call(RubyKernel$INVOKER$s$1$0$require.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:97)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:96)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:81)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:1218)
	at org.jruby.Ruby.loadFile(Ruby.java:2785)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:234)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:887)
	at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:535)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:402)
	at org.jruby.RubyKernel.requireCommon(RubyKernel.java:981)
	at org.jruby.RubyKernel.require(RubyKernel.java:974)
	at org.jruby.RubyKernel$INVOKER$s$1$0$require.call(RubyKernel$INVOKER$s$1$0$require.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:97)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:96)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:81)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:1218)
	at org.jruby.Ruby.loadFile(Ruby.java:2785)
	at org.jruby.runtime.load.LibrarySearcher$ResourceLibrary.load(LibrarySearcher.java:234)
	at org.jruby.runtime.load.LibrarySearcher$FoundLibrary.load(LibrarySearcher.java:34)
	at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:887)
	at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:535)
	at org.jruby.runtime.load.LoadService.require(LoadService.java:402)
	at org.jruby.RubyKernel.requireCommon(RubyKernel.java:981)
	at org.jruby.RubyKernel.require(RubyKernel.java:974)
	at org.jruby.RubyKernel$INVOKER$s$1$0$require.call(RubyKernel$INVOKER$s$1$0$require.gen)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:417)
	at org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:97)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174)
	at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
	at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
	at org.jruby.ir.interpreter.Interpreter.INTERPRET_ROOT(Interpreter.java:96)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:81)
	at org.jruby.ir.interpreter.Interpreter.execute(Interpreter.java:30)
	at org.jruby.ir.IRTranslator.execute(IRTranslator.java:42)
	at org.jruby.Ruby.runInterpreter(Ruby.java:1240)
	at org.jruby.Ruby.runInterpreter(Ruby.java:1244)
	at org.jruby.Ruby.runNormally(Ruby.java:1133)
	at org.jruby.Ruby.runNormally(Ruby.java:1146)
	at org.jruby.Ruby.runFromMain(Ruby.java:958)
	at org.jruby.Main.doRunFromMain(Main.java:400)
	at org.jruby.Main.internalRun(Main.java:292)
	at org.jruby.Main.run(Main.java:234)
	at org.jruby.Main.main(Main.java:206)

2021-11-29 07:51:02,147 main DEBUG PluginManager 'Core' found 125 plugins
2021-11-29 07:51:02,148 main DEBUG PluginManager 'Level' found 0 plugins
2021-11-29 07:51:02,149 main DEBUG PluginManager 'Lookup' found 16 plugins
2021-11-29 07:51:02,151 main DEBUG Building Plugin[name=AppenderRef, class=org.apache.logging.log4j.core.config.AppenderRef].
2021-11-29 07:51:02,157 main DEBUG PluginManager 'TypeConverter' found 26 plugins
2021-11-29 07:51:02,165 main DEBUG createAppenderRef(ref="console", level="null", Filter=null)
2021-11-29 07:51:02,166 main DEBUG Building Plugin[name=root, class=org.apache.logging.log4j.core.config.LoggerConfig$RootLogger].
2021-11-29 07:51:02,167 main DEBUG createLogger(additivity="null", level="DEBUG", includeLocation="null", ={console}, ={}, Configuration(LogstashRspecConfig), Filter=null)
2021-11-29 07:51:02,169 main DEBUG Building Plugin[name=loggers, class=org.apache.logging.log4j.core.config.LoggersPlugin].
2021-11-29 07:51:02,170 main DEBUG createLoggers(={root})
2021-11-29 07:51:02,170 main DEBUG Building Plugin[name=layout, class=org.apache.logging.log4j.core.layout.PatternLayout].
2021-11-29 07:51:02,173 main DEBUG PatternLayout$Builder(pattern="[%d{ISO8601}][%-5p][%-25c] %m%n", PatternSelector=null, Configuration(LogstashRspecConfig), Replace=null, charset="null", alwaysWriteExceptions="null", disableAnsi="null", noConsoleNoAnsi="null", header="null", footer="null")
2021-11-29 07:51:02,173 main DEBUG PluginManager 'Converter' found 46 plugins
2021-11-29 07:51:02,183 main DEBUG Building Plugin[name=appender, class=org.apache.logging.log4j.core.appender.ConsoleAppender].
2021-11-29 07:51:02,188 main DEBUG ConsoleAppender$Builder(target="null", follow="null", direct="null", bufferedIo="null", bufferSize="null", immediateFlush="null", ignoreExceptions="null", PatternLayout([%d{ISO8601}][%-5p][%-25c] %m%n), name="console", Configuration(LogstashRspecConfig), Filter=null, ={})
2021-11-29 07:51:02,190 main DEBUG Starting OutputStreamManager SYSTEM_OUT.false.false
2021-11-29 07:51:02,190 main DEBUG Building Plugin[name=appenders, class=org.apache.logging.log4j.core.config.AppendersPlugin].
2021-11-29 07:51:02,190 main DEBUG createAppenders(={console})
2021-11-29 07:51:02,191 main DEBUG Configuration org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@7d1cfe97 initialized
2021-11-29 07:51:02,196 main DEBUG Starting configuration org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@7d1cfe97
2021-11-29 07:51:02,197 main DEBUG Started configuration org.apache.logging.log4j.core.config.properties.PropertiesConfiguration@7d1cfe97 OK.
2021-11-29 07:51:02,198 main DEBUG Shutting down OutputStreamManager SYSTEM_OUT.false.false-1
2021-11-29 07:51:02,199 main DEBUG OutputStream closed
2021-11-29 07:51:02,199 main DEBUG Shut down OutputStreamManager SYSTEM_OUT.false.false-1, all resources released: true
2021-11-29 07:51:02,200 main DEBUG Appender DefaultConsole-1 stopped with status true
2021-11-29 07:51:02,200 main DEBUG Stopped org.apache.logging.log4j.core.config.DefaultConfiguration@3b11deb6 OK
2021-11-29 07:51:02,273 main DEBUG Registering MBean org.apache.logging.log4j2:type=2530c12
2021-11-29 07:51:02,276 main DEBUG Registering MBean org.apache.logging.log4j2:type=2530c12,component=StatusLogger
2021-11-29 07:51:02,277 main DEBUG Registering MBean org.apache.logging.log4j2:type=2530c12,component=ContextSelector
2021-11-29 07:51:02,278 main DEBUG Registering MBean org.apache.logging.log4j2:type=2530c12,component=Loggers,name=
2021-11-29 07:51:02,279 main DEBUG Registering MBean org.apache.logging.log4j2:type=2530c12,component=Appenders,name=console
2021-11-29 07:51:02,281 main DEBUG org.apache.logging.log4j.core.util.SystemClock supports precise timestamps.
2021-11-29 07:51:02,281 main DEBUG Reconfiguration complete for context[name=2530c12] at URI /src/logstash/log4j2.properties (org.apache.logging.log4j.core.LoggerContext@2433bcd4) with optional ClassLoader: null
2021-11-29 07:51:02,282 main DEBUG Shutdown hook enabled. Registering a new one.
2021-11-29 07:51:02,283 main DEBUG LoggerContext[name=2530c12, org.apache.logging.log4j.core.LoggerContext@2433bcd4] started OK.
Sending Logstash logs to null which is now configured via log4j2.properties
2021-11-29 07:51:04,476 main DEBUG Reconfiguration started for context[name=2530c12] at URI file:////src/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/log4j2.properties (org.apache.logging.log4j.core.LoggerContext@2433bcd4) with optional ClassLoader: null
2021-11-29 07:51:04,476 main DEBUG Using configurationFactory org.apache.logging.log4j.core.config.ConfigurationFactory$Factory@3dc70aa0
2021-11-29 07:51:04,477 main DEBUG Not in a ServletContext environment, thus not loading WebLookup plugin.
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}

Randomized with seed 6451
F

Failures:

  1) Test single log line with 3 trips in response body "09/28/2021 02:45:18 PM|INFO |poll                  ..." when processed
     Failure/Error: expect (subject.get("topic").to eq("2304053812") )

     LogStash::ConfigurationError:
       Aggregate plugin: For task_id pattern '%{transactionId}', there are more than one filter which defines timeout options. All timeout options have to be defined in only one aggregate filter per task_id pattern. Timeout options are : timeout, inactivity_timeout, timeout_code, push_map_as_event_on_timeout, push_previous_map_as_event, timeout_timestamp_field, timeout_task_id_field, timeout_tags
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-filter-aggregate-2.9.1/lib/logstash/filters/aggregate.rb:103:in `block in register'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-filter-aggregate-2.9.1/lib/logstash/filters/aggregate.rb:97:in `register'
     # org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:75:in `register'
     # ./logstash-core/lib/logstash/pipeline.rb:288:in `register_plugin'
     # ./logstash-core/lib/logstash/pipeline.rb:299:in `block in register_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:299:in `register_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:674:in `maybe_setup_out_plugins'
     # ./logstash-core/lib/logstash/pipeline.rb:550:in `filter'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:54:in `block in results'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:52:in `block in results'
     # ./vendor/bundle/jruby/2.5.0/gems/logstash-devutils-1.3.6-java/lib/logstash/devutils/rspec/logstash_helpers.rb:68:in `block in subject'
     # /work/spec/02_filter_spec.rb:124:in `block in <main>'
     # ./vendor/bundle/jruby/2.5.0/gems/rspec-wait-0.0.9/lib/rspec/wait.rb:46:in `block in <main>'
     # ./lib/bootstrap/rspec.rb:31:in `<main>'

Finished in 0.58592 seconds (files took 1.48 seconds to load)
1 example, 1 failure

Failed examples:

rspec /work/spec/02_filter_spec.rb:118 # Test single log line with 3 trips in response body "09/28/2021 02:45:18 PM|INFO |poll                  ..." when processed

Randomized with seed 6451

This was referenced Nov 29, 2021
Closed
Open
Open
@apple-corps
Copy link
Author

Another set of reproduction steps given here: cameronkerrnz/logstash-plugin-dev#5 (comment)

@apple-corps
Copy link
Author

Assuming 2021-11-29 07:51:02,145 main INFO Cannot initialize scripting support because this JRE does not support it. java.lang.NoClassDefFoundError: javax/script/ScriptEngineFactory

is unrelated to my issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@apple-corps