Option to toggle multi-line input "query" text box

[polyfractal]

When working on larger Timelion queries/charts, it can get tedious/confusing to write everything out as a single line. E.g. a single chart I made was very nested and confusing as a single line:

.es('metric:0', metric='avg:value').label("#0 90th surprise"), .es('metric:0', metric='avg:value').showifgreater(.es('metric:0', metric='avg:value').movingaverage(6).sum(.es('metric:0', metric='avg:value').movingstd(6).multiply(3))).bars().yaxis(2).label("#0 anomalies")

But easier to read when formatted as a multi-line:

.es('metric:0', metric='avg:value')
 .label("#0 90th surprise"),

.es('metric:0', metric='avg:value')
 .showifgreater(
   .es('metric:0', metric='avg:value')
    .movingaverage(6)
    .sum(
      .es('metric:0', metric='avg:value')
      .movingstd(6)
      .multiply(3)
    )
  ).bars()
  .yaxis(2)
  .label("#0 anomalies")

I'm not really sure how this would work in the interface, since a multi-line text box would obviously eat up a lot of screen real-estate. Perhaps a drop down? Or a fly-out from the side? Dunno :)

[l8liu]

I saw you used .movingstd(6) in your code. Does the Timelion support moving standard deviation? I couldn't find this function in my Timelion. Could you please let me know? I really need this function to plot a control chart similar as your code. Thanks.

[polyfractal]

@l8liu I don't believe it does natively yet...movingstd() was a custom function I made for a demo. You can find the source here: https://gist.github.com/polyfractal/5c584f515c4dbd2d5a8f

It can be added to Timelion by placing it in kibana/installedPlugins/timelion/series_functions/movingstd.js, removing the optimized bundle (rm kibana/optimize/bundles/timelion.bundle.js) and restarting Kibana.

Note: I'm neither a Kibana dev nor very good at Javascript, so this was just janky code I threw together to work. I'm sure Rashid et. al will be horrified by the snippet :)

Secondary note: I haven't updated my Timelion in a while, so I'm not 100% confident that will work with the most recent version. But it should at least get you close.

[l8liu]

Thank you very much for the help. The function has been installed in my Timelion and it works. However, it returned me much higher values than they were expected. Did you miss a square root when you compute the standard deviation?

[polyfractal]

Doh, yes, I did. Good catch, thanks :)

I'll edit the gist in a sec

[rashidkpc]

@polyfractal pull request :-) Code looks fine to me

[rashidkpc]

Also yeah, multi line input would be nice

[l8liu]

@polyfractal I found a quite weird thing when I used the function movingstd(). When the values that I want to compute std are large, say more than 100, the movingstd() return me correct answers. However, when the when I used the data with small values, especially when the values < 1, movingstd() returned results larger than it should be. The smaller values I used, the larger the error.

For example, I want to calculate 7 day std of a = c(0.01, 0.013, 0.012, 0.01, 0.013, 0.017, 0.013). The movingstd(7) returned me std about 0.04, while the true std should be 0.0024. However, if I compute the std of data set 10000 times of a, i.e., std(a x 10000), the movingstd(7) = 24, which is quite close to the true value. I suspect the data are rounded somewhere before computed the std. The function movingavg() seems don't have this problem. Thanks.

[polyfractal]

@l8liu Yep, I think this is expected unfortunately. See the notes on #35.

Essentially, this is a naive approach and is susceptible to numerical instability if the std << mean, or if the values are very large, or if the number of values is very large. It's because floating points are susceptible to rounding errors and cancellation (loss of significant digits) when working with very small or large values. It should really be replaced with a version that uses compensated summation to reduce floating point rounding error.

Let's move further conversation over to #35 so we don't clutter up this issue.

[pagenbag]

Only noticed this issue now.
+1
I (before I noticed this) opened #103 and started a bit of work on it.

[SimoneTosato]

@pagenbag any news about your implementation of multi-line editor for timelion?

[pagenbag]

@SimoneTosato , for my own project , I'm waiting on the 5.0 release of the elastic stack , so I've sort of paused my own development until then.
So , no , I have not done much else on it for now.

[SimoneTosato]

@pagenbag thank you for your answer, do you have any code to share? those pictures of your implementation on #103 looks good

[pagenbag]

@SimoneTosato - I'm heading home now , but I'll see if I can dig it up tomorrow morning .

[SimoneTosato]

@pagenbag thank you very much

[pagenbag]

@SimoneTosato :( ... looks like that code is on a VM that I decommissioned. (Cant think for what reason I would have deleted that branch from git though)

If I get some time I'll see if I can hack something together again - buit it'll be on the 5.0 kibana branch.

[SimoneTosato]

@pagenbag well, thanks anyway 😄