Code for the interactive sttistical aggregation and visualisation of multidimensional data in ROOT or native Python formats (Panda, numpy).
Support for ROOT data structures:
- TTree and TTreeFormula, aliases ...
- TFormula, or any static Root/AliRoot functions.
- RDataFrame <-> awkward - work in progress
Root and PyRoot (AliRoot/O2) data structures could be used as input data sources. However, the code also works with pandas only, without the need to have the ROOT package installed. Internally, these data structures are converted into the Bokeh CDS (ColumnDataSource) or in our RootInteractive CDS for the NDimensional histograms, projections and aggregated information.
- Interactive, easily configurable visualisation of non-binned and binned data.
- Interactive n-dimensional histogramming/projection and derived aggregated information extraction
- Client/server application Jupyter, Bokeh
- Standalone client application - (Bokeh Standalone Dashboard)
- Lossy and lossless data compression (server --> client)
- ROOT and RDataFrame tools and interfaces
The figure array declaration is used as an argument in bokehDrawSA to create an array of figures/graphs/scatter plots/ Unbined or binned (Ndimension histogram and derived statistics/projection) bokeh data sources and derived variables and aggregated statistics can be used for drawing.
The declarative programming used in bokehDrawSA is a type of coding where developers express the computational logic without having to programme the control flow of each process. This can help simplify coding, as developers only need to describe what they want the programme to achieve, rather than explicitly prescribing the steps or commands required to achieve the desired result.
The interactive visualization is declared in the 6 arrays as ine the example bellow
bokehDrawSA.fromArray(df, None, figureArray, widgetParams, layout=figureLayoutDesc, tooltips=tooltips, parameterArray=parameterArray,
widgetLayout=widgetLayoutDesc, sizing_mode="scale_width", nPointRender=300,
aliasArray=aliasArray, histogramArray=histoArray,arrayCompression=arrayCompression)- see READMEfigure
- Defining scatter/histogram/derived figures using input data source
- Example declaration of the figure from data source with columns ABCD
figureArray = [ [['A'], ['A*A-C*C'], {"size": 2, "colorZvar": "A", "errY": "errY", "errX":"0.01"}], [['A'], ['C+A', 'C-A', 'A/A']], [['B'], ['C+B', 'C-B'], { "colorZvar": "colorZ", "errY": "errY", "rescaleColorMapper": True}], [['D'], ['(A+B+C)*D'], {"colorZvar": "colorZ", "size": 10, "errY": "errY"} ], [['D'], ['D*10'], {"errY": "errY"}], {"size":"size", "legend_options": {"label_text_font_size": "legendFontSize"}} ]
- Defining interactive ND histogramsand derived statistics, updated based on the user selection, resp. by parametriz
- see READMEhistogram
- Example of creating a 3D histogram showing mean, sum and standard in the projection with colour code in the second dimension
histoArray = [ {"name": "histoABC", "variables": ["(A+C)/2", "B", "C"], "nbins": [8, 10, 12], "weights": "D", "axis": [0], "sum_range": [[.25, .75]]}, ] figureArray = [ [['bin_center_1'], ['mean']], [['bin_center_1'], ['sum_0']], [['bin_center_1'], ['std']], {"source": "histoABC_0", "colorZvar": "bin_center_2", "size": 7} ]
- see READMEaliase
- javascrript function with which you can define derived variables on the client. Used e.g. to parameterise the selection, histogram weights, efficiencies
- newly created variables can be used in histogramArray, figureAray, aliasArray
- Dependency trees to ensure consistency of aliases and the correct order of evaluation of derived variables and use in visualisation.
- Example declaration:
aliasArray = [ # They can also be used as selection (boolen) used e.g. for histogram weights { "name": "C_accepted", "variables": ["C"], "parameters": ["C_cut"], "func": "return C < C_cut" }, # User-defined JS columns can also be created in histograms by specifying the context (CDS) parameter { "name": "efficiency_A", "variables": ["entries", "entries_C_cut"], "func": "return entries_C_cut / entries", "context": "histoA" }, # Shorthand notation - only for scalar functions ("effC", "entries_C_cut / bin_count", "histoAC"), ]
- READMElayout
- Layout declared by and dictionary(tabs)/array of figure IDs (index or name ID)
- Properties per row/simple layout/tab layout can be specified. More local properties have priority.
- Example declaration:
layout = { "A": [ [0, 1, 2, {'commonX': 1, 'y_visible': 1, 'x_visible':1, 'plot_height': 300}], {'plot_height': 100, 'sizing_mode': 'scale_width', 'y_visible' : 2} ], "B": [ [3, 4, {'commonX': 1, 'y_visible': 3, 'x_visible':1, 'plot_height': 100}], {'plot_height': 100, 'sizing_mode': 'scale_width', 'y_visible' : 2} ] }
- see READMElayoutWidget
- Layout declared by and dictionary(tabs)/array of figure IDs (index or name ID)
- Properties per row/simple layout/tab layout can be specified. More local properties have priority.
- Example declaration:
- simple layout
widgetLayoutKine=[ ["dca0","tgl","qPt","ncl"], ["dEdxtot","dEdxmax","mTime0"], ["hasA","Run","IR","isMC"], {'sizing_mode': 'scale_width'} ]
- composed layout:
widgetLayoutDesc={ "Select":widgetLayoutKine, "Histograms":[["nbinsX","nbinsY", "varX","yAxisTransform"], {'sizing_mode': 'scale_width'}], "Legend": figureParameters['legend']['widgetLayout'], "Markers":["markerSize"] }
- simple layout
- see READMEcompression
- Significant data compression of the data (up to O(5-10%)) for server- > client transmission and for storage in the html file.
- Compression depends heavily on the entropy of the data after lossy compression and on data repetition
- Lossy and lossles compression expressed by regular expression per columns
- In realistic use cases facto 10-100 compression achieved
- further compression - using javascript aliases on client instead data transfer
- Example declaration:
arrayCompressionParam=[(".*conv.*Sigma.*",[("relative",7), ("code",0), ("zip",0), ("base64",0)]), (".*delta.*",[("relative",10), ("code",0), ("zip",0), ("base64",0)]), (".*i2.*",[("relative",7), ("code",0), ("zip",0), ("base64",0)]), (".*",[("relative",8), ("code",0), ("zip",0), ("base64",0)])]
- Wrappers for decision trees and Neural Net
- Provides interface for the reducible, irreducible errors, proability density function
- Local linear forest, resp. local kernel regression
- RootInteractive github (source code)
- Documentation server at CERN (TODO -add reular update)
- https://rootinteractive.web.cern.ch/RootInteractive/html/
- Not yet regularly updated - TODO
- /eos/user/r/rootinteractive/www/html/
- 1.) Bokeh draw standalone (graphs,compression, down-sampling)
- 2.) N dimensional histogramming on client (data aggregation)
- 3.) Custom function on client:
Sevearal ALICE use case (detector calibration, QA/QC)
-
Support material for RCU note [N2]
- [D1] Visualization of the common-mode effect dependencies using ROOT interactive ( 11 Dimensions)
- [D2] Visualization of the ion-tail fit parameters and correction graphs using ROOT interactive (12 Dimensions)
- [D3] Visualization of the toy MC results using ROOT interactive (13 Dimensions)
-
Support material for V0 reconstruction studies [P1]
- [D4] Interactive invariant mass histogram dashboards (6+2 Dimensions)
- [D5] Pt and invariant mass performance maps dashboards
-
QA and production preparation :
- [D6] QA comparison of ongoing MC and raw data production (LHC18q,r, LHC18c,LHC16f,LHC17g..) See interactive dashboards in agenda of calibration/tracking meeting:
-
PID
-
Fast MCkalman and event display
- [D8] Space charge distortion calibration (Run3) and performance optimization (Run2, Alice3) - [P9]
- [D9] High dEdx (spallation product) reconstruction and magnetic monopole tracking
-
Space charge distortion calibration
- [D10] digital current grouping and factorization studies