Skip to content

Commit

Permalink
Merge pull request #50 from c-dilks/range-based-it
Browse files Browse the repository at this point in the history
refactor!: replace `hipo::bank::iterator` with `hipo::bank::rowlist`
  • Loading branch information
gavalian authored May 6, 2024
2 parents d0805e3 + 3b4e642 commit e659099
Show file tree
Hide file tree
Showing 12 changed files with 562 additions and 308 deletions.
86 changes: 84 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ defaults:

jobs:

build:
build_cmake:
name: CMake build
strategy:
fail-fast: false
matrix:
Expand All @@ -27,8 +28,89 @@ jobs:
- name: build
run: |
cmake -S . -B build -DCMAKE_INSTALL_PREFIX=install
cmake --build build -j2
cmake --build build -j4
cmake --install build
- run: brew install tree
if: ${{ matrix.runner == 'macos-latest' }}
- run: tree install

build_make:
name: Makefile build
strategy:
fail-fast: false
matrix:
runner: [ ubuntu-latest, macos-latest ]
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v4
with:
path: hipo_src
submodules: recursive # FIXME: need `lz4` submoudle for Makefile, whereas `cmake` does not
- name: build
working-directory: hipo_src
run: |
make -j4
make -j4 -C examples
- run: brew install tree
if: ${{ matrix.runner == 'macos-latest' }}
- run: tree hipo_src
- name: tar
run: tar cavf hipo_src{.tar.zst,}
- uses: actions/upload-artifact@v4
with:
name: build_make_${{ matrix.runner }}
retention-days: 5
path: hipo_src.tar.zst

download_test_data:
name: Download test data
runs-on: ubuntu-latest
env:
type: physics
steps:
- uses: actions/cache@v4
id: cache
with:
key: test_data
path: test_data.hipo
lookup-only: true
- name: download
if: ${{ steps.cache.outputs.cache-hit != 'true' }}
run: |
# FIXME: it would be better to generate our own HIPO file here, rather than assuming this file will continue to exist
wget -nv --no-check-certificate http://clasweb.jlab.org/clas12offline/distribution/clas12-timeline/validation_files_${{ env.type }}.tar.zst
tar xvf validation_files_${{ env.type }}.tar.zst
mv -v $(find validation_files -type f -name "*.hipo" | head -n1) test_data.hipo
examples:
name: Run examples
needs: [ download_test_data, build_make ]
strategy:
fail-fast: false
matrix:
runner: [ ubuntu-latest, macos-latest ]
runs-on: ${{ matrix.runner }}
steps:
- uses: actions/download-artifact@v4
with:
name: build_make_${{ matrix.runner }}
- name: untar artifacts
run: |
ls *.tar.zst | xargs -I{} tar xvf {}
rm *.tar.zst
mv -v hipo_src/* ./
rm -rf hipo_src
- name: get test data
uses: actions/cache/restore@v4
with:
key: test_data
path: test_data.hipo
- run: examples/writeFile.exe
- run: examples/readFile.exe example_output.hipo
- run: examples/readFileTags.exe
- run: examples/showFile.exe example_output.hipo event::particle
- run: examples/readJson.exe test_data.hipo
- run: examples/tupleFile.exe
- run: examples/writeUserHeader.exe
- run: examples/bankRowList.exe test_data.hipo
- run: examples/histograms.exe test_data.hipo
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
*.o
*.exe
*.hipo
lib
slib
__pycache__
build
56 changes: 8 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,60 +65,20 @@ Output:

<img src="https://github.com/gavalian/hipo/blob/master/documents/screenshots/histogram_vz.png" width="800">

## Iterators
## Row Lists

Iterators can be used to read through bank. There are several ways to construct iterators, through expressions and
through lambda functions. Here is an example with expressions:
Row lists (`hipo::bank::rowlist`) can be used to iterate through a bank's rows. They may also be used for filtering banks (`hipo::bank::rowlist::filter`), using expressions or lambda functions. See [`examples/bankRowList.cc`](/examples/bankRowList.cc) for examples.

```c++
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Event"});
while( r.next(list)){
hipo::iterator it = hipo::iterator::reduce(list[0],"charge!=0");
for(it.begin(); !it.end(); it.next()){
printf("\t pid [%d] = %d\n",it.index(), list[0].getInt(0,it.index()));
}
}
```
Here is an example with lambda function:
```c++
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Event"});
std::function charged = [](hipo::bank &b, int row) { return b.getInt("charge",row)!=0 ? 1.0 : 0.0;};
while( r.next(list)){
hipo::iterator it = hipo::iterator::reduce(charged,list[0]);
for(it.begin(); !it.end(); it.next()){
printf("\t pid [%d] = %d\n",it.index(), list[0].getInt(0,it.index()));
}
}
If you want to loop over **all** of a bank's rows (not filtered or reduced):
```cpp
for(int row = 0; row < bank.getRows(); row++)
```

Iterators can be used also to get indicies of the refernce bank. For example:

```c++
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Calorimeter"});

while( r.next(list)){
int status = list[0].getInt("status",0);
if(list[0].getInt(0,0)==11&&abs(status)>=2000&&abs(status)<3000){
hipo::iterator it = hipo::iterator::link(list[1],0,1);
double energy = 0.0;
for(it.begin(); !it.end(); it.next()){
energy += list[1].getFloat("energy",it.index());
}
printf("total energy = %f\n",energy);
}
}
If you want to loop over the filtered (reduced) set of rows, use `getRowList()` instead; if the row list has not yet been filtered, this will loop over all the banks rows:
```cpp
for(auto const& row : bank.getRowList())
```

In the example above, the link function will return an iterator for the bank "REC::Calorimeter"
where the column number 1 (which is "pindex") has a value of 0. Then by itarating over the returned
indicies the total energy can be calculated.
## Analysis

A simple analysis package (include file only) is included in the hipo package for
Expand Down
5 changes: 4 additions & 1 deletion examples/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,14 @@ LD := g++
LDFLAGS :=


all: writeFile readFile readFileTags showFile schema writeUserHeader tupleFile readJson dataframe
all: writeFile readFile readFileTags showFile schema writeUserHeader tupleFile readJson dataframe histograms bankRowList

histograms: histograms.o
$(CXX) -o histograms.exe $< $(HIPOLIBS) $(LZ4LIBS)

bankRowList: bankRowList.o
$(CXX) -o bankRowList.exe $< $(HIPOLIBS) $(LZ4LIBS)

readEvents: readEvents.o
$(CXX) -o readEvents.exe $< $(HIPOLIBS) $(LZ4LIBS)

Expand Down
174 changes: 174 additions & 0 deletions examples/bankRowList.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
//******************************************************************
//* ██╗ ██╗██╗██████╗ ██████╗ ██╗ ██╗ ██████╗
//* ██║ ██║██║██╔══██╗██╔═══██╗ ██║ ██║ ██╔═████╗
//* ███████║██║██████╔╝██║ ██║ ███████║ ██║██╔██║
//* ██╔══██║██║██╔═══╝ ██║ ██║ ╚════██║ ████╔╝██║
//* ██║ ██║██║██║ ╚██████╔╝ ██║██╗╚██████╔╝
//* ╚═╝ ╚═╝╚═╝╚═╝ ╚═════╝ ╚═╝╚═╝ ╚═════╝
//************************ Jefferson National Lab (2017) ***********
//******************************************************************
//* Example program for reading HIPO-4 Files..
//*--
//* Author: G.Gavalian
//*

#include <cstdlib>
#include <iostream>
#include <cassert>
#include "reader.h"
#include "twig.h"
#include "reaction.h"

// example using hipo::bank::rowlist to loop through a bank's rows
void example1(const char *file){
printf("===== EXAMPLE 1 =====\n");
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Event"});
int counter = 0;
while( r.next(list)&&counter<350){
counter++;
int nrows = list[0].getRows();
printf("\n>>> NEW EVENT <<<\n");
printf(" rows = %d\n",nrows);

// first, loop over all the rows
printf(" loop over all rows:\n");
for(auto const& row : list[0].getRowList())
printf("\t pid [%d] = %d\n", row, list[0].getInt(0,row));

// then, set the rowlist to contain only a few rows, and loop over those
if(nrows>6){

// set the list of rows, then loop over those
printf(" there are more than 6 rows, let's loop over some specific rows: 0, 1, and 4\n");
list[0].getMutableRowList().setList({0, 1, 4});
for(auto const& row : list[0].getRowList())
printf("\t pid [%d] = %d\n", row, list[0].getInt(0,row));
assert((list[0].getRowList().size() == 3)); // test

// it's still possible to access the full set of rows
printf(" we can still loop over all the rows with `getFullRowList()`\n");
for(auto const& row : list[0].getFullRowList())
printf("\t pid [%d] = %d\n", row, list[0].getInt(0,row));
printf(" or similarly with `getRows()`\n");
for(int row=0; row<list[0].getRows(); row++)
printf("\t pid [%d] = %d\n", row, list[0].getInt(0,row));
assert((static_cast<int>(list[0].getFullRowList().size()) == list[0].getRows())); // test

// you may also reset the list to its original, full state
printf(" resetting the rowlist restores the full row list\n");
list[0].getMutableRowList().reset();
for(auto const& row : list[0].getRowList())
printf("\t pid [%d] = %d\n", row, list[0].getInt(0,row));
assert((static_cast<int>(list[0].getRowList().size()) == list[0].getRows())); // test
}

}
}

// example showing how to filter a bank's rowlist with an expression
void example2(const char *file){
printf("===== EXAMPLE 2 =====\n");
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Event"});
int counter = 0;
while( r.next(list)&&counter<350){
counter++;
printf("\n>>> NEW EVENT <<<\n");
printf("=== BEFORE ==================================\n");
list[0].show();
list[0].getMutableRowList().filter("charge!=0");
printf("=== AFTER ===================================\n");
list[0].show();
// list[0].show(true); // call `show(true)` if you still need to see the full (not filtered) bank
}
}

// example showing how to filter a bank's rowlist with an first-order (lambda) function
void example3(const char *file){
printf("===== EXAMPLE 3 =====\n");

hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Event"});
std::function charged = [](hipo::bank &b, int row) { return b.getInt("charge",row)!=0 ? 1.0 : 0.0;};

int counter = 0;
while( r.next(list)&&counter<350){
counter++;
printf("\n>>> NEW EVENT <<<\n");
printf("=== BEFORE ==================================\n");
list[0].show();
list[0].getMutableRowList().filter(charged);
printf("=== AFTER ===================================\n");
list[0].show();
// list[0].show(true); // call `show(true)` if you still need to see the full (not filtered) bank
}
}

// Example showing how rowlists can be used to get indicies of the reference bank.
// The link function will return a hipo::bank::rowlist for the bank "REC::Calorimeter"
// where the column number 1 (which is "pindex") has a value of 0. Then by itarating over the returned
// indicies the total energy can be calculated.
void example4(const char *file){
printf("===== EXAMPLE 4 =====\n");
hipo::reader r(file);
hipo::banklist list = r.getBanks({"REC::Particle","REC::Calorimeter"});
int const pindex_column = 1; // of REC::Calorimeter
int const electron_row = 0;
int counter = 0;

while( r.next(list)&&counter<350){
counter++;
int status = list[0].getInt("status",electron_row);
if(list[0].getInt("pid",electron_row)==11&&abs(status)>=2000&&abs(status)<3000){
printf("\n>>> NEW EVENT <<<\n");
printf("found electron in row %d\n", electron_row);
double energy = 0.0;
for(auto const& it : list[1].getRowListLinked(electron_row, pindex_column)){
auto e = list[1].getFloat("energy",it);
printf(" => links REC::Calorimeter row=%d, which has energy %f\n", it, e);
energy += e;
}
printf("=========================\ntotal energy = %f\n=========================\n",energy);
list[0].show();
list[1].show();
}
}
}

int main(int argc, char** argv) {

std::cout << " reading CLAS12 hipo file and plotting missing mass (ep->epi+pi-X) " << __cplusplus << std::endl;
//axis x(120,0.,1.0);
//printf("bin = %d\n",x.find(0.4));

char inputFile[256];
int example_num = -1;

if(argc>1) {
snprintf(inputFile,256,"%s",argv[1]);
//sprintf(outputFile,"%s",argv[2]);
} else {
std::cerr << " *** please provide a file name..." << std::endl;
exit(1);
}
if(argc>2)
example_num = std::stoi(argv[2]);

switch(example_num) {
case -1:
example1(inputFile);
example2(inputFile);
example3(inputFile);
example4(inputFile);
break;
case 1: example1(inputFile); break;
case 2: example2(inputFile); break;
case 3: example3(inputFile); break;
case 4: example4(inputFile); break;
default:
std::cerr << " *** ERROR: unknown example..." << std::endl;
exit(1);
}

}
Loading

0 comments on commit e659099

Please sign in to comment.