Skip to content

Commit

Permalink
Make blocks_per_json_file flag not output extra files (#63)
Browse files Browse the repository at this point in the history
Currently, the logic in convert_bhive_to_llvm_exegesis_input will create
extra (empty) files as the output at the end runs unconditionally rather
than when there are actually blocks to output. This means that if we
have a number of blocks that is an exact multiple of the value in
blocks_per_json_file, we end up getting no blocks after the loop, which
means we output an empty JSON file.

Fixes #61.
  • Loading branch information
boomanaiden154 authored Mar 11, 2024
1 parent c1231ad commit 67c86b5
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 42 deletions.
2 changes: 1 addition & 1 deletion gematria/datasets/convert_bhive_to_llvm_exegesis_input.cc
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ int main(int argc, char* argv[]) {
file_counter++;
}

if (!json_output_dir.empty()) {
if (!json_output_dir.empty() && processed_snippets.size() != 0) {
size_t json_file_number = file_counter / blocks_per_json_file;
bool write_successfully = WriteJsonFile(std::move(processed_snippets),
json_file_number, json_output_dir);
Expand Down
86 changes: 45 additions & 41 deletions gematria/datasets/tests/blocks_per_json_file.test
Original file line number Diff line number Diff line change
@@ -1,52 +1,56 @@
; Test that splitting a dataset among multiple JSON files works as expected.

; Test currently fails as underlying functionality needs to be fixed, see #61.
; XFAIL: *

; RUN: split-file %s %t
; RUN: mkdir %t.jsondir
; RUN: %convert_bhive_to_llvm_exegesis_input --json_output_dir=%t.jsondir --bhive_csv=%t/test.csv
; RUN: %convert_bhive_to_llvm_exegesis_input --json_output_dir=%t.jsondir --bhive_csv=%t/test.csv --blocks_per_json_file=1
; RUN: cat %t.jsondir/0.json | FileCheck --check-prefix FILE1 %s
; RUN: cat %t.jsondir/1.json | FileCheck --check-prefix FILE2 %s

; FILE1-CHECK: [
; FILE1-CHECK: {
; FILE1-CHECK: "Hex": "85c044897c2460",
; FILE1-CHECK: "MemoryDefinitions": [
; FILE1-CHECK: {
; FILE1-CHECK: "Name": "MEM",
; FILE1-CHECK: "Size": 4096,
; FILE1-CHECK: "Value": 305419776
; FILE1-CHECK: }
; FILE1-CHECK: ],
; FILE1-CHECK: "MemoryMappings": [
; FILE1-CHECK: {
; FILE1-CHECK: "Address": 65536,
; FILE1-CHECK: "Value": "MEM"
; FILE1-CHECK: }
; FILE1-CHECK: ]
; FILE1-CHECK: }
; FILE1-CHECK: ]
; Ensure that we don't have any "leftover" files.
; RUN: ls %t.jsondir | FileCheck --check-prefix DIR %s

; FILE1: [
; FILE1: {
; FILE1: "Hex": "85c044897c2460",
; FILE1: "MemoryDefinitions": [
; FILE1: {
; FILE1: "Name": "MEM",
; FILE1: "Size": 4096,
; FILE1: "Value": 305419776
; FILE1: }
; FILE1: ],
; FILE1: "MemoryMappings": [
; FILE1: {
; FILE1: "Address": 65536,
; FILE1: "Value": "MEM"
; FILE1: }
; FILE1: ]
; FILE1: }
; FILE1: ]

; FILE2: [
; FILE2: {
; FILE2: "Hex": "3b31",
; FILE2: "MemoryDefinitions": [
; FILE2: {
; FILE2: "Name": "MEM",
; FILE2: "Size": 4096,
; FILE2: "Value": 305419776
; FILE2: }
; FILE2: ],
; FILE2: "MemoryMappings": [
; FILE2: {
; FILE2: "Address": 65536,
; FILE2: "Value": "MEM"
; FILE2: }
; FILE2: ]
; FILE2: }
; FILE2: ]

; FILE2-CHECK: [
; FILE2-CHECK: {
; FILE2-CHECK: "Hex": "3b31",
; FILE2-CHECK: "MemoryDefinitions": [
; FILE2-CHECK: {
; FILE2-CHECK: "Name": "MEM",
; FILE2-CHECK: "Size": 4096,
; FILE2-CHECK: "Value": 305419776
; FILE2-CHECK: }
; FILE2-CHECK: ],
; FILE2-CHECK: "MemoryMappings": [
; FILE2-CHECK: {
; FILE2-CHECK: "Address": 65536,
; FILE2-CHECK: "Value": "MEM"
; FILE2-CHECK: }
; FILE2-CHECK: ]
; FILE2-CHECK: }
; FILE2-CHECK: ]
; DIR: 0.json
; DIR: 1.json
; DIR-NOT: 2.json

;--- test.csv
3b31,45.000000
85c044897c2460,98.000000
3b31,45.000000

0 comments on commit 67c86b5

Please sign in to comment.