Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature list depth option #34

Open
wants to merge 63 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
862778c
test(toKey): add check step value
mcakircali Jul 24, 2024
d60aeb8
feat(list): cherry-pick level
mcakircali Jul 29, 2024
d23778c
test(fdb-list): add new
mcakircali Jul 30, 2024
3be1f22
test(fdb-list): prepare grib file
mcakircali Jul 30, 2024
e0a0421
fix(fdb-list): init class member
mcakircali Jul 30, 2024
c2703ab
fix(fdb-list): unnecessary copy in loop
mcakircali Jul 30, 2024
36e6a54
fix(fdb-list): use make_unique not "new"
mcakircali Jul 30, 2024
77e3d77
fix(fdb-list): use "override" (not virtual)
mcakircali Jul 30, 2024
1e3381b
fix(fdb-list): default member init and tab size
mcakircali Jul 30, 2024
7f6be88
test(fdb-list): added --depth option
mcakircali Jul 30, 2024
5dfb6a0
fix(ListVisitor): empty keys
mcakircali Jul 30, 2024
71bbf51
style(fdb-list): clean code
mcakircali Jul 30, 2024
c4ebc5b
fix(fdb-list): porcelain defaults other options
mcakircali Jul 30, 2024
8ce657e
chore(remote/Handler): todo list w/ level
mcakircali Jul 30, 2024
8d9db4e
feat(list)!: separate ListElement
mcakircali Aug 5, 2024
fb4b8a0
fix(ListVisitor): use discrete ListElement ctor for DB, Cat, and Datum
mcakircali Aug 5, 2024
2c2b79d
fix(FDB): missing include
mcakircali Aug 5, 2024
31edc16
fix(FDB): name collisions
mcakircali Aug 5, 2024
376d842
fix(FDB): unnecessary copy in loop
mcakircali Aug 5, 2024
4a81dad
fix(FDB): after new ListElement
mcakircali Aug 5, 2024
e5edc6c
fix(FDB): 'some' include issues
mcakircali Aug 5, 2024
81a3924
chore(c api): param should be sha1
mcakircali Aug 5, 2024
6d67ae6
fix(c api): splitkey iterator
mcakircali Aug 5, 2024
b68231b
fix(fdb-patch): after ListElement
mcakircali Aug 5, 2024
a0cb5fa
fix(fdb-list): includes
mcakircali Aug 5, 2024
dbf974e
feat(fdb-list): options; assert depth and porcelain text
mcakircali Aug 5, 2024
4595fa7
fix(fdb-list): keys api
mcakircali Aug 5, 2024
7c42889
fix(FieldLocation): remove nullLocation
mcakircali Aug 5, 2024
f41ca4d
fix(MultiRetrieveVisitor): include issues
mcakircali Aug 5, 2024
70768b9
feat(remote): added level to Handler
mcakircali Aug 5, 2024
d4ebfa9
fix(RemoteFDB): fix include issues
mcakircali Aug 5, 2024
7ff9f39
fix(RemoteFDB): fix ListElement
mcakircali Aug 5, 2024
dfc1204
fix(LocalFDB): cleanup include
mcakircali Aug 5, 2024
7cc6f33
test(service): fix after ListElement with levels
mcakircali Aug 5, 2024
e085f57
test(fdb-list): fix after new output format
mcakircali Aug 5, 2024
bc0d333
fix(fdb-list): update porcelain text
mcakircali Aug 6, 2024
6d3fc15
fix(c api): added depth to fdb_list
mcakircali Aug 6, 2024
713c8f3
fix(ListElement): initial size = 0
mcakircali Aug 6, 2024
f1cc638
fix(c api): guard offset/len < 0
mcakircali Aug 6, 2024
24b70de
fix(fdb-list): updated doc for depth option
mcakircali Aug 7, 2024
39b27cd
test(fdb-list): rename cf -> an
mcakircali Aug 9, 2024
d8c3ec2
test(fdb-list): moved it to its own directory
mcakircali Aug 9, 2024
38c24d3
fix(c api): revert fdb_splitkey_t API
mcakircali Aug 13, 2024
12b398f
fix(c api): make fdb_splitkey_t work
mcakircali Aug 13, 2024
8cd9711
fix(c api): removed junk find()
mcakircali Aug 13, 2024
376d511
style(c api): typo
mcakircali Aug 13, 2024
de79888
feat(ListElement): throw with a msg
mcakircali Aug 13, 2024
59fdb36
feat(ListElement): added KeyChain (aka SplitKey)
mcakircali Aug 13, 2024
8e209ea
style: preserve api
mcakircali Aug 14, 2024
bea681d
style: replace auto
mcakircali Aug 14, 2024
7e73116
fix(fdb-list): param defaults and throw for porcelain option
mcakircali Aug 14, 2024
05890a3
Merge branch 'develop' into feature/list-depth-option
mcakircali Aug 14, 2024
346a47f
fix(c api): fdb_list assert bug
mcakircali Aug 14, 2024
7f72cc3
fix(ListElement): offset/length returns 0 for non-datum elements
mcakircali Aug 14, 2024
14d0f74
test(c api): added fdb_list with depth=1,2,3
mcakircali Aug 14, 2024
cbef3e9
fix(ListElement): bug print length
mcakircali Aug 14, 2024
75bcf1c
fix(ListElement): api issues around FieldLocation shared_ptr
mcakircali Aug 15, 2024
d638c98
docs(fdb-list): corrected depth range
mcakircali Aug 15, 2024
25e03d1
test(fdb-list): rename cf -> an
mcakircali Aug 9, 2024
d82131f
test(fdb-list): add marsflow tests
mcakircali Sep 12, 2024
13eeff9
test(fdb-list): fix macos grep issue
mcakircali Sep 13, 2024
bf03921
test(fdb-list): cleanup after success
mcakircali Sep 13, 2024
5aca515
Merge branch 'develop' into feature/list-depth-option
mcakircali Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
275 changes: 211 additions & 64 deletions .clang-format
Original file line number Diff line number Diff line change
@@ -1,108 +1,255 @@
---
Language: Cpp
Language: Cpp
BasedOnStyle: llvm
AccessModifierOffset: -4
AlignAfterOpenBracket: Align
AlignConsecutiveAssignments: true
AlignConsecutiveDeclarations: false
AlignArrayOfStructures: None
AlignConsecutiveAssignments:
Enabled: true
AcrossEmptyLines: false
AcrossComments: false
AlignCompound: true
AlignFunctionPointers: false
PadOperators: true
AlignConsecutiveBitFields:
Enabled: false
AcrossEmptyLines: false
AcrossComments: false
AlignCompound: false
AlignFunctionPointers: false
PadOperators: false
AlignConsecutiveDeclarations:
Enabled: true
AcrossEmptyLines: false
AcrossComments: true
AlignCompound: true
AlignFunctionPointers: false
PadOperators: false
AlignConsecutiveMacros:
Enabled: true
AcrossEmptyLines: false
AcrossComments: false
AlignCompound: true
AlignFunctionPointers: false
PadOperators: false
AlignConsecutiveShortCaseStatements:
Enabled: true
AcrossEmptyLines: true
AcrossComments: true
AlignCaseColons: false
AlignEscapedNewlines: Left
AlignOperands: true
AlignTrailingComments: true
AlignOperands: AlignAfterOperator
AlignTrailingComments:
Kind: Always
OverEmptyLines: 1
AllowAllArgumentsOnNextLine: false
AllowAllParametersOfDeclarationOnNextLine: true
AllowShortBlocksOnASingleLine: true
AllowShortCaseLabelsOnASingleLine: false
AllowBreakBeforeNoexceptSpecifier: Never
AllowShortBlocksOnASingleLine: Always
AllowShortCaseLabelsOnASingleLine: true
AllowShortCompoundRequirementOnASingleLine: true
AllowShortEnumsOnASingleLine: true
AllowShortFunctionsOnASingleLine: Inline
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AllowShortIfStatementsOnASingleLine: AllIfsAndElse
AllowShortLambdasOnASingleLine: All
AllowShortLoopsOnASingleLine: true
AlwaysBreakAfterDefinitionReturnType: None
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: true
AlwaysBreakTemplateDeclarations: true
AlwaysBreakBeforeMultilineStrings: false
AlwaysBreakTemplateDeclarations: Yes
AttributeMacros:
- __capability
- __forceinline
- __host__
- __noinline
- __nonnull
- __nothrow
- __nullable
- __output
- __unused
BinPackArguments: true
BinPackParameters: true
BitFieldColonSpacing: Both
BraceWrapping:
AfterClass: false
AfterControlStatement: false
AfterEnum: true
AfterFunction: false
AfterNamespace: false
AfterCaseLabel: false
AfterClass: false
AfterControlStatement: Never
AfterEnum: false
AfterExternBlock: false
AfterFunction: false
AfterNamespace: false
AfterObjCDeclaration: false
AfterStruct: false
AfterUnion: false
BeforeCatch: true
BeforeElse: true
IndentBraces: false
SplitEmptyFunction: true
SplitEmptyRecord: true
AfterStruct: false
AfterUnion: false
BeforeCatch: false
BeforeElse: false
BeforeLambdaBody: false
BeforeWhile: false
IndentBraces: false
SplitEmptyFunction: false
SplitEmptyRecord: false
SplitEmptyNamespace: true
BreakAdjacentStringLiterals: false
BreakAfterAttributes: Leave
BreakAfterJavaFieldAnnotations: false
BreakArrays: true
BreakBeforeBinaryOperators: None
BreakBeforeConceptDeclarations: Always
BreakBeforeBraces: Custom
BreakBeforeInheritanceComma: false
BreakBeforeInlineASMColon: OnlyMultiline
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
BreakConstructorInitializers: AfterColon
BreakAfterJavaFieldAnnotations: false
BreakConstructorInitializers: BeforeColon
BreakInheritanceList: BeforeColon
BreakStringLiterals: true
ColumnLimit: 120
CommentPragmas: '^ IWYU pragma:'
CompactNamespaces: false
ConstructorInitializerAllOnOneLineOrOnePerLine: true
ColumnLimit: 120
CommentPragmas: '^ IWYU pragma:'
CompactNamespaces: true
ConstructorInitializerIndentWidth: 4
ContinuationIndentWidth: 4
Cpp11BracedListStyle: true
DerivePointerAlignment: false
DisableFormat: false
DisableFormat: false
EmptyLineAfterAccessModifier: Never
EmptyLineBeforeAccessModifier: LogicalBlock
ExperimentalAutoDetectBinPacking: false
FixNamespaceComments: true
ForEachMacros:
- foreach
- Q_FOREACH
- BOOST_FOREACH
IfMacros:
- KJ_IF_MAYBE
IncludeBlocks: Preserve
IncludeCategories:
- Regex: '^<.*\.h>'
Priority: 1
- Regex: '^<.*'
Priority: 2
- Regex: '.*'
Priority: 3
IncludeIsMainRegex: '([-_](test|unittest))?$'
- Regex: '^<.*\.h>'
Priority: 2
- Regex: '^<.*\.hpp>'
Priority: 3
- Regex: '^"(llvm|llvm-c|clang|clang-c)/'
Priority: 4
SortPriority: 4
CaseSensitive: true
- Regex: '^((<|")(gtest|gmock|isl|json)/)'
Priority: 5
- Regex: '<[[:alnum:]._]+>'
Priority: 6
- Regex: '.*'
Priority: 1
SortPriority: 0
IncludeIsMainRegex: '(Test)?$'
IncludeIsMainSourceRegex: ''
IndentAccessModifiers: false
IndentCaseBlocks: false
IndentCaseLabels: true
IndentWidth: 4
IndentExternBlock: AfterExternBlock
IndentGotoLabels: true
IndentPPDirectives: None
IndentRequiresClause: true
IndentWidth: 4
IndentWrappedFunctionNames: false
InsertBraces: true
InsertNewlineAtEOF: true
InsertTrailingCommas: None
IntegerLiteralSeparator:
Binary: 0
BinaryMinDigits: 0
Decimal: 0
DecimalMinDigits: 0
Hex: 0
HexMinDigits: 0
JavaScriptQuotes: Leave
JavaScriptWrapImports: true
KeepEmptyLinesAtTheStartOfBlocks: true
KeepEmptyLinesAtEOF: false
LambdaBodyIndentation: Signature
LineEnding: DeriveLF
MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 2
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 1
NamespaceIndentation: None
ObjCBlockIndentWidth: 2
ObjCBinPackProtocolList: Never
ObjCBlockIndentWidth: 4
ObjCBreakBeforeNestedBlockParam: true
ObjCSpaceAfterProperty: false
ObjCSpaceBeforeProtocolList: false
PenaltyBreakAssignment: 2
PenaltyBreakBeforeFirstCallParameter: 1
ObjCSpaceBeforeProtocolList: true
PackConstructorInitializers: NextLine
PenaltyBreakAssignment: 20
PenaltyBreakBeforeFirstCallParameter: 19
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyExcessCharacter: 1000000
PenaltyReturnTypeOnItsOwnLine: 200
PenaltyBreakOpenParenthesis: 0
PenaltyBreakScopeResolution: 500
PenaltyBreakString: 100000
PenaltyBreakTemplateDeclaration: 10
PenaltyExcessCharacter: 10
PenaltyIndentedWhitespace: 0
PenaltyReturnTypeOnItsOwnLine: 1000000
PointerAlignment: Left
ReflowComments: true
SortIncludes: true
SortUsingDeclarations: true
PPIndentWidth: -1
QualifierAlignment: Leave
ReferenceAlignment: Pointer
ReflowComments: true
RemoveBracesLLVM: false
RemoveParentheses: Leave
RemoveSemicolon: false
RequiresClausePosition: OwnLine
RequiresExpressionIndentation: OuterScope
SeparateDefinitionBlocks: Always
ShortNamespaceLines: 3
SkipMacroDefinitionBody: false
SortIncludes: CaseSensitive
SortJavaStaticImport: Before
SortUsingDeclarations: Lexicographic
SpaceAfterCStyleCast: false
SpaceAfterTemplateKeyword: true
SpaceAfterLogicalNot: false
SpaceAfterTemplateKeyword: false
SpaceAroundPointerQualifiers: Default
SpaceBeforeAssignmentOperators: true
SpaceBeforeCaseColon: false
SpaceBeforeCpp11BracedList: true
SpaceBeforeCtorInitializerColon: true
SpaceBeforeInheritanceColon: true
SpaceBeforeJsonColon: false
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpaceBeforeParensOptions:
AfterControlStatements: true
AfterForeachMacros: true
AfterFunctionDefinitionName: false
AfterFunctionDeclarationName: false
AfterIfMacros: true
AfterOverloadedOperator: false
AfterPlacementOperator: true
AfterRequiresInClause: false
AfterRequiresInExpression: false
BeforeNonEmptyParentheses: false
SpaceBeforeRangeBasedForLoopColon: true
SpaceBeforeSquareBrackets: false
SpaceInEmptyBlock: true
SpacesBeforeTrailingComments: 2
SpacesInAngles: false
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: false
SpacesInParentheses: false
SpacesInAngles: Never
SpacesInContainerLiterals: false
SpacesInLineCommentPrefix:
Minimum: 1
Maximum: -1
SpacesInParens: Never
SpacesInParensOptions:
InCStyleCasts: false
InConditionalStatements: false
InEmptyParentheses: false
Other: false
SpacesInSquareBrackets: false
Standard: Auto
TabWidth: 4
UseTab: Never
BreakBeforeBinaryOperators: All
ColumnLimit: 0
...
Standard: c++17
StatementAttributeLikeMacros:
- Q_EMIT
StatementMacros:
- Q_UNUSED
- QT_REQUIRE_VERSION
TabWidth: 4
UseTab: Never
VerilogBreakBetweenInstancePorts: true
WhitespaceSensitiveMacros:
- BOOST_PP_STRINGIZE
- CF_SWIFT_NAME
- NS_SWIFT_NAME
- PP_STRINGIZE
- STRINGIZE
19 changes: 19 additions & 0 deletions debug-fdb.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

yell() { echo "$(basename "$0"): $*" >&2; }
die() { yell "$*"; exit 1; }
try() { "$@" || die "Errored HERE => '$*'"; }

export FDB_DIR="~/dev/bundle_stack/fdb"

export PATH="$FDB_DIR/bin:$PATH"
# export FDB_HOME="$FDB_DIR/build"
export FDB5_CONFIG_FILE="local.yaml"

REQUEST='class=rd,expver=wxyz,stream=enfo,date=20240902,time=0000,domain=g,type=pf,levtype=ml,step=9,number=9,levelist=9,param=1'

########################################################################################################################

try time fdb-inspect $REQUEST

try xctrace record --template 'Metins Instrument' --launch -- $FDB_DIR/bin/fdb-inspect $REQUEST
48 changes: 48 additions & 0 deletions fdb-write-test-db.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/usr/bin/env bash

yell() { echo "$(basename "$0"): $*" >&2; }
die() { yell "$*"; exit 1; }
try() { "$@" || die "Errored HERE => '$*'"; }

export FDB_DIR=~/dev/bundle_stack/fdb

# export PATH=~/workspace/bundle_stack/build/bin:$PATH
# export ECCODES_DEFINITION_PATH=~/workspace/bundle_stack/build/share/eccodes/definitions
# export FDB_HOME="$FDB_DIR/build"
export FDB5_CONFIG_FILE="local.yaml"

DB_NAME=large_fdb_test
BASE_GRIB=test.grib

########################################################################################################################

# rm -rf "./$DB_NAME" || true
# mkdir -p "./$DB_NAME/localroot"

try cd ./$DB_NAME

try cp "$FDB_DIR/$BASE_GRIB" .
try cp "$FDB_DIR/build/etc/fdb/schema" .
try cp "$FDB_DIR/tests/fdb/tools/list/local.yaml" .

########################################################################################################################

echo "Populating testing FDB using $BASE_GRIB"

for i in $(seq -f "%02g" 1 2); do
# echo "number $i"
try grib_set -s date=202409"$i" $BASE_GRIB tmp.grib

# fdb-hammer --class=od --expver=0001 --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
# fdb-hammer --class=od --expver=xxxx --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
# fdb-hammer --class=od --expver=wxyz --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib

fdb-hammer --class=rd --expver=0001 --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
fdb-hammer --class=rd --expver=xxxx --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
fdb-hammer --class=rd --expver=wxyz --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib

# fdb-hammer --class=ea --expver=0001 --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
# fdb-hammer --class=ea --expver=xxxx --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib
# fdb-hammer --class=ea --expver=wxyz --nlevels=5 --nparams=5 --nensembles=10 --nsteps=10 tmp.grib

done
7 changes: 7 additions & 0 deletions fdb_test_env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash

export PATH="$HOME/dev/bundle_stack/eccodes/build/bin/:$HOME/dev/bundle_stack/fdb/build/bin:$PATH"

export FDB_HOME="$HOME/dev/bundle_stack/fdb/build"

export FDB5_CONFIG_FILE="local.yaml"
Loading
Loading