name and the nwb.file schema
#552
Comments
The language is designed for hierarchical data organizations. All data objects in a data hierarchy must have a If you contrast this with a data model that is more oriented around the concept of collections of objects (e.g,. in a object store), there you then have some form of and
This is not fully correct. The name of the
The storage model is hierarchical. The
With So for tells |
|
OK! THANK YOU! This clears up what has been the most mysterious part of reading the spec:
I wasn't sure what that meant, because it seemed like So let me see if i've got this correct - every instantiated object that goes in an NWB file needs a I think this would be great to have in the docs, because it took me quite a bit of puzzling to make sense of the class and inheritance system. Maybe amending this section to further describe that the presence or absence of a name is meaningful about the disposition of the object - that "any group (or dataset) with a variable name must have a unique data type." means that a group or dataset with a variable name also means that the object must be given a name on instantiation? That also could clarify the So, eg. (ignoring incompleteness and other schema violations, just for the sake of minimal illustration to test my understanding)
In fact, looking at the schema now, it seems like and thank you for the info on It looks like the link for the tutorial is the same as that for the link to the class definition, but i would love to see the tutorial. Thanks for the answer!! Need to go revise my translation code to preserve nesting of unnamed classes and add a required |
No. Think of
The name of instances at particular location must be unique, i.e., you cannot have multiple instances of a dataset with the name
Yes, because when you have a fixed instance
That is correct for the autogenerated
They differ in the type of objects they add. The location in the hierarchy is not necessarily different. Consider you have a button in your file browser for |
|
Alright thanks again, I think i've got it now. name:
So in LinkML I would model unnamed classes with a required Timeseries:
attributes:
name:
required: true
range: string
data:
range: Timeseries__Dataand I would model a named class like: Timeseries__Data:
attributes:
name:
required: true
ifabsent: "data"
equals_string: "data"
range: stringquantity:
which also resolves my question about using groups:
- neurodata_type_inc: NWBDataInterface
doc: Acquired, raw data.
quantity: '*'
- neurodata_type_inc: DynamicTable
doc: Tabular data that is relevant to acquisition
quantity: '*'as a way of declaring which types are allowed in a group without being a special case. Think i am squared away on that and will update my model!
OK this is a little confusing to me, doesn't so nwbfile.add_acquisition(TimeSeries(name='myname', ...))
nwbfile.add_stimulus(TimeSeries(name='myname', ...))would make thanks again for the time spent clarifying! would be happy to help draft additions to the docs to clarify these things, the role of |
|
There, I think that's pretty nice :)
(with some fields omitted in the YAML to emphasize the nice parts) NWBFile:
name: NWBFile
description: An NWB file storing cellular-based neurophysiology data from a single
experimental session.
is_a: NWBContainer
attributes:
name:
name: name
ifabsent: string(root)
range: string
required: true
equals_string: root
file_create_date:
name: file_create_date
description: '...'
multivalued: true
range: isodatetime
required: true
acquisition:
name: acquisition
multivalued: true
any_of:
- range: NWBDataInterface
- range: DynamicTable
intervals:
name: intervals
description: Experimental intervals, whether that be logically distinct sub-experiments
having a particular scientific goal, trials (see trials subgroup) during
an experiment, or epochs (see epochs subgroup) deriving from analysis of
data.
multivalued: false
range: NWBFile__intervals
required: false
tree_root: true
NWBFile__intervals:
name: NWBFile__intervals
description: '...'
attributes:
name:
name: name
ifabsent: string(intervals)
range: string
required: true
equals_string: intervals
epochs:
name: epochs
description: Divisions in time marking experimental stages or sub-divisions
of a single recording session.
multivalued: false
range: TimeIntervals
required: false
time_intervals:
name: time_intervals
description: Optional additional table(s) for describing other experimental
time intervals.
multivalued: true
range: TimeIntervals
required: falsegenerates pydantic model: class NWBFile(NWBContainer):
"""
An NWB file storing cellular-based neurophysiology data from a single experimental session.
"""
nwb_version: Optional[str] = Field(None, description="""File version string. Use semantic versioning, e.g. 1.2.1. This will be the name of the format with trailing major, minor and patch numbers.""")
name: str = Field("root", const=True)
file_create_date: List[datetime ] = Field(default_factory=list, description="""...""")
identifier: str = Field(..., description="""...""")
session_description: str = Field(..., description="""...""")
session_start_time: datetime = Field(..., description="""...""")
timestamps_reference_time: datetime = Field(..., description="""...""")
acquisition: Optional[List[Union[DynamicTable, NWBDataInterface]]] = Field(default_factory=list)
analysis: Optional[List[Union[DynamicTable, NWBContainer]]] = Field(default_factory=list)
scratch: Optional[List[Union[DynamicTable, NWBContainer]]] = Field(default_factory=list)
processing: Optional[List[ProcessingModule]] = Field(default_factory=list)
stimulus: NWBFileStimulus = Field(..., description="""...""")
general: NWBFileGeneral = Field(..., description="""...""")
intervals: Optional[NWBFileIntervals] = Field(None, description="""...""")
units: Optional[Units] = Field(None, description="""...""")
class NWBFileIntervals(ConfiguredBaseModel):
"""
Experimental intervals, whether that be logically distinct sub-experiments having a particular scientific goal, trials (see trials subgroup) during an experiment, or epochs (see epochs subgroup) deriving from analysis of data.
"""
name: str = Field("intervals", const=True)
epochs: Optional[TimeIntervals] = Field(None, description="""Divisions in time marking experimental stages or sub-divisions of a single recording session.""")
trials: Optional[TimeIntervals] = Field(None, description="""Repeated experimental events that have a logical grouping.""")
invalid_times: Optional[TimeIntervals] = Field(None, description="""Time intervals that should be removed from analysis.""")
time_intervals: Optional[List[TimeIntervals]] = Field(default_factory=list, description="""Optional additional table(s) for describing other experimental time intervals.""")
|
Trying to figure out how the
namefield works in containers:When a dataset/group, say
TimeSeriesfor concreteness, is stored in an NWBFile, it is stored using anameproperty in a top-level container group like/acquisition/raw_running_wheel_rotationTimeSeriesand its ancestor classes do not have anamefield, however (except for thenamefield in the NWB schema language which optionally sets the name of the class, but it is unset for classes likeTimeSeriesthat define a type withneurodata_type_def).nwb-storagedocs describe anamefield, but there seems to be an amiguity between the class-levelnameand the implicit instancename- https://nwb-storage.readthedocs.io/en/latest/storage_hdf5.htmlThe description of how a dataset is to be saved is relatively abstract (which is good) in the
nwb.fileschema, specifying just that the file contains groups likeacquisition, which seem to use sub-groups as a list of allowable types - a special case not described in the schema language spec.The
nameproperty seems to be implemented in thehdmf.container.AbstractContainerclass - https://github.com/hdmf-dev/hdmf/blob/e801d9ee76e73ebfc8bf926e64a5a1a65337aebe/src/hdmf/container.py#L285 - but the schema forContaineralso lacks anamefield - https://github.com/hdmf-dev/hdmf-common-schema/blob/9b2580a21647e06be54708fabf2d44cef73d32cb/common/base.yaml#L7The implementation of
add_acquisitioncalls_add_acquisition_internal, which seems to be specified in an abstract way in__clsconf__but I can't trace it through the logic in hdmf to see how that actually works.So my question is basically: Am I missing something in the schema, or is there some accounting of things that are "outside" the schema somewhere? trying to make as general of a schema mapping as possible.
Additionally, any help understanding how the
__clsconf__works and where_add_acquisition_internalmight be found would be lovely :)The text was updated successfully, but these errors were encountered: