Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.0 - Initial work #5

Merged
merged 31 commits into from
Feb 26, 2024
Merged

3.0 - Initial work #5

merged 31 commits into from
Feb 26, 2024

Conversation

cpiker
Copy link
Member

@cpiker cpiker commented Feb 26, 2024

This merge provides the beginning of v3.0 stream handling, namely:

  1. All existing das v2 unit-tests pass, new lib is almost 100% backwards compatible
  2. Packet data are stored in much more versatile DasAry dynamic type and dynamic memory structures instead of coercing all values to doubles.
  3. (Related to 2) TT2000 values are handled internally as integers and thus support nanosecond resolution over their entire legal range.
  4. The non-namespace of the new stream format, "3.0/basic" is parsed into memory structures
  5. DasVar internal indicies have a defined purpose. They can correspond to:
    a. String lengths, this way strings are treated as singletons
    b. Record geometric vector lengths, so that geometric vectors are treated as singletons.
  6. The new stream object allows vector reference frames to be defined and referenced explicitly.
  7. A significant new utility, das3_cdf is provided which serializes DasDs objects to ISTP CDF format.

Significant items that still need work on the road to v3.0, release-candidate-1 include:

  1. DasBldr doesn't register callbacks to ingest v3.0 data, yet. (This is easy and required to support das2py, das2dlm etc.)
  2. Legacy formats are not streamed into DasAry structures, yet.
  3. The stream processors don't have v3.0 format versions, yet.
  4. The pure xml format, *.d3x is not supported by DasIO, yet.

cpiker added 30 commits January 15, 2024 17:32
DasVar object with an interal index now have specific purposes for
that index that is denoted by the value type.  There are three defined:

vtText - Internal rank is 1, denotes the byte offset in utf-8 string
vtGeoVec - Internal rank is 1, denotes a geometric vector in a frame
vtByteSeq - Internal rank is 1, denote byte sequences of unknown purpose
In addition
   (1) user data points were added to the dataset,dimension,variable hierarchy.
	(2) Accessing dimensions and variables by index or by generala type
	    was standardized.  This may break so old apps, but should be
		 a quick fix.
@cpiker cpiker self-assigned this Feb 26, 2024
@cpiker cpiker merged commit a56e17a into master Feb 26, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant