Feature: Masked Dataset #151
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
samuelburbulla
requested changes
Jul 31, 2024
| from continuiti.transforms import Transform | ||
| from continuiti.operators.shape import OperatorShapes, TensorShape | ||
|
|
||
|
|
||
| class OperatorDatasetBase(td.Dataset, ABC): | ||
| """Abstract base class of a dataset for operator training.""" | ||
|
|
||
| shapes: OperatorShapes | ||
| def __init__(self, shapes: OperatorShapes, n_observations: int) -> None: |
There was a problem hiding this comment.
Suggested change
| def __init__(self, shapes: OperatorShapes, n_observations: int) -> None: | |
| def __init__(self, shapes: OperatorShapes, n_observations: int): |
| """Applies class transformations to four tensors. | ||
|
|
||
| Args: | ||
| src: |
There was a problem hiding this comment.
Suggested change
| src: | |
| src: List of tuples containing a tensor and a transformation to apply to it. |
Comment on lines
+40
to
+41
| continue | ||
| out.append(transformation(src_tensor)) |
There was a problem hiding this comment.
Suggested change
| continue | |
| out.append(transformation(src_tensor)) | |
| else: | |
| out.append(transformation(src_tensor)) |
| self, x: torch.Tensor, u: torch.Tensor, y: torch.Tensor, v: torch.Tensor | ||
| ) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]: | ||
| """Applies class transformations to four tensors. | ||
| return tensors[0], tensors[1], tensors[2], tensors[3] |
There was a problem hiding this comment.
Suggested change
| return tensors[0], tensors[1], tensors[2], tensors[3] | |
| return tuple(tensors) |
| """A dataset for operator training containing masks in addition to tensors describing the mapping. | ||
|
|
||
| Data, especially described on unstructured grids, can vary in the number of evaluations or sensors. Even | ||
| measurements of phenomena do not always contain the same number of sensors and or evaluations. This dataset is able |
There was a problem hiding this comment.
Suggested change
| measurements of phenomena do not always contain the same number of sensors and or evaluations. This dataset is able | |
| measurements of phenomena do not always contain the same number of sensors and/or evaluations. This dataset is able |
Comment on lines
+280
to
+282
| assert not any( | ||
| [torch.any(torch.isinf(mi)) for mi in member] | ||
| ), "Expects domain to be truncated in finite space." |
There was a problem hiding this comment.
Why is this assertion necessary? Someone might come up with a good reason for using infs in the data, do we have to prevent that?
There was a problem hiding this comment.
Ah, I see we're using inf for padding. However, does it hurt to have more infs (non-masked) in the dataset?
| padding_value=torch.inf, | ||
| ).transpose(1, 2) | ||
| values_padded = pad_sequence( | ||
| [vi.transpose(0, 1) for vi in values], batch_first=True, padding_value=0 |
There was a problem hiding this comment.
Suggested change
| [vi.transpose(0, 1) for vi in values], batch_first=True, padding_value=0 | |
| [vi.transpose(0, 1) for vi in values], | |
| batch_first=True, | |
| padding_value=0, |
There was a problem hiding this comment.
Why do we pad once with inf and once with 0? Seems arbitrary
Comment on lines
+293
to
+296
| mask = member_padded != torch.inf | ||
| member_padded[ | ||
| ~mask | ||
| ] = 0 # mask often applied by adding a tensor with -inf values in masked locations (e.g. in scaled dot product). |
There was a problem hiding this comment.
What is different here if why just used 0 for padding in l. 287?
|
|
||
| return sample["x"], sample["u"], sample["y"], sample["v"] | ||
| return tensors[0], tensors[1], tensors[2], tensors[3], ipt_mask, opt_mask |
There was a problem hiding this comment.
Suggested change
| return tensors[0], tensors[1], tensors[2], tensors[3], ipt_mask, opt_mask | |
| return *tuple(tensors), ipt_mask, opt_mask |
| dataloader = DataLoader(dataset, batch_size=self.batch_size) | ||
|
|
||
| for x, u, y, v, ipt_mask, opt_mask in dataloader: | ||
| assert True |
There was a problem hiding this comment.
Can we do more here?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Feature: Masked Dataset
Description
Not all datasets are consisent in the number of sensors and evaluations. Simulations or measurements are not only performed on a multitude of different grids, but may also contain different numbers of samples in both function spaces/sets. To reflect this the
MaskedOperatorDatasetclass is introduced. It is able to handle datasets with this property.This PR introduces two new operator classes
MaskedOperatorDataset. TheMaskedOperatorDatasetis able to process datasets with varying number of sensors or evaluations.Which issue does this PR tackle?
OperatorDatasetclass is able to only handle uniform evaluation- and sensor-numbers.How does it solve the problem?
MaskedOperatorDatasetclass to allow for masked sensors and evaluations.How are the changes tested?
Notes
_get_item_method is a method specific to theOperatorDatasetandMaskedOperatorDatasetclasses for separation.Checklist for Contributors
feature/title-slugconvention.Bugfix: Titleconvention.Checklist for Reviewers: