Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Physical boundaries not filled correctly in RefineSchedule after regridding #186

Open
borjaMinano opened this issue Nov 9, 2021 · 4 comments

Comments

@borjaMinano
Copy link

Hi,
We want to do a synchronization between levels after regridding. We use the following instruction:
d_fill_new_level->createSchedule(level,old_level,level_number-1,hierarchy,this)->fillData(data_time, true);
So data values are taken from the old level and parent level in the hierarchy.
The values in physical boundary nodes are supposed to be taken from the parent level. What occurs is that those values are 0, which is incorrect, since parent level values are not 0. However, values for internal ghost zones have the correct parent level values.
I am using a custom spatial interpolator with stencil 3. If I reduce the stencil to 1 it works since there are not all values in the physical ghost zone which are 0. The nearest values to internal domain are correctly set, but not the farther ones.

This behaviour only happens after a regrid. During simulation with schedules create like the following one all required physical boundary values from a parent level are correctly filled:
d_bdry_fill_advance12->createSchedule(level,ln-1,new_hierarchy,this)

So I suspect there is a difference for the setup of the schedule that is generating the incorrect behaviour.

@nselliott
Copy link
Collaborator

Do you have an implementation of RefinePatchStrategy::setPhysicalBoundaryConditions() to fill the values in physical boundary ghosts? The values in physical boundary ghosts are not filled from a coarser level; they are filled through your implementation of this pure virtual method in your application.

@borjaMinano
Copy link
Author

I don't mean filling the finer level physical boundary points with the coarser level ones. I need the coarser level physical boundary points to be able to interpolate finer level points near physical boundary (because of my stencil). After that interpolation I would set the physical boundary conditions in the finer level.

What I don't understand is why I have access to that coarser level physical boundary points when doing interpolation during a step but I get zeros for that same points just after a regridding operation.
So I don't know why there is that difference between using one schedule (createSchedule(level,ln-1,new_hierarchy,this)) or another (createSchedule(level,old_level,level_number-1,hierarchy,this)).

@nselliott
Copy link
Collaborator

The same setPhysicalBoundaryConditions() call should fill the physical boundary ghosts on the coarse level before the interpolation. It sounds like this is happening correctly in your non-regrid schedules, but not in the regrid schedule. I don't know why this would happen if you are setting the stencil to the same value in both cases.

@borjaMinano
Copy link
Author

The weird thing is that I checked that physical boundaries in the coarse level are filled before both kinds of interpolations.
I have tried to see any difference in the variable values inside RefineSchedule.C to guess if it was a configuration problem or a kind of bug, but I couldn't find anything, it's pretty complicated if you don't know where to look.

I am really interested in solving this problem, so if you can guide me a little bit I can try to solve the problem.
I tried to make a "dirty" fix just for my case adding
if (d_src_level != d_dst_level) transformed_src_box.grow(dst_gcw);
in constructScheduleTransactions method just after the comment "Create overlap for normal cases (all but enhanced connectivity).". And that change do fill the stencill in the physical boundaries correctly, but had repercussions in other data transactions. So I think there are more things to modify to make it consistent, but I don't know where to look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants