Commonly used SF and vectors in mef90

Rationale

Boundary conditions are handled using PETSc constraints. As such, constrained degrees of freedom are eliminated from Global but not from Local vectors. In order to perform IO operations, we must add back the constrained dof and renumber / redistribute the vectors so that the ordering matches that of the input file and the distribution allows load-balance parallel IO.

Views into a vector

• local: Petsc local vector with constraints (non-collective)
• global: Petsc global vector without constraints (collective)
• cglobal: Petsc global vector with constraints (collective)
• natural: cglobal reordered in the initial DM ordering (collective, all values on processor 0)
• IO: natural load-balanced (collective)

cglobal and natural vectors are internal only. IO is only needed in order to implement new file formats

SF

• XXXToIOSF,IOToXXXSF: used to copy from local to IO vectors and back using MEF90VecCopySF
• XXXConstraintsSF: used to copy constrained values between local vectors using MEF90VecCopySF

SF creation

• MEF90IOSFCreate: Create the SF to move data from a local vector onto an IO vector saved as nodal or zonal values over the whole domain.
• MEF90FaceSetIOSFCreate: Create the SF to move data from a local vector onto an IO vector saved as zonal values over all face sets.
• MEF90ConstraintSFCreate: Create the SF to move data at constrained degrees of freedom between two local vectors
• CreateNaturalToIOSF_Private: create a SF used to broadcast from a natural to an IO vector (generalization of VecScatterToZero using SF and respecting block size
• CreateLocalToCGlobalSF_Private: Create the local to cglobal SF
• CreateCGlobalToLocalSF_Private: Create the SF to broadcast from cglobal to local
• CreateSideSF_Private: equivalent of a natural ordering of sides in exodus side sets

Broadcast operations using SF

• MEF90VecCopySF: wraps calls to VecGetArray, SFBroadcast, and VecRestoreArray. Can be used to copy the constrained values from one local vector into another local vector or to copy values from a local vector into an IO vector.
• MEF90VecGlobalToLocalConstraint(g,c,l): given global vector g, and two local vectors c and l, copy all unconstrained values of g and all constrained values of c into l
• MEF90VecCreateIO(v,bs,sf): create an IO vector given the XXXToIOSF and block size bs (the parallel data layout of the IO vector is given by the leaves of the sf.

Other functions (may need to be moved somewhere else):

• MEF90VecSetValuesFromOptions: Fill values of a local vector using command line options
• MEF90VecSetBCValuesFromOptions: Fill constrained values of a local vector using command line options