Lookup fields provision workaround

[vlad-ivanov-d]

Hello

As you know there is some kind of issue with provisioning lookup fields: list, which is using for lookup field, must exists before field creation. In that case you have to create different models / make some hacks for provisioning. So you can't create one big model with lookup fields, subwebs, lists etc.

A long time ago we create a solution for such situation and I would like to share it with you, may be it will be heplful or integrated in SPMeta2. We usually creates our solution based on CSOM, so it works only for CSOM, but can be easily adapted for SSOM.

Main idea: We have just one function "DeployModel", which takes 2 arguments - ClientContext (Microsoft,SharePoint.Client) and SPMeta2 model (now it supports only Site and Web models). On a background this function:

1. Receives your model (we names it "Main model"), parse it and creates a new "Preparing model", which includes only your empty webdefinitions and listdefinitions.
2. Starts provisioning "Preparing model"
3. After preparing model provision completed it starts provisioning your full "Main model".
   So, at the moment when your own main model will start provisioning, all lists are already exists and it's become impossible to see an issue with lookup field which can't find list.
   Also our extensions supports incremental provision.

You can test it in this way:

1. Add our extensions to your solution
2. Add using
3. Use ClientContext.Deploy([your model])

Extensions.zip

[andreasblueher]

Hey,

I like the idea, because I would like to not write custom code and instead use one big model instead. However, to be able to deploy listdefinitions you might need fields to be deployed as site columns. You could try to split listdefinitions and added content types and fields, so "Preparing model" would only deploy basic list definitions, but this already seems to be potentially dangerous.

While the idea itself sounds good, the tearing things apart part sounds not reliable. What do you think?

[vlad-ivanov-d]

Hey @andreasblueher

I like the idea, because I would like to not write custom code and instead use one big model instead.
Yeah, it was the main reason why we developed such method.

And why do you think it is potentially dangerous? First of all, only your web and list definitions are provisioned (no any new artefacts, just your own), and then full model provision starts.

One disadvantage: web definitions and list definitions will be provisioned twice (in preparing model and in your full model). But I have and idea how to deal with this disadvantage and make a provision faster. I don't want to tell about this idea, because it needs some tests. Right now provision becomes a little bit slower, because of preparing model, but full model and final code of provisioning looks more structured.

This case is used in all our projects. Some projects rather big (more than 3000 SPMeta nodes). We didn't face with any issues.

I have plans to create a nuget package for this feature. So it would be easy for other people to try and test it.

[andreasblueher]

If you list definition contains AddField, AddListView, AddContentTypeLink and or any subelements of list definition, you need to separate those from the base list definition and that's what I find dangerous.

[vlad-ivanov-d]

Yeah, we exclude fields, content types, list views from a list definition. It's simple because of SPMeta2 architecture. By the way SPMeta do the same during provision: first of all SPMeta deploy the base list definition and then all child nodes.

So there are no any hacks.

[SubPointSupport]

That's a nice feature we would like to support and implement. Lookup provision is a bit of a challenge and needs to be simplified.

As you guys suggested, with some assumptions and conventions, SPMeta2 might manipulate provision order to make things right. Could be a new provision option, a set of APIs and extensible callbacks/services. Looking forward to see can be done here.

[vlad-ivanov-d]

Hey guys
I've made a nuget package as I promised you. You could test it and may be it will be easier to understand how it works. Please, check the link: https://github.com/vladivanovrf/spmeta-easy-deploy