Meta modelling (#845)

* create meta R type and remove from RAttribute * handle adding meta attributes in type provider * fix all references to meta except generator code * fix expression generator * fix ModelObjectGenerator * fix ModelObjectBuilderGenerator * fix MetaFieldGenerator * build subtype checker * save * save * add switch to handle all R types * implement join logic * fix validation sub type check for location and address * refactor subtype code * refactor meta checks into utils * fix validation * fix implicit enum rtype provider return * add meta to equals and hash codes * Create an RAnnotatedType and revert RType * Make RAttribute return RAnnotatedType * fix type translator * fix RObjectFActory and RosettaTypeProvider * organise imports * revert xsemantics changes * make type provider hand back annotated type * fix object factory and ecore util * fix expected type and scoping * revert TypeFactory changes * fix TypeSystem * fix DeepFeatureCallUtil * fix ExternalAnnotationUtil * fix MetaUtil * fix StandaloneRosettaTypingValidator * add sub type and join to SubTypeRelation * fix SubtypeRelationTest * fix DeepPathUtilGenerator * add annotated sub type check * rename top level RMetaAnnotatedType * fix RosettaFunctionExtensions * fix JavaTypeTranslator * rename getter * fix simple validator * fix ValidatorGenerator * fix expression gen * fix FunctionGenerator * fix MetaFieldGenerator * fix ModelObjectBoilerPlate * fix ModelObjectBuilderGenerator * fix generators * add some more sub type test * remove whitespace diff * fix hasMeta access in JavaTypeTranslator * remove named injection * fix superset logic * organise imports * remove unnecessary logic * fix validator * add type utils for meta * refactor expression to return RMetaAnnotatedType from RosettaTypeProvider * rename method * fix validation * fix needs builder check * remove meta from xsemantics * fix scope provider * remove commented code * fix injection in JavaDependencyProvider * remove todo for settings meta on rule outputs * get rid of RAnnotateType * remove redundant hasMeta from RType * fix issue with scoping * write failing code gen test * Fix semantic tokens * disable failing gen tests until we do code gen * remove meta from subtype relationship * remove unused injections * validate meta attributes are set on RMetaAnnotatedType * move withEmptyMeta to a factory method on type * neaten subtype test * remove redundant tests * move factory methods * fix validator typos * add some built in types with meta * use meta built in type for checkParseOpArgument validation * remove sub type checks for mixed meta and rtype * back out metaType changes * fix typo * remove unused methods * remove unused methods * move meta util method to SubtypeRelation * return early in subtype check * return early on Rtype sub type check * use meta types on reduce operation validation * update JavaType translator and add new RJava types * add RJavaReferenceWithMeta * hand back correct types in type translator for metas * remove deprecated hasReferenceOrAddressMetadata * add test for meta coercion * bring back disabled tests * fix meta generator * fix constructor value setting for expressions * bring back metaType on test * add test for setting output value with meta in function * build metaToItem coercion * build item to meta conversion * implement meta to meta conversion * tidy up expression mappings * fix all coercion * fix test * fix binary operation expressions * fix attribute calls * neaten up test * fix expression reciever type * refactor meta generator logic * add meta comparison tests * formatting * fix expression generator symbol reference case for meta types * fix implicit type for Data types so they don't contain meta * add failing test * strip meta on constructor and path setting * fix setting values for as-key * create kek synonym scope for JavaStatementBuilder declared variables * fix to-string for meta enums * fix min and max operations on meta items * build util for enum instance creation * add documentation for metadata in functions and expressions * use scheme instead of reference for docs example * indent meta in docs * put a warning on getMetaDescriptions method * rename method * removed commented code * fix caseToTimeOperation * remove meta from choice operation scoping * fix sum on meta * fix meta sort operation * use test util for creating FieldWithMetaString * fix constraint meta operations * add validation to prevent using only exist on meta * preserve meta data on function operations * move RJavaWithMetaValue * ignore meta on switch inputs * preserve metadata on inputsAsParameters in fucntion generator * fix indent * refactor MetaFieldGenerator * fix contributeHashCode meta check * refactor TabulatorGenerator * add test for passing meta out of a function * fix whitespace * refactor method * include metadata on choice checkChoiceOptionsDoNotOverlap * fix checkChoiceSwitch * fix error on checkChoiceSwitch * validator checkType should look at meta * make getQualifyingFunctionInterface consider meta for parameterVariable * fix checkSymbolReference attribute check * update comment
finos · Oct 21, 2024 · fb297a7 · fb297a7
1 parent 4159a4e
commit fb297a7
Show file tree

Hide file tree

Showing 62 changed files with 2,457 additions and 866 deletions.
diff --git a/docs/rune-modelling-component.md b/docs/rune-modelling-component.md
@@ -425,6 +425,33 @@ Each attribute of the `metadata` annotation corresponds to a different qualifier
 - The `template` qualifier indicates that a data type is eligible to be used as a [data template](#data-template). Data templates provide a way to store data which may be duplicated across multiple objects into a single template, to be referenced by all these objects.
 - the other metadata annotations are used for [cross-referencing](#cross-referencing).
 
+#### Meta-Data Use In Functions And Expressions 
+
+It is possible to use `metadata` annotated function inputs and expressions outputs to access the value of a piece of metadata. The following is an example of using `metadata` that has been passed into a function:
+
+```Haskell
+func MyFunc:
+    inputs:
+        myInput string (1..1)
+            [metadata scheme]
+    output:
+        myResult string (1..1)
+
+    set myResult: myInput -> scheme
+```
+
+Additionally here is an example of how to work with metadata that is the output of an expression:
+
+```Haskell
+func MyFunc:
+    inputs:
+        myInput string (1..*)
+            [metadata scheme]
+    output:
+        myResult string (1..*)
+    set myResult: myInput extract scheme
+```
+
 ### Document Reference
 
 #### Purpose

diff --git a/rosetta-ide/src/main/java/com/regnosys/rosetta/ide/hover/RosettaDocumentationProvider.java b/rosetta-ide/src/main/java/com/regnosys/rosetta/ide/hover/RosettaDocumentationProvider.java
@@ -57,7 +57,7 @@ public List<String> getDocumentationFromOwner(EObject o) {
 		if (o instanceof RosettaSymbolReference) {
 			RosettaSymbol symbol = ((RosettaSymbolReference)o).getSymbol();
 			if (symbol instanceof RosettaEnumValue) {
-				RType t = expectedTypeProvider.getExpectedTypeFromContainer(o);
+				RType t = expectedTypeProvider.getExpectedTypeFromContainer(o).getRType();
 				docs.add(t.toString());
 			}
 		}

diff --git a/rosetta-ide/src/main/java/com/regnosys/rosetta/ide/inlayhints/RosettaInlayHintsService.java b/rosetta-ide/src/main/java/com/regnosys/rosetta/ide/inlayhints/RosettaInlayHintsService.java
@@ -58,7 +58,7 @@ public InlayHint checkFunctionalOperation(RosettaFunctionalOperation op) {
 		if (op.getFunction() != null && operationHasBrackets(op.getFunction())) {
 			if (op instanceof ReduceOperation || op instanceof MapOperation) {
 				if (extensions.isResolved(op.getFunction())) {
-					RType outputType = types.getRType(op);
+					RType outputType = types.getRMetaAnnotatedType(op).getRType();
 					boolean outputMulti = card.isMulti(op);
 
 					if (outputType != null) {

diff --git a/...e/src/main/java/com/regnosys/rosetta/ide/semantictokens/RosettaSemanticTokensService.java b/...e/src/main/java/com/regnosys/rosetta/ide/semantictokens/RosettaSemanticTokensService.java
@@ -54,7 +54,7 @@
 import com.regnosys.rosetta.rosetta.simple.Segment;
 import com.regnosys.rosetta.rosetta.simple.ShortcutDeclaration;
 import com.regnosys.rosetta.types.CardinalityProvider;
-import com.regnosys.rosetta.types.RType;
+import com.regnosys.rosetta.types.RMetaAnnotatedType;
 import com.regnosys.rosetta.types.RosettaTypeProvider;
 
 import static com.regnosys.rosetta.rosetta.RosettaPackage.Literals.*;
@@ -263,7 +263,7 @@ public SemanticToken markFeature(RosettaFeatureCall featureCall) {
 
 	private SemanticToken markSymbol(EObject objectToMark, EStructuralFeature featureToMark, RosettaSymbol symbol) {
 		if (symbol instanceof Attribute) {
-			RType implicitType = typeProvider.typeOfImplicitVariable(objectToMark);
+			RMetaAnnotatedType implicitType = typeProvider.typeOfImplicitVariable(objectToMark);
 			if (implicitType != null) {
 				Set<? extends RosettaFeature> implicitFeatures = Sets.newHashSet(extensions.allFeatures(implicitType, objectToMark));
 				if (implicitFeatures.contains(symbol)) {

diff --git a/rosetta-lang/model/Rosetta.xcore b/rosetta-lang/model/Rosetta.xcore
@@ -126,7 +126,7 @@ class RosettaTypeAlias extends RosettaRootElement, RosettaType, RosettaTyped, Ro
 }
 
 
-class RosettaMetaType extends RosettaRootElement, RosettaTypedFeature, RosettaType {
+class RosettaMetaType extends RosettaRootElement, RosettaTypedFeature, RosettaType, RosettaSymbol {
 
 }
 

diff --git a/rosetta-lang/src/main/java/com/regnosys/rosetta/RosettaEcoreUtil.xtend b/rosetta-lang/src/main/java/com/regnosys/rosetta/RosettaEcoreUtil.xtend
@@ -2,6 +2,7 @@ package com.regnosys.rosetta
 
 import com.google.common.base.CaseFormat
 import com.regnosys.rosetta.rosetta.RosettaEnumeration
+import com.regnosys.rosetta.rosetta.RosettaFactory
 import com.regnosys.rosetta.rosetta.RosettaFeature
 import com.regnosys.rosetta.rosetta.RosettaRecordType
 import com.regnosys.rosetta.rosetta.RosettaSynonym
@@ -12,41 +13,48 @@ import com.regnosys.rosetta.rosetta.simple.Attribute
 import com.regnosys.rosetta.rosetta.simple.Condition
 import com.regnosys.rosetta.rosetta.simple.Data
 import com.regnosys.rosetta.rosetta.simple.Function
+import com.regnosys.rosetta.rosetta.simple.SimpleFactory
+import com.regnosys.rosetta.scoping.RosettaScopeProvider
 import com.regnosys.rosetta.types.RAttribute
+import com.regnosys.rosetta.types.RChoiceType
 import com.regnosys.rosetta.types.RDataType
 import com.regnosys.rosetta.types.REnumType
+import com.regnosys.rosetta.types.RMetaAnnotatedType
+import com.regnosys.rosetta.types.RObjectFactory
 import com.regnosys.rosetta.types.RType
+import com.regnosys.rosetta.types.builtin.RBuiltinTypeService
+import com.regnosys.rosetta.types.builtin.RRecordType
+import com.regnosys.rosetta.utils.PositiveIntegerInterval
+import com.regnosys.rosetta.utils.RosettaConfigExtension
 import java.util.Collection
+import java.util.LinkedHashSet
+import java.util.List
 import java.util.Set
 import javax.inject.Inject
+import javax.inject.Singleton
 import org.eclipse.emf.ecore.EObject
 import org.eclipse.emf.ecore.resource.ResourceSet
-
-import com.regnosys.rosetta.types.builtin.RRecordType
-import com.regnosys.rosetta.types.builtin.RBuiltinTypeService
-import javax.inject.Singleton
-import java.util.List
-import com.regnosys.rosetta.utils.PositiveIntegerInterval
 import org.eclipse.xtext.util.SimpleCache
-import com.regnosys.rosetta.rosetta.RosettaFactory
-import com.regnosys.rosetta.scoping.RosettaScopeProvider
-import com.regnosys.rosetta.rosetta.simple.SimpleFactory
-import com.regnosys.rosetta.types.RObjectFactory
-import java.util.LinkedHashSet
-import com.regnosys.rosetta.types.TypeSystem
-import com.regnosys.rosetta.types.RChoiceType
+
+import static extension com.regnosys.rosetta.types.RMetaAnnotatedType.withMeta
+import org.eclipse.emf.ecore.util.EcoreUtil
 
 @Singleton // see `metaFieldsCache`
 class RosettaEcoreUtil {
 
 	@Inject RBuiltinTypeService builtins
-	@Inject RObjectFactory objectFactory
-	@Inject extension TypeSystem typeSystem
+	@Inject extension RObjectFactory objectFactory
+	@Inject extension RosettaConfigExtension configs
 
 	def boolean isResolved(EObject obj) {
 		obj !== null && !obj.eIsProxy
 	}
 
+	def Iterable<? extends RosettaFeature> allFeatures(RMetaAnnotatedType t, EObject context) {
+		val List<RosettaFeature>  metas = getMetaDescriptions(t, context)
+		allFeatures(t.RType, context?.eResource?.resourceSet) + metas
+	}
+
 	def Iterable<? extends RosettaFeature> allFeatures(RType t, EObject context) {
 		allFeatures(t, context?.eResource?.resourceSet)
 	}
@@ -134,10 +142,6 @@ class RosettaEcoreUtil {
 		metaAnnotations.exists[attribute?.name == "template"]
 	}
 
-	def boolean hasMetaDataAnnotations(RAttribute attribute) {
-		attribute.metaAnnotations.exists[name == "reference" || name == "location" || name == "scheme" || name == "id"]
-	}
-
 	def boolean hasMetaDataAnnotations(Annotated it) {
 		metaAnnotations.exists[attribute?.name == "reference" || attribute?.name == "location" || attribute?.name == "scheme" || attribute?.name == "id"]
 	}
@@ -146,8 +150,12 @@ class RosettaEcoreUtil {
 		metaAnnotations.exists[attribute?.name != "reference" && attribute?.name != "address"]
 	}
 
+	def boolean hasMetaDataReference(RAttribute attribute) {
+		attribute.RMetaAnnotatedType.getMetaAttributes.exists[name == "reference"]
+	}
+
 	def boolean hasMetaDataAddress(RAttribute attribute) {
-		attribute.metaAnnotations.exists[name == "address"]
+		attribute.RMetaAnnotatedType.getMetaAttributes.exists[name == "address"]
 	}
 
 	def boolean hasMetaDataAddress(Annotated it) {
@@ -158,7 +166,7 @@ class RosettaEcoreUtil {
 		metaAnnotations.exists[attribute?.name == "id"]
 	}
 	def boolean hasIdAnnotation(RAttribute it) {
-		metaAnnotations.exists[name == "id"]
+		RMetaAnnotatedType.getMetaAttributes.exists[name == "id"]
 	}
 	def boolean hasReferenceAnnotation(Annotated it) {
 		metaAnnotations.exists[attribute?.name == "reference"]
@@ -167,11 +175,8 @@ class RosettaEcoreUtil {
 		allAnnotations.exists[annotation?.name == "calculation"]
 	}
 
-	def boolean isReference(Attribute attribute) {
-		return attribute.hasMetaDataAnnotations || attribute.hasMetaDataAddress
-	}
 	def boolean isReference(RAttribute attribute) {
-		return attribute.hasMetaDataAnnotations || attribute.hasMetaDataAddress
+		return attribute.hasMetaDataReference || attribute.hasMetaDataAddress
 	}
 
 	def private allAnnotations(Annotated withAnnotations) {
@@ -222,6 +227,30 @@ class RosettaEcoreUtil {
 		return '''«containerName»«name»'''
 	}
 
+	/*
+	 * This method is resolving references during scoping which is not an advised approach.
+	 * It could lead to poor performance as it is possible that it could be called upon to
+	 * resolve across multiple files. For now this is acceptable as in reality it's not going
+	 * going to get called to run across multiple files.
+	 * 
+	 * TODO: find an alternative approach to this.
+	 * 
+	 */
+ 	private def List<RosettaFeature> getMetaDescriptions(RMetaAnnotatedType type, EObject context) {
+ 		val metas = type.metaAttributes.map[it.name].toList
+ 		if (!metas.isEmpty) {
+ 			configs.findMetaTypes(context).filter[
+ 				metas.contains(it.name.lastSegment.toString)
+ 			]
+ 			.map[it.EObjectOrProxy]
+			.map[EcoreUtil.resolve(it, context)]
+ 			.filter(RosettaFeature)
+ 			.toList
+ 		} else {
+ 			emptyList
+ 		}
+ 	}
+
 	@Deprecated
 	def String toConditionJavaType(String conditionName) {
 		val allUnderscore = CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, conditionName)
@@ -231,9 +260,9 @@ class RosettaEcoreUtil {
 
 	@Deprecated
 	def String javaAnnotation(RAttribute attr) {
-		if (attr.name == "key" && attr.RType.name == "Key" && attr.RType.namespace.toString == "com.rosetta.model.lib.meta") {
+		if (attr.name == "key" && attr.RMetaAnnotatedType.RType.name == "Key" && attr.RMetaAnnotatedType.RType.namespace.toString == "com.rosetta.model.lib.meta") {
 			return 'location'
-		} else if (attr.name == "reference" && attr.RType.name == "Reference" && attr.RType.namespace.toString == "com.rosetta.model.lib.meta") {
+		} else if (attr.name == "reference" && attr.RMetaAnnotatedType.RType.name == "Reference" && attr.RMetaAnnotatedType.RType.namespace.toString == "com.rosetta.model.lib.meta") {
 			return 'address'
 		} else
 			return attr.name
@@ -247,8 +276,7 @@ class RosettaEcoreUtil {
 				'meta',
 				null,
 				emptyList,
-				provideMetaFieldsType(t),
-				emptyList,
+				provideMetaFieldsType(t).withMeta(#[]),
 				PositiveIntegerInterval.bounded(0, 1),
 				null,
 				null

diff --git a/.../src/main/java/com/regnosys/rosetta/generator/java/expression/DeepPathUtilGenerator.xtend b/.../src/main/java/com/regnosys/rosetta/generator/java/expression/DeepPathUtilGenerator.xtend
@@ -24,6 +24,7 @@ import java.util.HashSet
 import com.regnosys.rosetta.generator.java.statement.builder.JavaVariable
 import com.regnosys.rosetta.types.RAttribute
 import com.regnosys.rosetta.types.RChoiceType
+import static extension com.regnosys.rosetta.types.RMetaAnnotatedType.*
 
 class DeepPathUtilGenerator {
 	@Inject extension ImportManagerExtension
@@ -54,7 +55,7 @@ class DeepPathUtilGenerator {
 		val deepFeatures = choiceType.findDeepFeatures
 		val dependencies = new HashSet<JavaClass<?>>()
 		val recursiveDeepFeaturesMap = choiceType.allNonOverridenAttributes.toMap([it], [
-			val attrType = it.RType
+			val attrType = it.RMetaAnnotatedType.RType
 			deepFeatures.toMap([it], [
 				var t = attrType
 				if (t instanceof RChoiceType) {
@@ -98,13 +99,12 @@ class DeepPathUtilGenerator {
 	}
 
 	private def JavaStatementBuilder deepFeatureToStatement(RDataType choiceType, JavaVariable inputParameter, RAttribute deepFeature, Map<RAttribute, Map<RAttribute, Boolean>> recursiveDeepFeaturesMap, JavaScope scope) {
-		val deepFeatureHasMeta = !deepFeature.metaAnnotations.empty
 		val attrs = choiceType.allNonOverridenAttributes.toList
 		var JavaStatementBuilder acc = JavaExpression.NULL
 		for (a : attrs.reverseView) {
 			val currAcc = acc
 			acc = inputParameter
-					.featureCall(choiceType, a, false, scope, true)
+					.attributeCall(choiceType.withEmptyMeta, a, false, scope)
 					.declareAsVariable(true, a.name.toFirstLower, scope)
 					.mapExpression[attrVar|
 						attrVar.exists(ExistsModifier.NONE, scope)
@@ -114,7 +114,8 @@ class DeepPathUtilGenerator {
 								val deepFeatureExpr = if (deepFeature.match(a)) {
 									attrVar
 								} else {
-									var attrType = a.RType
+									val metaRType = a.RMetaAnnotatedType
+									var attrType = metaRType.RType
 									if (attrType instanceof RChoiceType) {
 										attrType = attrType.asRDataType
 									}
@@ -124,7 +125,7 @@ class DeepPathUtilGenerator {
 									} else {
 										(attrType as RDataType).allNonOverridenAttributes.findFirst[name.equals(deepFeature.name)]
 									}
-									attrVar.featureCall(attrType, actualFeature, needsToGoDownDeeper, scope, !deepFeatureHasMeta)
+									attrVar.attributeCall(metaRType, actualFeature, needsToGoDownDeeper, scope)
 								}
 								new JavaIfThenElseBuilder(it, deepFeatureExpr, currAcc, typeUtil)
 							]