Skip to content

Commit

Permalink
A) Moved from concrete objects to supplier during LeiaBundle init
Browse files Browse the repository at this point in the history
B) Linting fixes and sonar fixes
  • Loading branch information
koushikr committed Nov 19, 2024
1 parent 36c1df9 commit 6f1d90b
Show file tree
Hide file tree
Showing 41 changed files with 223 additions and 93 deletions.
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,16 @@

All notable changes to this project will be documented in this file.

## [0.0.1-RC5]

- Introduced a PermissionValidator and introduced suppliers for versionGenerator, SchemaUpdater and PermissionValidator, to facilitate clients implement them via Dependency Injection
- Removed the unnecessary `LeiaMessages` data-model and converged to a Map Structure
- A few linting fixes, along the way

## [0.0.1-RC4]

- Periodic Refresh - Added support to enable/disable periodicRefresh. Default value is true
- Event Multiplexing - Added support for event transformations. Can multiplex one event into multiple events.
- Event Multiplexing - Added support for event transformations. Can multiplex one event into multiple events.

## [0.0.1-RC3]

Expand Down
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ Leia is a governance and metadata framework aimed at meeting compliance requirem

### Using the schema registry

#### Build your own dropwizard schema server by using the `LeiaElastic` bundle.
#### Build your own dropwizard schema server by using the `LeiaElastic` bundle.

```
new LeiaElasticBundle<TestConfiguration, SchemaUpdater>() {
Expand All @@ -71,9 +71,10 @@ Leia is a governance and metadata framework aimed at meeting compliance requirem
}
```

- **SchemaUpdater** is an RBAC governing class, please extend this SchemaUpdater to implement your own UserProfile
- **CacheConfig** - If the schema will be always resolved from the dataStore (Elasticsearch) or from the in-memory cache with a refreshInterval to refresh the data
- **CacheConfig** - If the schema will be always resolved from the dataStore (Elasticsearch) or from the in-memory cache
with a refreshInterval to refresh the data
- **VersionIdGenerator** - Your own version id generator, to generate a unique versionId for every document
- **ElasticConfig** - Elasticsearch configuration to bring up the schema server

Expand Down Expand Up @@ -126,7 +127,10 @@ A sample schema looks like the following
```

- **AttributeInfo** : There are various type of attributes you can define, please refer to the `SchemaAttribute` class.
- **TransformationTargets** - Helps in event multiplexing, in the above example, when provided with the namespace, `testNamespace` and schemaName, `testSchema` with version `V1234`, during message production the `LeiaMessageProduceClient`, will multiplex the testSchema to both the versions, the transformationTargets ought to be jsonPathRules.
- **TransformationTargets** - Helps in event multiplexing, in the above example, when provided with the
namespace, `testNamespace` and schemaName, `testSchema` with version `V1234`, during message production
the `LeiaMessageProduceClient`, will multiplex the testSchema to both the versions, the transformationTargets ought to
be jsonPathRules.

#### Using the LeiaClientBundle

Expand Down
2 changes: 1 addition & 1 deletion leia-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>com.grookage.leia</groupId>
<artifactId>leia</artifactId>
<version>0.0.1-RC4</version>
<version>0.0.1-RC5</version>
</parent>

<artifactId>leia-bom</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion leia-client-dropwizard/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>com.grookage.leia</groupId>
<artifactId>leia-parent</artifactId>
<version>0.0.1-RC4</version>
<version>0.0.1-RC5</version>
<relativePath>../leia-parent</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion leia-client/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<parent>
<groupId>com.grookage.leia</groupId>
<artifactId>leia-parent</artifactId>
<version>0.0.1-RC4</version>
<version>0.0.1-RC5</version>
<relativePath>../leia-parent</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
package com.grookage.leia.client;

import com.grookage.leia.models.mux.LeiaMessage;
import com.grookage.leia.models.mux.LeiaMessages;
import com.grookage.leia.models.schema.SchemaKey;
import com.grookage.leia.models.schema.transformer.TransformationTarget;
import com.jayway.jsonpath.DocumentContext;
Expand All @@ -33,6 +32,7 @@
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Optional;
import java.util.function.UnaryOperator;


@EqualsAndHashCode(callSuper = true)
Expand Down Expand Up @@ -80,29 +80,35 @@ private Optional<JsonPath> getJsonPath(SchemaKey schemaKey, String attributeName
Optional.ofNullable(compiledPaths.get(schemaKey).get(attributeName)) : Optional.empty();
}

@SneakyThrows
public LeiaMessages getMessages(SchemaKey schemaKey, byte[] sourceMessage) {
private Map<SchemaKey, LeiaMessage> getMessages(SchemaKey schemaKey, byte[] sourceMessage) {
final var messages = new HashMap<SchemaKey, LeiaMessage>();
messages.put(schemaKey, LeiaMessage.builder()
.schemaKey(schemaKey)
.message(sourceMessage)
.build()
);
final var sourceSchemaDetails = super.getSchemaDetails()
.stream().filter(each -> each.getSchemaKey().equals(schemaKey))
.findFirst().orElse(null);
final var messages = new LeiaMessages();
messages.add(
LeiaMessage.builder()
.schemaKey(schemaKey)
.message(sourceMessage)
.build()
);

final var transformationTargets = null == sourceSchemaDetails ? null :
sourceSchemaDetails.getTransformationTargets();
if (null == transformationTargets) {
return messages;
}
final var documentContext = JsonPath.parse(new String(sourceMessage));
transformationTargets.forEach(transformationTarget ->
createMessage(documentContext, transformationTarget).ifPresent(messages::add));
createMessage(documentContext, transformationTarget).ifPresent(message ->
messages.put(message.getSchemaKey(), message)));
return messages;
}

public void processMessages(final SchemaKey schemaKey,
final byte[] sourceMessage,
final UnaryOperator<Map<SchemaKey, LeiaMessage>> messageHandler) {
messageHandler.apply(getMessages(schemaKey, sourceMessage));
}

@Override
public void start() {
super.getSchemaDetails().forEach(schemaDetails -> {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,10 @@ void testLeiaMessageProduceClient() {
.schemaUnits(List.of(TestSchemaUnit.builder()
.registeredName("testRegisteredName").build()))
.build();
final var messages = schemaClient.getMessages(sourceSchema, mapper.writeValueAsBytes(testSchema));
Assertions.assertFalse(messages.getMessages().isEmpty());
Assertions.assertEquals(2, messages.getMessages().size());
schemaClient.processMessages(sourceSchema, mapper.writeValueAsBytes(testSchema), messages -> {
Assertions.assertFalse(messages.isEmpty());
Assertions.assertEquals(2, messages.size());
return messages;
});
}
}
2 changes: 1 addition & 1 deletion leia-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<parent>
<groupId>com.grookage.leia</groupId>
<artifactId>leia-parent</artifactId>
<version>0.0.1-RC4</version>
<version>0.0.1-RC5</version>
<relativePath>../leia-parent</relativePath>
</parent>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,14 @@
import java.util.Arrays;
import java.util.Map;
import java.util.Optional;
import java.util.function.Supplier;

@Slf4j
public class SchemaProcessorHub {

private final Map<SchemaEvent, SchemaProcessor> processors = Maps.newHashMap();
private SchemaRepository schemaRepository;
private VersionIDGenerator versionIDGenerator;
private Supplier<VersionIDGenerator> versionSupplier;

private SchemaProcessorHub() {

Expand All @@ -49,14 +50,14 @@ public SchemaProcessorHub withSchemaRepository(SchemaRepository schemaRepository
return this;
}

public SchemaProcessorHub withVersionIDGenerator(VersionIDGenerator versionIDGenerator) {
this.versionIDGenerator = versionIDGenerator;
public SchemaProcessorHub wtihVersionSupplier(Supplier<VersionIDGenerator> versionSupplier) {
this.versionSupplier = versionSupplier;
return this;
}

public SchemaProcessorHub build() {
Preconditions.checkNotNull(schemaRepository, "Schema Repository can't be null");
Preconditions.checkNotNull(versionIDGenerator, "Version ID Generator can't be null");
Preconditions.checkNotNull(versionSupplier, "Version ID Generator can't be null");
Arrays.stream(SchemaEvent.values()).forEach(this::buildProcessor);
return this;
}
Expand All @@ -67,31 +68,31 @@ public void buildProcessor(final SchemaEvent event) {
public SchemaProcessor schemaCreate() {
return CreateSchemaProcessor.builder()
.schemaRepository(schemaRepository)
.versionIDGenerator(versionIDGenerator)
.versionSupplier(versionSupplier)
.build();
}

@Override
public SchemaProcessor schemaUpdate() {
return UpdateSchemaProcessor.builder()
.schemaRepository(schemaRepository)
.versionIDGenerator(versionIDGenerator)
.versionSupplier(versionSupplier)
.build();
}

@Override
public SchemaProcessor schemaApprove() {
return ApproveSchemaProcessor.builder()
.schemaRepository(schemaRepository)
.versionIDGenerator(versionIDGenerator)
.versionSupplier(versionSupplier)
.build();
}

@Override
public SchemaProcessor schemaReject() {
return RejectSchemaProcessor.builder()
.schemaRepository(schemaRepository)
.versionIDGenerator(versionIDGenerator)
.versionSupplier(versionSupplier)
.build();
}
}));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ public void process(SchemaContext context) {
}
final var userName = ContextUtils.getUser(context);
final var email = ContextUtils.getEmail(context);
final var schemaDetails = SchemaUtils.toSchemaDetails(createSchemaRequest, userName, email, getVersionIDGenerator());
final var schemaDetails = SchemaUtils.toSchemaDetails(createSchemaRequest, userName, email, getVersionSupplier());
getSchemaRepository().create(schemaDetails);
context.addContext(SchemaDetails.class.getSimpleName(), schemaDetails);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@
import lombok.Data;
import lombok.experimental.SuperBuilder;

import java.util.function.Supplier;

@SuperBuilder
@Data
@AllArgsConstructor
public abstract class SchemaProcessor {

private final SchemaRepository schemaRepository;
private final VersionIDGenerator versionIDGenerator;
private final Supplier<VersionIDGenerator> versionSupplier;

public abstract SchemaEvent name();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,4 @@ public static String getEmail(final SchemaContext schemaContext) {
return schemaContext.getValue(EMAIL)
.orElseThrow((Supplier<Throwable>) () -> LeiaException.error(LeiaErrorCode.VALUE_NOT_FOUND));
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,20 @@
import com.grookage.leia.models.schema.ingestion.CreateSchemaRequest;
import lombok.experimental.UtilityClass;

import java.util.function.Supplier;

@UtilityClass
public class SchemaUtils {

public SchemaDetails toSchemaDetails(final CreateSchemaRequest createSchemaRequest,
final String userName,
final String email,
final VersionIDGenerator versionIDGenerator) {
final Supplier<VersionIDGenerator> versionSupplier) {
return SchemaDetails.builder()
.schemaKey(SchemaKey.builder()
.namespace(createSchemaRequest.getNamespace())
.schemaName(createSchemaRequest.getSchemaName())
.version(versionIDGenerator.generateVersionId("V"))
.version(versionSupplier.get().generateVersionId("V"))
.build()
)
.schemaState(SchemaState.CREATED)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ public String generateVersionId(String prefix) {
};
final var hub = SchemaProcessorHub.of()
.withSchemaRepository(schemaRepository)
.withVersionIDGenerator(generator)
.wtihVersionSupplier(() -> generator)
.build();
Assertions.assertNotNull(hub.getProcessor(SchemaEvent.CREATE_SCHEMA).orElse(null));
Assertions.assertThrows(NullPointerException.class, () -> SchemaProcessorHub.of()
.withVersionIDGenerator(generator)
.wtihVersionSupplier(() -> generator)
.build());
Assertions.assertThrows(NullPointerException.class, () -> SchemaProcessorHub.of()
.withSchemaRepository(schemaRepository)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ class ApproveSchemaProcessorTest extends SchemaProcessorTest {
SchemaProcessor getSchemaProcessor() {
return ApproveSchemaProcessor.builder()
.schemaRepository(getSchemaRepository())
.versionIDGenerator(getGenerator())
.versionSupplier(getGenerator())
.build();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ class CreateSchemaProcessorTest extends SchemaProcessorTest {
@Override
SchemaProcessor getSchemaProcessor() {
return CreateSchemaProcessor.builder()
.versionIDGenerator(getGenerator())
.versionSupplier(getGenerator())
.schemaRepository(getSchemaRepository())
.build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ public class RejectSchemaProcessorTest extends SchemaProcessorTest {
SchemaProcessor getSchemaProcessor() {
return RejectSchemaProcessor.builder()
.schemaRepository(getSchemaRepository())
.versionIDGenerator(getGenerator())
.versionSupplier(getGenerator())
.build();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,12 @@
import org.junit.jupiter.api.BeforeEach;
import org.mockito.Mockito;

import java.util.function.Supplier;

public abstract class SchemaProcessorTest {

@Getter
private static final VersionIDGenerator generator = new VersionIDGenerator() {
@Override
public String generateVersionId(String prefix) {
return "V1234";
}
};
private static final Supplier<VersionIDGenerator> generator = () -> prefix -> "V1234";
@Getter
private static SchemaRepository schemaRepository;
private static SchemaProcessor schemaProcessor;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ void testUpdateSchemasNoDetails() {
@Override
SchemaProcessor getSchemaProcessor() {
return UpdateSchemaProcessor.builder()
.versionIDGenerator(getGenerator())
.versionSupplier(getGenerator())
.schemaRepository(getSchemaRepository())
.build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ public String name() {
public String email() {
return "email";
}

@Override
public String namespace() {
return "namespace";
}
});
Assertions.assertNotNull(ContextUtils.getEmail(schemaContext));
Assertions.assertNotNull(ContextUtils.getUser(schemaContext));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

import java.util.function.Supplier;

class SchemaUtilsTest {

@Test
Expand All @@ -40,7 +42,7 @@ public String generateVersionId(String prefix) {
}
};
final var schemaDetails = SchemaUtils.toSchemaDetails(createSchemaRequest, "testUser",
"testEmail", generator
"testEmail", () -> generator
);
Assertions.assertNotNull(schemaDetails);
final var schemaKey = schemaDetails.getSchemaKey();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,10 @@ public String name() {
public String email() {
return "[email protected]";
}

@Override
public String namespace() {
return "grookage";
}

}
Loading

0 comments on commit 6f1d90b

Please sign in to comment.