Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Testcontainers And Add Different Cursor Behaviour #72

Merged
merged 37 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
921a15f
Reference new JsonPatch Based maven plugin
geso02 Jun 9, 2023
71cbf42
Add extension search and combinedWith aggregation of queries
geso02 Jun 22, 2023
c8e416c
Remove return value of path visitor
geso02 Jun 22, 2023
2bbfc66
Add extension indeces
geso02 Jun 22, 2023
4fa1ebc
Define source folders in POM
geso02 Jun 22, 2023
e8a53c2
Add client documentation for aggregate and extension search
geso02 Jun 22, 2023
6a13c0b
Use fixed version for json patch
geso02 Jun 23, 2023
640c648
Remove unused empty file
geso02 Jun 23, 2023
3017eab
Merge branch 'eclipse-basyx:main' into main
geso02 Jun 26, 2023
dc5df83
Add submodel registry implementation
geso02 Jun 29, 2023
02b9aa6
Add submodel registry to parent aggregator
geso02 Jun 29, 2023
0fa56ed
Add and readme and fix readme files
geso02 Jun 29, 2023
8576e70
Enable kafak logs and increase connection timeout for tests
geso02 Jun 29, 2023
19b78e6
Add additional Wait strategy to wait for KAFKA
geso02 Jul 4, 2023
4e7bf34
Merge branch 'eclipse-basyx:main' into main
geso02 Jul 4, 2023
8e825cb
Merge branch 'main' of https://github.com/geso02/basyx-java-server-sdk
geso02 Jul 4, 2023
efb779f
Implement v3.0.1 features
geso02 Jul 5, 2023
0ef0a8b
Add location-header support
geso02 Jul 6, 2023
e2f0f29
Update build and integration test process and remove altered openapi …
geso02 Jul 10, 2023
278b412
Merge branch 'eclipse-basyx:main' into main
geso02 Jul 10, 2023
82d0bc9
Update integration tests and check each event
geso02 Jul 14, 2023
1a4b988
Merge branch 'main' of https://github.com/geso02/basyx-java-server-sdk
geso02 Jul 14, 2023
3005f90
Merge branch 'eclipse-basyx:main' into main
geso02 Jul 14, 2023
c2fc619
Resolve merge problems
geso02 Jul 14, 2023
6dac98b
Merge branch 'eclipse-basyx:main' into main
geso02 Aug 9, 2023
236465d
Use Amazon Corretto base image
geso02 Aug 9, 2023
442c51c
Remove test containers and use external docker-compose instead
geso02 Aug 14, 2023
e9996fc
Return the last element id as cursor for inmemory as we do it for mongo
geso02 Aug 14, 2023
383f769
Merge branch 'eclipse-basyx:main' into main
geso02 Aug 14, 2023
5d1e969
Remove experimental Decorator lombok annotation
geso02 Aug 16, 2023
65f88ae
Merge branch 'main' of https://github.com/geso02/basyx-java-server-sdk
geso02 Aug 16, 2023
535480c
Merge branch 'eclipse-basyx:main' into main
geso02 Aug 16, 2023
edf018e
Wait for Kafka partition assignment in testcases
geso02 Sep 20, 2023
531b5e7
Remove remaining lombok.experimental Annotations
geso02 Sep 20, 2023
36cd451
Merge branch 'main' into main
geso02 Sep 20, 2023
ee48a22
Use a differnt submodel then requesting paginated as we return a diff…
geso02 Sep 20, 2023
fbb8913
Remove test files
geso02 Sep 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
import lombok.RequiredArgsConstructor;
import lombok.Setter;
import lombok.ToString;
import lombok.experimental.UtilityClass;

@Data
public class PathInfo {
Expand Down Expand Up @@ -187,10 +186,12 @@ public ModelInfo(String name) {

}

@UtilityClass
private static class ConstantGenerator {
private static final class ConstantGenerator {

public String generateConstant(String name) {
private ConstantGenerator() {
}

public static String generateConstant(String name) {
StringBuilder builder = new StringBuilder();
for (int i = 0, len = name.length(); i < len; i++) {
char c = name.charAt(i);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ We use an extra project here instead of *test-jar* maven artifact generation, as

Have a look at the mongoDb-storage project or the release projects to see how the abstract test classes defined here can be used. The classes provide good test coverage. You can extend them without writing additional test cases for your storage.

Use [testcontainers](https://www.testcontainers.org/) for your integration or storage tests.

The integration test of the [kafka-mongodb release](../basyx.aasregistry-service-release-kafka-mongodb/Readme.md) project starts an Apache Kafka and an MongoDb instance using testcontainers. Don't forget to start a docker daemon before running the tests.
Don't forget to start a docker daemon before running the tests and to run the docker-compose file of the parent folder.



5 changes: 0 additions & 5 deletions basyx.aasregistry/basyx.aasregistry-service-basetests/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,6 @@
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>${testcontainers.version}</version>
</dependency>

<dependency>
<groupId>junit</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,22 +29,21 @@
import org.eclipse.digitaltwin.basyx.aasregistry.model.AssetAdministrationShellDescriptor;
import org.eclipse.digitaltwin.basyx.aasregistry.model.SubmodelDescriptor;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.AasRegistryStorage;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.AasRegistryStorageDelegate;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.DescriptorCopies;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.DescriptorFilter;
import org.eclipse.digitaltwin.basyx.core.pagination.CursorResult;
import org.eclipse.digitaltwin.basyx.core.pagination.PaginationInfo;

import lombok.NonNull;
import lombok.RequiredArgsConstructor;
import lombok.experimental.Delegate;

//performs additional cloning for in memory tests so that
//altering the objects during tests will then not affect the storage
@RequiredArgsConstructor
public class CloningAasRegistryStorageDecorator implements AasRegistryStorage {
public class CloningAasRegistryStorageDecorator extends AasRegistryStorageDelegate {

@Delegate
private final AasRegistryStorage storage;
public CloningAasRegistryStorageDecorator(AasRegistryStorage storage) {
super(storage);
}

@Override
public CursorResult<List<AssetAdministrationShellDescriptor>> getAllAasDescriptors(@NonNull PaginationInfo pRequest, @NonNull DescriptorFilter filter) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,9 @@
import org.junit.Test;
import org.mockito.ArgumentMatchers;
import org.mockito.Mockito;
import org.testcontainers.shaded.com.fasterxml.jackson.databind.ObjectMapper;

import com.fasterxml.jackson.annotation.JsonInclude.Include;
import com.fasterxml.jackson.databind.ObjectMapper;


public abstract class ExtensionsTest extends BaseInterfaceTest {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,6 @@
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
Expand Down Expand Up @@ -137,21 +136,18 @@ public abstract class BaseIntegrationTest {
@Rule
public TestResourcesLoader resourceLoader = new TestResourcesLoader(BaseIntegrationTest.class.getPackageName(), mapper);

@Autowired
private BaseEventListener listener;

protected RegistryAndDiscoveryInterfaceApi api;

@Before
public void initClient() throws ApiException {
api = new RegistryAndDiscoveryInterfaceApi("http", "127.0.0.1", port);
api.deleteAllShellDescriptors();
listener.assertNoAdditionalMessage();
queue().assertNoAdditionalMessage();
}

@After
public void cleanup() throws ApiException {
listener.assertNoAdditionalMessage();
queue().assertNoAdditionalMessage();
GetAssetAdministrationShellDescriptorsResult result = api.getAllAssetAdministrationShellDescriptors(null, null, null, null);
for (AssetAdministrationShellDescriptor eachDescriptor : result.getResult()) {
api.deleteAssetAdministrationShellDescriptorById(eachDescriptor.getId());
Expand All @@ -178,7 +174,7 @@ public void whenWritingParallel_transactionManagementWorks() throws ApiException
assertThat(IntStream.iterate(0, i -> i + 1).limit(300).parallel().mapToObj(op).filter(i -> i > 300).findAny()).isEmpty();
assertThat(api.getAssetAdministrationShellDescriptorById(descriptor.getId()).getSubmodelDescriptors()).hasSize(300);
for (int i = 0; i < 300; i++) {
RegistryEvent evt = listener.poll();
RegistryEvent evt = queue().poll();
assertThat(evt.getId()).isEqualTo(descriptor.getId());
assertThat(Integer.parseInt(evt.getSubmodelId())).isGreaterThanOrEqualTo(0).isLessThan(300);

Expand Down Expand Up @@ -207,6 +203,7 @@ private Integer writeSubModel(String descriptorId, int idx) {

@Test
public void whenDeleteAll_thenAllDescriptorsAreRemoved() throws ApiException {

for (int i = 0; i < DELETE_ALL_TEST_INSTANCE_COUNT; i++) {
AssetAdministrationShellDescriptor descr = new AssetAdministrationShellDescriptor();
String id = "id_" + i;
Expand All @@ -227,13 +224,13 @@ public void whenDeleteAll_thenAllDescriptorsAreRemoved() throws ApiException {
HashSet<RegistryEvent> events = new HashSet<>();
// we do not have a specific order, so read all events first
for (int i = 0; i < DELETE_ALL_TEST_INSTANCE_COUNT; i++) {
events.add(listener.poll());
events.add(queue().poll());
}
for (int i = 0; i < DELETE_ALL_TEST_INSTANCE_COUNT; i++) {
assertThat(events.remove(RegistryEvent.builder().id("id_" + i).type(EventType.AAS_UNREGISTERED).build())).isTrue();
}
assertThat(events.isEmpty());
listener.assertNoAdditionalMessage();
queue().assertNoAdditionalMessage();
}

@Test
Expand All @@ -249,7 +246,7 @@ public void whenCreateAndDeleteDescriptors_thenAllDescriptorsAreRemoved() throws
all = api.getAllAssetAdministrationShellDescriptors(port, null, null, null).getResult();
assertThat(all).isEmpty();

listener.assertNoAdditionalMessage();
queue().assertNoAdditionalMessage();
}

@Test
Expand Down Expand Up @@ -282,7 +279,7 @@ public void whenRegisterAndUnregisterSubmodel_thenSubmodelIsCreatedAndDeleted()
aasDescriptor = api.getAssetAdministrationShellDescriptorById(aasId);
assertThat(aasDescriptor.getSubmodelDescriptors()).doesNotContain(toRegister);

listener.assertNoAdditionalMessage();
queue().assertNoAdditionalMessage();
}

@Test
Expand Down Expand Up @@ -616,7 +613,7 @@ private void assertRestResourceAvailable(String location) throws IOException {
}

private void deleteAdminAssetShellDescriptor(String aasId) throws ApiException {
listener.reset();
queue().reset();

int response = api.deleteAssetAdministrationShellDescriptorByIdWithHttpInfo(URLEncoder.encode(aasId, StandardCharsets.UTF_8)).getStatusCode();
assertThat(response).isEqualTo(NO_CONTENT);
Expand All @@ -640,7 +637,7 @@ private List<AssetAdministrationShellDescriptor> initialize() throws IOException
}

private void assertThatEventWasSend(RegistryEvent expected) {
RegistryEvent evt = listener.poll();
RegistryEvent evt = queue().poll();
assertThat(evt).isEqualTo(expected);
}

Expand Down Expand Up @@ -674,4 +671,6 @@ private <O,I> O convert(I in, Class<I> inCls, Class<O> outCls) throws JsonProces
String data = mapper.writerFor(inCls).writeValueAsString(in);
return mapper.readerFor(outCls).readValue(data);
}

public abstract EventQueue queue();
}
Original file line number Diff line number Diff line change
Expand Up @@ -31,51 +31,51 @@
import org.eclipse.digitaltwin.basyx.aasregistry.client.model.ProtocolInformation;
import org.eclipse.digitaltwin.basyx.aasregistry.client.model.SubmodelDescriptor;

import lombok.experimental.UtilityClass;

@UtilityClass
public class ClientRegistryTestObjects {

public AssetAdministrationShellDescriptor newAssetAdministrationShellDescriptor(String id) {
private ClientRegistryTestObjects() {
}

public static AssetAdministrationShellDescriptor newAssetAdministrationShellDescriptor(String id) {
AssetAdministrationShellDescriptor descriptor = new AssetAdministrationShellDescriptor();
descriptor.setId(id);
return descriptor;
}

public SubmodelDescriptor newSubmodelDescriptor(String id) {
public static SubmodelDescriptor newSubmodelDescriptor(String id) {
return newSubmodelDescriptorWithIdShort(id, null);
}

public SubmodelDescriptor newSubmodelDescriptorWithDescription(String id, String description) {
public static SubmodelDescriptor newSubmodelDescriptorWithDescription(String id, String description) {
SubmodelDescriptor descriptor = new SubmodelDescriptor();
descriptor.setId(id);
addDescription(descriptor, description);
return descriptor;
}

private void addDescription(SubmodelDescriptor descriptor, String description) {
private static void addDescription(SubmodelDescriptor descriptor, String description) {
if (description != null) {
LangStringTextType lString = newDescription(description);
descriptor.addDescriptionItem(lString);
}
}

private LangStringTextType newDescription(String sDescr) {
private static LangStringTextType newDescription(String sDescr) {
LangStringTextType descr = new LangStringTextType();
descr.setLanguage("de-DE");
descr.setText(sDescr);
return descr;
}

public SubmodelDescriptor newSubmodelDescriptorWithIdShort(String id, String idShort) {
public static SubmodelDescriptor newSubmodelDescriptorWithIdShort(String id, String idShort) {
SubmodelDescriptor descriptor = new SubmodelDescriptor();
descriptor.setId(id);
descriptor.setIdShort(idShort);

return descriptor;
}

public void addDefaultEndpoint(SubmodelDescriptor descriptor) {
public static void addDefaultEndpoint(SubmodelDescriptor descriptor) {
Endpoint endpoint = new Endpoint();
endpoint.setInterface("https://admin-shell.io/aas/API/3/0/SubmodelServiceSpecification/SSP-003");
ProtocolInformation protocolInfo = new ProtocolInformation();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,17 +28,18 @@
import java.util.concurrent.TimeUnit;

import org.eclipse.digitaltwin.basyx.aasregistry.service.events.RegistryEvent;
import org.springframework.beans.factory.annotation.Autowired;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

public abstract class BaseEventListener {
import lombok.RequiredArgsConstructor;

@RequiredArgsConstructor
public class EventQueue {

private LinkedBlockingQueue<String> messageQueue = new LinkedBlockingQueue<>();

@Autowired
private ObjectMapper mapper;
private final ObjectMapper mapper;

public boolean offer(String message) {
return messageQueue.offer(message);
Expand Down Expand Up @@ -68,7 +69,7 @@ public void assertNoAdditionalMessage() {

public RegistryEvent poll() {
try {
String message = messageQueue.poll(15, TimeUnit.SECONDS);
String message = messageQueue.poll(1, TimeUnit.MINUTES);
if (message == null) {
throw new EventListenerException("timeout");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@
import org.eclipse.digitaltwin.basyx.aasregistry.model.ShellDescriptorQuery.QueryTypeEnum;

import lombok.RequiredArgsConstructor;
import lombok.experimental.UtilityClass;

@UtilityClass
class SearchMatchers {

private SearchMatchers() {
}

public Map<String, List<BiPredicate<Object[], String>>> createMatchers(List<ShellDescriptorQuery> queryList) {
public static Map<String, List<BiPredicate<Object[], String>>> createMatchers(List<ShellDescriptorQuery> queryList) {
Map<String, List<BiPredicate<Object[], String>>> matchers = new HashMap<>();
for (ShellDescriptorQuery eachQuery : queryList) {
String path = eachQuery.getPath();
Expand All @@ -54,7 +54,7 @@ public Map<String, List<BiPredicate<Object[], String>>> createMatchers(List<Shel
}


public BiPredicate<Object[], String> createMatcher(ShellDescriptorQuery query) {
public static BiPredicate<Object[], String> createMatcher(ShellDescriptorQuery query) {
QueryTypeEnum queryType = query.getQueryType();
String value = query.getValue();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,11 +126,6 @@
</build>

<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>mongodb</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,7 @@

import static org.assertj.core.api.Assertions.assertThat;

import java.util.List;
import java.util.Optional;
import java.util.concurrent.ExecutionException;

import org.bson.Document;
import org.eclipse.digitaltwin.basyx.aasregistry.model.AssetKind;
Expand All @@ -38,46 +36,29 @@
import org.eclipse.digitaltwin.basyx.aasregistry.service.configuration.MongoDbConfiguration;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.DescriptorFilter;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.ShellDescriptorSearchRequests;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.ShellDescriptorSearchRequests.GroupedQueries;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.mongodb.MongoDbAasRegistryStorage;
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.mongodb.SearchQueryBuilder;
import org.junit.ClassRule;
import org.junit.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.TestPropertySource;
import org.testcontainers.containers.MongoDBContainer;
import org.testcontainers.utility.DockerImageName;

import com.mongodb.ExplainVerbosity;
import com.mongodb.client.MongoCollection;

@TestPropertySource(properties = { "registry.type=mongodb", "spring.data.mongodb.database=aasregistry" })
@TestPropertySource(properties = { "registry.type=mongodb", "spring.data.mongodb.database=aasregistry"
, "spring.data.mongodb.uri=mongodb://mongoAdmin:mongoPassword@localhost:27017" })
@ContextConfiguration(classes = { MongoDbConfiguration.class })
@EnableAutoConfiguration
public class MongoDbAasRegistryStorageTest extends AasRegistryStorageTest {

@Value("${spring.data.mongodb.database}")
private static String DATABASE_NAME;

@ClassRule
public static final MongoDBContainer MONGODB_CONTAINER = new MongoDBContainer(DockerImageName.parse("mongo:5.0.10"));

@Autowired
private MongoTemplate template;

@DynamicPropertySource
static void assignAdditionalProperties(DynamicPropertyRegistry registry) throws InterruptedException, ExecutionException {
String uri = MONGODB_CONTAINER.getConnectionString() + "/" + DATABASE_NAME;
registry.add("spring.data.mongodb.uri", () -> uri);
}

@Test
public void whenGetAllByFullFilter_NotAllDocumentsScannedButIndexUsed() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,5 @@
<artifactId>basyx.aasregistry-service-basetests</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>kafka</artifactId>
<version>${testcontainers.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Loading