Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/borisu/bricks
Browse files Browse the repository at this point in the history
  • Loading branch information
@borisu
borisu committed Jun 22, 2024
2 parents 116745f + 12bdfb7 commit e8179d8
Showing 1 changed file with 56 additions and 35 deletions.
91 changes: 56 additions & 35 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,24 @@

# BRICKS

Set of C++ abstract interfaces and pluggable implementations for microservices network services patterns.
Set of C++ abstract interfaces and pluggable implementations for microservices' network services patterns.

❗The project is in early stages under active development. Help, if you think it is valuable.❗



## About

The goal of the bricks project is to suggest API framework and POC implementations for the microservices communication patterns. By using abstract interface which hides the actual provider, developer is enabled to replace the communication engine at any time, use several engines in parallel in a uniform manner, avoid vendor lock-in, and mostly important replace networking services by mock objects, which improves drastically the testability of your C++ network services applications.
The goal of the bricks project is to suggest API framework and POC implementations for the microservices communication patterns. By using abstract interface, which hides the actual provider, developer is enabled to replace the communication engine at any time, use several engines in parallel in a uniform manner, avoid vendor lock-in, and mostly important, replace networking services by mock objects, which improves drastically the testability of your C++ network services applications.

Current supported communication paradigms are :-
- Publish/Subscribe (aka producer/consumer).
- P2P (a.k.a Bidi, a.k.a Pair)
- Request/Response
Following table summarizes currently available plugins and their capabilities :-

One intresting directions, that tthis abstraction is taking us, is implementation of network services on top of other services abstractions. For example, we can create request/response communication pattern, on top of abstract publish/subscribe service. That what is called meta plugin. Currently only one such plugin is implemented - pubsub_client and pubsub_server, both are tested with redis pub/sub plugins as underlying abstraction.

Following table summarizes currently available plugins and their capabilities :-
<table>
<tr>
<th>Plugin</th>
Expand Down Expand Up @@ -64,9 +68,49 @@ Following table summarizes currently available plugins and their capabilities :-
</tr>
</table>

## Example of Redis Sub Plugin
```cpp
/* create configuration for plugin */
auto xt = create_xtree_from_xml(
"<bricks>"
" <redispp>"
" <subscriber name=\"redispp_server_subscriber\" >"
" <methods>"
" <init>"
" <connection url=\"tcp://127.0.0.1:6379\" />"
" </init>"
" </methods>"
" </subscriber>"
" </redispp>"
"</bricks>"
);
auto subscriber = create_redispp_subscriber();

/* !!! from here the code is generic and should not change for any other plugin !!! */
/* create callback queue selector and start poller thread */
auto cb_q = create_callback_queue();
auto selector = create_selector();
selector->init(cb_q);
auto poller = create_poller(10000, selector);

/* create callback for handling topic */
auto on_topic_cb = [&](const string& topic, buffer_t* buf, xtree_t* xt)
{

printf("Received msg on topic %s\n", topic.c_str());
/* do something on topic update */
};


subscriber->init(selector->queue(), on_topic_cb, xt);
auto rc = subscriber->subscribe("some.interesting.topic", xt);

/* proceed to application logic and don't forget to release memory */
```
## Plugin Interface Design
Interface of the plugins was designed around following principles :-
Interface of the plugins was designed around following design principles :-
- Modern OOP (prefer function object callbacks to opaque handles)
- Be as slim as possible (avoid managing maps, or any other persistent data-structures)
- Avoid introducing interface which is peculiar to certain implementation.
Expand All @@ -76,34 +120,13 @@ Interface of the plugins was designed around following principles :-
All brick object are created with the help of factory methods and released via brick_t::release interface. This is to ensure, that plugin implementors can choose the memory handling mechanisms of their own, similar to MS COM interfaces design.
### XTree
All specific communication engine peculiarities are handled by xtree data structures which is passed as optional object to most of the interface methods. Xtree is conceptually an XML or alternatively hierarchical FS in OOP, e.g. tree like data structures where nodes can be accessed by string path, have set of properties of predefined list of types and can hold a single large char buffer as a value. XTree is not bound to XML in a way other than concept.

An example of xtree created directly from XML for ZeroMQ plugin intialization :-
All specific communication engine peculiarities, are handled by xtree data structures, which is passed as optional object to most of the interface methods. Xtree is conceptually an XML or alternatively hierarchical FS in OOP, e.g. tree like data structures where nodes can be accessed by string path, have set of properties of predefined list of types and can hold a single large char buffer as a value. XTree is not bound to XML in a way other than concept.
```
brick_uptr<xtree_t> xt1(
create_xtree_from_xml(
"<bricks>"
" <zmq>"
" <bidi name=\"zmq_p1\">"
" <methods>"
" <init>"
" <socket url=\"tcp://127.0.0.1:7858\" is_server=\"true\">"
" <sockopt id=\"ZMQ_BACKLOG\" value=\"100\" />"
" <sockopt id=\"ZMQ_MAXMSGSIZE\" value=\"%L:-1\" />"
" </socket>"
" </init>"
" </methods>"
" </bidi>"
" </zmq>"
"</bricks>"
));
```
### Threading Model And Safety
The flexibility of the engines introduces the challenge of consistent thread model. In such way, that replacing Kafka engine with ZeromMQ engine will not change the threading architecture of your program. The solution is to communciate with the engine via abstract queue of simple callbacks. To receive callback as result of topic subscription for example, one has to supply queue object when initiating the plugin and then "pump" the queue from the chosen thread. The callbacks in this case are promised to be called from the the same thread that is pumping the queue. Classes poller and selector are provided to automate the task of waiting on queue and invoking the callbacks. :-
The flexibility of the engines introduces the challenge of consistent thread model. In such way, that replacing Kafka engine with ZeromMQ engine will not change the threading architecture of your program. The solution is to communciate with the engine via abstract queue of simple callbacks. To receive callback, as result of topic subscription for example, one has to supply queue object when initiating the plugin and then "pump" the queue from the chosen thread. The callbacks in this case are promised to be called from the the same thread that is pumping the queue. Classes poller and selector are provided to automate the task of waiting on queue and invoking the callbacks. :-
```
```cpp
void
poller_impl_t::run()
{
Expand All @@ -113,17 +136,17 @@ poller_impl_t::run()
}
}
```
All public plugin methods are thread safe
All public plugin methods are thread safe.

### Plugins

Best way to see plugin usage is looking at implementation and their usgae in bricks_ut project
Best way to learn about plugin usage is by looking at their usage in bricks_ut project.

## How To Build

bricks uses vcpkg for the 3rd party packages.

Follow the same proces on Windows and Linux. For Window you should either run cmake --build or use generated solution instead of make.
Follow the same proces on Windows and Linux.

1. Ensure that cmake, developement tools, and git are available for Linux. For Windows, the build was tested with Visual Studio 2022 installed. Please be sure to install the gtest adapter extension to run bricks tests

Expand All @@ -134,19 +157,17 @@ Ensure that you have VCPKG_ROOT variable set to the vcpkg installation directory
```
export VCPKG_ROOT=<vcpkg root>
```

3. Clone the bricks directory and init all submodules
```
git clone https://github.com/borisu/bricks.git
cd bricks
git submodule update --init --recursive
```
4. Build bricks
```
cd <bricks>/src
mkdir build;cd build;cmake ..;make
mkdir build;cd build;cmake ..
```
5. Link your program to binary products

6. Enjoy :heart:

0 comments on commit e8179d8

Please sign in to comment.