Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meta secret core version 2 #3

Merged
merged 50 commits into from
Aug 8, 2024
Merged

Meta secret core version 2 #3

merged 50 commits into from
Aug 8, 2024

Conversation

cypherkitty
Copy link
Contributor

The next version of meta secret: decentralized database with state machine replication

@cypherkitty cypherkitty marked this pull request as ready for review April 11, 2023 03:45
@cypherkitty cypherkitty marked this pull request as draft April 11, 2023 03:49
v2, github actions: unit tests must be triggered for all pr-s
…log (#8)

core v2: implement basic join_cluster functionality on top of commit log
core v2:
 - implement vaults index
- fix the design of AppOperationType by adding operation types (Request,
Update)
 - fix join_cluster, sign_up functions according to the new approach
 - code reformat with: cargo fmt
#11)

V2: initial (draft) version of a communication protocol between client
and server. Implement Genesis, VaultsIndex, and SignUp requests/updates.
v2: make the "app" module work and bring embedded db:
 - change project structure, add: mock server and integration tests.
- migrate from rocksdb to SQLite. Rocksdb build is super slow and also,
SQLite will be used in the mobile app
 - fix app test: fix accept_join function
This pr adds links between events making events in a commit log in the
form of a linked list.
We need to have a linked list-like data structure:
 - to be able to search events 
 - have ordering between events
 - efficiently implement state machine replication
Added:
 - initial implementation of the mock server
 - sql migration scripts
 - unit test for the mock server and embedded migrations
- "indexes" has been added, and now it's possible to navigate forward having
only a simple KV store.
- Object layer has been added. Now we can build an object querying only
a subset of records relevant to the object. The querying mechanism works
on top of "indexes"
v2: implementing global_index and sing_up functionality on sqlite server
join_cluster implementation: accepts join request and adds a new member
into a vault
v2: 
- bring Cloudflare based serverless meta server
 - abstract away from using a specific database in the server
v2:
-  integrate cf worker with meta-server
- move meta-server into core module
v2: 
- bring "wasm" module from meta-secret-node repo, for faster development
and to reduce the complexity of the architecture of a multi-repo
project.
 - implement the "send" command in the worker
v2: type-driven design:
- migrate meta server to the type-driven composition of layers of the
application
Make object layer generic, which helps unify access to the databases on
any platform
v2: 
- redesign KvLogEvent's architecture on top of enums with generic
values, which simplified and clarified communication protocol and
reduced logical redundancy and complexity
 - improve code working with MetaVault
v2: application architecture redesign
- KvLogEvent enums, for better json serialization and more type safe
code
- going back to the classical way of app components composition (structs
+ traits -> DI). Prev version was based on traits mixins which is not
flexible enough.
v2, wasm: registration basics
…) improvements (#31)

v2:
 - wasm: basic implementation of meta secret server in browser
- basic version of database synchronization (embedded in browser)
between meta secret server and client
 
- Add "Unit" type into ObjectId to implement proper ordering of an
object creation: Unit(allocation request)->Genesis(server pk,
initialization)->Regular(updates and requests)
v2:
 - db sync between meta server and a client
 - MetaDb sync between local Db and a in memory snapshot (view) in app
v2, db sync: global index sync improvements
v2, state sync: 
 - mem-pool implementation for requests (join request so far)
- bring app state in rust from js (reactive updates of application state
from rust with notification to vue js app)
 - synchronization improvements
 - signup/join first full implementation
v2: add vault into the app state
v2 meta password: 
- initial, incomplete implementation of meta password
 - get rid of VaultObject (redundant abstraction over ObjectDescriptor)
v2, sync: complete refactoring of synchronization functionality,
simpler, cleaner workflow
v2, run meta server emulator asynchronously and independently of the
client workflow
#40)

v2: migrate to dynamic dispatching from generics (static ones) it
reduces a lot of boilerplate code
V2:
 - big refactoring
 - functionality of secret sharing  (initial steps)
v2: secret shares synchronization, part 1. 
Sync meta passwords with application state, which allows to show them in
the list
v2: 
- device specific mem pool (for shared secrets/split operation) for data
synchronization
v2:
 - code refactoring 
 - move business code from the wasm module to the core module
 - make wasm code more generic to reuse from wasm_bindgen tests
v2: move app manager initialization code from wasm to the core
v2, design improvements: 
- use channels instead of mutex to manage app state
v2, multiplatform 
- run core on wasm (wasm futures) and on a native platform (regular
threads)
 - write an integration test for the entire app
 - migrate to tracing (logger)
v2, tracing/bug fix:
 - tracing adoption
 - fixing "join cluster" bug
v2, "join cluster" fix:
 - add more verification steps into meta_app_service_test
 - "join cluster" bug has FIXED
 - use meta_dd_service from data_sync
v2, "secret sharing" design improvements 
- now we have an audit table that keeps all
split/recover/recovery_request events
- secret sharing synchronization uses the audit table to track all
records which needs to be synchronized
 - it makes system more deterministic and simpler
Test all the things!
 - object_descriptor: db_tail
@cypherkitty cypherkitty marked this pull request as ready for review August 8, 2024 01:28
@cypherkitty cypherkitty merged commit d9c3651 into main Aug 8, 2024
1 check failed
@cypherkitty cypherkitty deleted the v2 branch August 8, 2024 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant