Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add more MLS methods #321

Merged
merged 37 commits into from
Oct 27, 2023
Merged

Add more MLS methods #321

merged 37 commits into from
Oct 27, 2023

Conversation

neekolas
Copy link
Collaborator

@neekolas neekolas commented Oct 24, 2023 •
edited
Loading

Summary

  • Implements PublishToGroup method
  • Implements PublishWelcomes method
  • Implements GetIdentityUpdates method
  • Adds more validations to incoming RPCs

neekolas added 23 commits October 19, 2023 14:50
@neekolas
Scaffold MLS server
94460bc
@neekolas
Update go.mod
ae8023f
@neekolas
Fix missing argument
0fccac9
@neekolas
Add unsaved file
9f98795
@neekolas
Lint
d496271
@neekolas
Working end-to-end
0eda254
@neekolas
Merge branch 'mls' into validation-service-scaffold
2db5d9d
@neekolas
Lint
a3f7352
@neekolas
Add new push action
b833901
@neekolas
Add a bunch of new endpoints
f9f1c4e
@neekolas
Address review comments
8c94283
@neekolas
Change method casing
da11034
@neekolas
Change casing of server options
e09fca1
@neekolas
Change casing of validation options
293053a
@neekolas
Remove unused function
8849ad5
@neekolas
Merge branch 'validation-service-scaffold' into nmolnar/message-publi…
c00c75b 
…shing-services
@neekolas
Remove double pointer
17ea986
@neekolas
Make private again
b8b8b4d
@neekolas
Merge branch 'validation-service-scaffold' into nmolnar/message-publi…
dd63aba 
…shing-services
@neekolas
Fix pointer to key package
08266da
@neekolas
Capitalize more things
0ba5291
@neekolas
Update server fields
1e68170
@neekolas
Merge branch 'mls' into nmolnar/message-publishing-services
e3ab161
neekolas
neekolas commented Oct 24, 2023
View reviewed changes
pkg/store/store.go
@@ -20,6 +21,22 @@ import (

const maxPageSize = 100

const timestampGeneratorSql = `(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran all the SQL through a formatter and this is what came out. 🤷

@neekolas neekolas marked this pull request as ready for review October 24, 2023 17:07
snormore
snormore reviewed Oct 24, 2023
View reviewed changes
pkg/api/message/v3/service.go Outdated Show resolved Hide resolved
pkg/api/message/v3/service_test.go Outdated Show resolved Hide resolved
pkg/migrations/mls/20231023050806_init-schema.up.sql Show resolved Hide resolved
pkg/mlsstore/store.go Show resolved Hide resolved
pkg/store/store.go Outdated Show resolved Hide resolved
@neekolas
Copy link
Collaborator Author

I've switched things in both the Rust validation service and the node so that installation_id is just the full public key of the installation as bytes. I'm also now storing the credential in the DB and including it in the GetIdentityUpdates response, so that clients can perform validation purely on the response from GetIdentityUpdates.

@neekolas neekolas merged commit a6d28e0 into mls Oct 27, 2023
3 checks passed
@neekolas neekolas deleted the nmolnar/message-publishing-services branch October 27, 2023 01:44
snormore pushed a commit that referenced this pull request Jan 8, 2024
@neekolas
Add more MLS methods (#321)
4367d64 
* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity
snormore pushed a commit that referenced this pull request Jan 23, 2024
@neekolas
Add more MLS methods (#321)
4596a88 
* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity
snormore pushed a commit that referenced this pull request Jan 23, 2024
@neekolas
Add more MLS methods (#321)
8f6f633 
* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity
neekolas added a commit that referenced this pull request Jan 23, 2024
@neekolas @richardhuaaa
@Bren2010 @snormore
Add hpke public key (#337)
2164303 
* Validation service scaffold (#320)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Add more MLS methods (#321)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity

* Fix migrations

* refactor: walletaddress->accountaddress (#325)

Settling on consistent terminology as discussed

* go mod tidy

* Assume KeyPackages are always last resort. (#329)

* Assume KeyPackages are always last resort.

* Missing KeyPackages don't cause entire request to fail.

* Update go.mod

* UpdateKeyPackage returns an error if installation is unknown.

* refactor: move api/message/v3 to mls/api/v1 (#332)

* refactor: move message_api/v3 to mls/api/v1

* refactor: move pkg/mlsstore to pkg/mls/store

* Rename mls build/push workflow

* Decouple MLS messages from messagev1 (#333)

* Separate MLS messages and implement service methods

* fix: group id and installation id are bytes

* fix: idempotent send group/welcome messages via uniquness in db

* fix: hex decode group id from mls validation service

* fix: s/Cursor/IdCursor

* fix: pass message data only in send group message request

* refactor: add mls {Group,Welcome}MessageInput types for send requests

* refactor: s/installation_id/installation_key in mls/api

* fix: clean up mls query page size logic

* feat: implement mls subscribe group/welcome messages

* Hex encode group ID

* fix: remove duplicate import

* fix: return grpc invalidargument on invalid group id

---------

Co-authored-by: Nicholas Molnar <[email protected]>

* Generate protos in repo (#335)

* Add buf.gen.yaml

* Gen protos

* Update build steps

* Update protos

* Add hpke pub key

* Specify column

* Fix field name

* Fix migration

* Add data to test

* Do more validations on welcome messages

* Simplify check

* Fix insert statement

* Push up mls image

* Add new generated file

---------

Co-authored-by: Richard Hua <[email protected]>
Co-authored-by: Steven Normore <[email protected]>
Co-authored-by: Brendan McMillion <[email protected]>
Co-authored-by: Steven Normore <[email protected]>
snormore pushed a commit that referenced this pull request Jan 23, 2024
@neekolas @richardhuaaa @Bren2010
Merge mls branch into main (#343)
86fc580 
* Validation service scaffold (#320)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Add more MLS methods (#321)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity

* Fix migrations

* refactor: walletaddress->accountaddress (#325)

Settling on consistent terminology as discussed

* go mod tidy

* Assume KeyPackages are always last resort. (#329)

* Assume KeyPackages are always last resort.

* Missing KeyPackages don't cause entire request to fail.

* Update go.mod

* UpdateKeyPackage returns an error if installation is unknown.

* refactor: move api/message/v3 to mls/api/v1 (#332)

* refactor: move message_api/v3 to mls/api/v1

* refactor: move pkg/mlsstore to pkg/mls/store

* Rename mls build/push workflow

* Decouple MLS messages from messagev1 (#333)

* Separate MLS messages and implement service methods

* fix: group id and installation id are bytes

* fix: idempotent send group/welcome messages via uniquness in db

* fix: hex decode group id from mls validation service

* fix: s/Cursor/IdCursor

* fix: pass message data only in send group message request

* refactor: add mls {Group,Welcome}MessageInput types for send requests

* refactor: s/installation_id/installation_key in mls/api

* fix: clean up mls query page size logic

* feat: implement mls subscribe group/welcome messages

* Hex encode group ID

* fix: remove duplicate import

* fix: return grpc invalidargument on invalid group id

---------

Co-authored-by: Nicholas Molnar <[email protected]>

* Generate protos in repo (#335)

* Add buf.gen.yaml

* Gen protos

* Update build steps

* go mod tidy

* Remove push-mls github workflow

* Add hpke public key (#337)

* Validation service scaffold (#320)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Add more MLS methods (#321)

* Scaffold MLS server

* Update go.mod

* Fix missing argument

* Add unsaved file

* Lint

* Working end-to-end

* Lint

* Add new push action

* Add a bunch of new endpoints

* Address review comments

* Change method casing

* Change casing of server options

* Change casing of validation options

* Remove unused function

* Remove double pointer

* Make private again

* Fix pointer to key package

* Capitalize more things

* Update server fields

* Add test for sort methods

* Save change to capitalization

* Fix lint warnings

* Fix problem with mocks

* Fix index name

* Move sorting to the store

* Fix ciphertext validation

* Make installation_id bytes

* Add missing credential identity

* Hack sql in query

* Revert "Hack sql in query"

This reverts commit 168b78a.

* Remove custom type

* Update to latest protos

* Add CredentialIdentity

* Fix migrations

* refactor: walletaddress->accountaddress (#325)

Settling on consistent terminology as discussed

* go mod tidy

* Assume KeyPackages are always last resort. (#329)

* Assume KeyPackages are always last resort.

* Missing KeyPackages don't cause entire request to fail.

* Update go.mod

* UpdateKeyPackage returns an error if installation is unknown.

* refactor: move api/message/v3 to mls/api/v1 (#332)

* refactor: move message_api/v3 to mls/api/v1

* refactor: move pkg/mlsstore to pkg/mls/store

* Rename mls build/push workflow

* Decouple MLS messages from messagev1 (#333)

* Separate MLS messages and implement service methods

* fix: group id and installation id are bytes

* fix: idempotent send group/welcome messages via uniquness in db

* fix: hex decode group id from mls validation service

* fix: s/Cursor/IdCursor

* fix: pass message data only in send group message request

* refactor: add mls {Group,Welcome}MessageInput types for send requests

* refactor: s/installation_id/installation_key in mls/api

* fix: clean up mls query page size logic

* feat: implement mls subscribe group/welcome messages

* Hex encode group ID

* fix: remove duplicate import

* fix: return grpc invalidargument on invalid group id

---------

Co-authored-by: Nicholas Molnar <[email protected]>

* Generate protos in repo (#335)

* Add buf.gen.yaml

* Gen protos

* Update build steps

* Update protos

* Add hpke pub key

* Specify column

* Fix field name

* Fix migration

* Add data to test

* Do more validations on welcome messages

* Simplify check

* Fix insert statement

* Push up mls image

* Add new generated file

---------

Co-authored-by: Richard Hua <[email protected]>
Co-authored-by: Steven Normore <[email protected]>
Co-authored-by: Brendan McMillion <[email protected]>
Co-authored-by: Steven Normore <[email protected]>

* Populate options from MLS_DB_CONNECTION_STRING too

---------

Co-authored-by: Nicholas Molnar <[email protected]>
Co-authored-by: Richard Hua <[email protected]>
Co-authored-by: Brendan McMillion <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardhuaaa richardhuaaa richardhuaaa left review comments

@snormore snormore snormore approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@neekolas @snormore @richardhuaaa