Skip to content

Commit

Permalink
Merge pull request #3881 from wireapp/release_2024-02-12_10_08
Browse files Browse the repository at this point in the history
* replace example.com with wire.example, only in charts

* add basic information on debugging helm errors

* some minor additions for the helm troubleshooting docs

* remove mention of mandarin hostname

* [fix]: flaky test for leaving self-conv MLS (#3664)

* [WPB-4981] replace unclaimed keypackages atomically (#3654)

* add replace unclaimed key-packages route and endpoint
* Add key package replace test

---------

Co-authored-by: Paolo Capriotti <[email protected]>

* WPB-5098 Backend-to-backend OpenApi Docs (#3666)

* Integration tests flake when assuming federation ingress is up. (#3670)

* Added ingress check for dynamic backends in integration tests.

* Moved some args around. Better error for ingress.

* Restored nginz special handling.

* WIP: check 533 reason

* [WPB-5103] Add users to MLS conversation when some backends are unreachable (#3673)

* Add the copyright header to test modules

* Add two integration tests

The tests simply assert the expected behavior in MLS and confirm it is
the same as for Proteus

* Add a changelog

* A test case on adding an unreachable user

This is a scenario where a conversation already has a member from that
backend, but now the backend is unreachable. The test case has both the
Proteus and the MLS implementation and they are consistent in the
observed behavior.

* Fix extra remove proposal bug (#3672)

We were sending external remove proposals for each client of a user that was kicked out of a conversation following a remove commit. This was caused by some overgeneralisation of the mechanism that removes clients from subconversations when a user is deleted from the main.

* chore: [charts] Update team-settings version (#3658)

Co-authored-by: Zebot <[email protected]>

* Remove leftover PublicGroupState (#3675)

* Consume MLS messages from websocket (#3671)

* More robust consuming of MLS messages

This commit changes the behaviour of `sendAndConsumeMessage` and
`sendAndConsumeCommitBundle` to actually wait for those messages on the
client's websocket.

This should fix a lot of the flakiness of MLS tests that appeared after
the introduction of message queuing.

* Fix testAppMessageSomeReachable

When some backends are down, the new `sendAndConsume*` functions do not
work, because they expect a message to be received by all clients.

This commit changes tests with such a scenario to only post the message,
and not consume it.

* Add protocol field to MLS test state

This is necessary because new users in mixed MLS conversations don't get
join events, and we are waiting for such events before consuming MLS
messages.

* Add CHANGELOG entry

* Remove client check for subconversations (#3677)

* Update group state after application messages (#3678)

After an application message the ratchet is updated, therefore we need to save the updated group state so that future messages are generated correctly.

This commit includes an mls-test-cli update. The new mls-test-cli version modifies the `message` command to include both `group-in` and `group-out` options, as other similar commands already do.

* Fix galley DB migrations (#3680)

* Remove create-user/team scripts (#3683)

* nix/wire-server.nix: nixpkgs-fmt

* shell: add crate2nix

* cryptobox: package with crate2nix

* mls-test-cli: remove dead code, nixpkgs-fmt

These function args are unused.

* mls-test-cli: add TODO

* libs/libzauth/libzauth-c: run crate2nix generate

* libzauth-c: bump jwt-simple dep

Still using that wireapp/rust-jwt-simple repository, but at least the
latest version of the code, not a commit from Feb 10.

* nix/overlay.nix: nixpkgs-fmt

* zauth: build with crate2nix

This needs crate2nix 0.11.0 (from a more recent nixpkgs checkout), but
only during Cargo.nix recreation.

Let's hope it's there the next time we update this file.

* rusty_jwt_tools: describe why crate2nix doesn't work here

* add changelog

* WPB-5143 locked status for mls config (#3681)

* [feat] use nom in the direnv invocation if it is available (#3687)

* Refactor notification API descriptions (#3685)

* Remove Servant info from HasNotificationEndpoint

* Make component type family depend only on the kind

* WPB-4853: Swagger cleanup (#3674)

* [WPB-5208] Allow adding users to conversations when other backends are unreachable (#3688)

* Improve the RemoteDomains type

* Fix a reachable user test (some members unreachable)

* Fix the MLS test: testAddReachableWithUnreachableRemoteUsers

* Tests: assert an unreachable user cannot be added

* Add a changelog

* Swagger docs: new line after fed call tag (#3691)

* WPB-4848 Flaky test (#3689)

* Hotfix: Fixing how mls-test-cli is called (#3690)

WPB-5330: Remove command args that are upsetting mls-test-cli

* Revert "Hotfix: Fixing how mls-test-cli is called (#3690)" (#3694)

This reverts commit 58bcc0c.

It was not required, this breaks tests for everyone.

* [WPB-5241] add the timeout to the global and local environment (#3692)

* [feat] add the timeout to the global and local environment
- have the functions that want a timeout `ask` it from the environment
- adjust all usages to not take a timeout explicitly
- add asked timeout to Notifications

* Add tool to aggregate and push test statistics from junit/ant XML reports (#3652)

Co-authored-by: Akshay Mankar <[email protected]>

* [WPB-5042] upgrade nixpkgs to upgrade haskell-language-server (#3650)

* [feat] upgrade nixpkgs to upgrade hls and the hs pkg-set
- upgrade nixpkgs and nixpkgs cargo
- add necessary overrides to nixpkgs set
- fix fsnotify API changes
- pin ormolu
- updated hlint
- don't use Hashable for hashes that are supposed to be stable
- change the algorithm to calculate the hash in Prekey
- some minor changes to mls-test-cli
- port over the test from the old to the new testsuite
- fix behaviour of the cleanup function within `withResource`
- don't check the trailing dot test in nix tests
- restore consumeMessage1
- fix the test-suite such that we don't react on proposals

* WPB-1906 - Unverified users can no longer create assets (#3604)


Co-authored-by: Akshay Mankar <[email protected]>

* Coturn chart: allow installing multiple times in multiple namespaces (#3698)

* WPB-5204 Remove unused APNS_VOIP code (#3695)

We're no longer using the APNS_VOIP channel for native push notifications. Thus, we can delete the now unused code.

---------

Co-authored-by: Sven Tennie <[email protected]>

* Increase SQS timeout in galley integration (#3699)

It seems 3 seconds is not enough sometimes, and it was causing flakiness
of legalhold tests in CI.

* [WPB 5356] fix brig flaking (#3701)

* [WPB-2565] Do not send member updates to all (#3703)

* Do not send member updates to all (#3431)

---------

Co-authored-by: Paolo Capriotti <[email protected]>
Co-authored-by: Stefan Matting <[email protected]>

* Reduce the size for CI image by getting rid of 2 GHCs (#3712)

apply-refact refers to the GHC that it builds with by using the GHC.Paths
module. This tool is actually not required in CI.

ormolu was referring to GHC and all its haskell dependencies as
"propagatedBuildInputs". Using `hlib.justStaticExecutables` we can get rid of
these.

* [WPB-5175] upgrade to ghc 9.4 (#3679)

- fix swagger-json
- fix transitive-anns
- fix profunctor-schema
- fix ghc-source-gen (dependency of proto-lens-protoc)
- allow for loser package bounds on tools installed by cabal-install in cabal.project
- fix newly-introduced tests
- unmask the cleanup function after breaking change to `resource-pool` 
- fork and update text-icu-translit to text 2.0
- fix documentation generation 
- apply hlint hints and restructure illegible code

* [fix] set notificationTimeOut to 28 days, make it legible (#3714)

* WPB-5385 Extend internal federation config API with team ID (#3697)

* [docs] Update number of days the login cookie is valid for (#3717)

* [WPB-5376] Migrated from cryptonite to crypton. (#3711)

* [feat] add support for ghc-flakr's hs-run executable (#3716)

* Use Word64 to represent a ClientId (#3713)

* Use Word64 to represent a ClientId

* Rename client to clientToText

* Regenerate nix package

* Add openapi documentation for ClientId

* Fix golden tests

* Fix ClientId instances

* Preserve previous ClientId generation

* Add CHANGELOG entry

* Fix bound check in ClientId parser

* Document client ID generation

* Update group ID documentation (#3705)

* [fix] fix the envrc invocation (#3721)

* Turn long summaries into descriptions (#3706)

* Turn long summaries into descriptions

* [WPB-1226] Servantify internal Galley conversation endpoints (#3718)

* Rename WAI sitemaps

* Drop PUT /i/conversations/:cnv/channel

- This is an unused endpoint.

* Migrate GET /i/conversations/:cnv/members/:usr

* Migrate PUT /i/conversations/:cnv/accept/v2

* Migrate PUT /i/conversations/:cnv/block

* Migrate PUT /i/conversations/:cnv/unblock

* Migrate GET /i/conversations/:cnv/meta

* Add a changelog

* Mangoiv/fix envrc (#3724)

* [fix] add the $NIX_CONFIG environment variable

* Delete shell.nix (#3726)

It's broken. And, the official way to get a nix env for this project is
to use direnv.

* [fix] stern/backoffice conference calling TTL (#3723)

* change type of param to string

* [chore] Remove client ID conversion roundtrip (#3727)

* [chore] add link to PR to HaskellNetSSL upstream (#3728)

* fix local hspec option (#3730)

* WBP-5577 make replay nonce header accessible for frontend (#3729)

* [fix] fix discovery of directory (#3733)

* Add note about users own domain.

* updating the diagram and the source file

* fix: TURN tests failing b/c of fsnotify polling failing to detect (#3743)

changes.

* Better english sentence

Co-authored-by: Sven Tennie <[email protected]>

* CI: Increase memory limit for brig to 1Gi (#3751)

Co-authored-by: Igor Ranieri <[email protected]>

* docker-ephemeral: Run federation-v0 services for backwards compat testing (#3719)

Also in the commit:

Run docker-compose in daemon mode so there is less noise in the terminal.

* tasty-cannon: Delete awaitMatch_ (#3754)

* tasty-cannon: Delete awaitMatch_

This function doesn't tell the caller whether the expected event came or not and
hence pointless to be used in testing. It is used only in 1 place and that place
should be using assertMatch_ instead.

* galley-integration: Don't expect non-owner team members to get team join events

This functionality was removed in #3703

* Spar: Ensure mkValidExternalId returns a valid URef  (#3747)

* Spar: Ensure `mkValidExternalId` returns a valid URef

A valid URef can be used for lookups in tables spar.user and spar.user_v2 even
after issuer updates.

Co-authored-by: Akshay Mankar <[email protected]>

* WBP-5388 restrict contact search results according to team federation policy (#3732)

* WPB-5417 limit file upload to 100MiB (#3752)

* [fix] WPB-5715 data access layer of `federation_remotes` (#3758)

* WPB-4887 increased ingress payload size from 256k to 512k (#3756)

* [WPB-5603] Deleting a team member does not result in a conversation event (#3745)

* Add a delete team member test

* Make team member removal conditional in a helper

* Make a helper a top-level function

* Send `conversation.member-leave` to remotes too

* Add a changelog

* WBP-5133 External partners search restriction enforced by backend (#3708)

* galley-integration: Wait for starting legalhold test device (#3755)

* Fix calendar integration setting in backoffice / stern (#3761)

* Fix calenter integration setting in backoffice / stern.

* Improve spar test coverage (#3757)

- Some users had an email externalId when they were meant to have a nick, but only in *some* test runs.
- Test IdP update also for SAML-provisioned users.
- no new failures, yeay!

* [WPB-5603] Fix the team member deleted event reason (#3764)

* Write a test confirming the bug

* Parameterise the leave action by a reason

* Golden tests for ConversationRemoveMembers

* Update a changelog

* Federation error wrapping (#3742)

* Remove redundant copy of error body from Wai.Error

* Prevent unnecessary federator error wrapping

Federator is return Wai errors extended with extra data. However, that
extra data contains the infrastructure domain of the target backend,
which is not the right domain to show in the error.

Furthermore, when running integration tests locally, the domain reported
there is simply `localhost`, which is not considered a valid domain by
our JSON parser. That caused the error not to be recognised as a valid
Wai.Error, and therefore the error-catching middleware was rewrapping
it.

* Remove dead code

* Remove more dead code.

The `AsWai` class had a `waiErrorDescription` method, which forced every
error to implement that function even if they were not using it to
construct a `Wai.Error` value.  This is now gone, which means that two
of the errors don't have to implement it.

* Add inner error to Wai.Error

This can be used to represent nested failures (e.g. a federator
reporting a remote error) without having to serialise the nested error
into the message.

* Add nested error to federation remote error value

* Add CHANGELOG entry

* Test error wrapping

This test creates a fake ingress that always returns an error, then
tries to access it by making a federated user query.

* Lint

* Fix federation denied check in startBackend

* Make sure mock server is killed in the finaliser

* Fix root path in integration Mock

* Lint

* Use correct certificate paths in CI

* Set fallback inner error

* Spawn federator instead of ingress on error test

* Minor refactoring

Co-authored-by: Mango The Fourth <[email protected]>

* Restore explicit pattern matching

* Avoid boolean argument in mock server

---------

Co-authored-by: Mango The Fourth <[email protected]>

* Refactor getOptions (#3707)

We can combine the two parsers instead of invoking them both. This way
we get a help text even if no configuration file is passed.

* Simplify process spawning in integration tests (#3759)

* [feat] refactoring: use proper bracketing of services

* [chore] some minor cleanups and more comments

* [fix] class continuation in the right place

* [wip] some print statements and more experimentations around interrupt
signals

* Add bracketed service spawning

* Use codensity spawner

* Pass service map to liveness check

* Reimplement timeout using Async

* Use a static service map

Since service ports are allocated statically, there is no point anymore
in dynamically reconfiguring the environment when a new backend is
spawned. This simplifies the logic dramatically.

* Use traverseConcurrentlyCodensity

* Cleanup and fix warnings

* Add CHANGELOG entry

* Minor cleanups

---------

Co-authored-by: Magnus Viernickel <[email protected]>

* Use ElasticMQ instead of fake_sqs (#3750)

* local-setup: Use ElasticMQ instead of fake_sqs for speed

* chrats/fake-aws-sqs: Use ElasticMQ

* CI Setup: Create SQS queues using config

Not sure why we created the script, perhaps people didn't know about existence
of this config value.

* SQSWatcher: Use smaller wait time

ElasticMQ allows max 20 seconds.

* SQSWatcher: Ensure thread being killed is flagged properly

* SQSWatcher: Use 5 concurrent loops to increase throughput

Each recieve takes 300ms. When 16 tests run in parallel, this poor thread cannot
keep up and causes timeouts. Instead of increasing the timeout increasing
threads will ensure tests don't fail.

* brig-integration: Use the queue name for SQSWatcher

Galley uses the queue name, brig was using the queue-url, this is not correct.
With the old fake-sqs implementation it still worked.

* [feat] bombon derivations (#3744)

* [feat] bombon derivations

* [feat] add script to upload bom to releases.

* [WPB 5356] fix brig flaking (#3769)

* [feat] move testKeyPacakgeUploadNoKey to integration

* [feat] move testKeyPackageClaim to new integration test suite

* [feat] testKeyPackagesSelfClaim to new integration test suite

* [feat] move testKeyPackageRemoteClaim to new integration test suite

* [chore] remove replaced brig tests and clean up

* galley-integration: Give legalhold service longer to be connectable from galley (#3776)

Earlier we gave it up to 3.1 milliseconds, now its up to 5 seconds.

* Use fork of warp which closes connections gracefully (#3775)

Upstream PR: yesodweb/wai#958

* [fix] remove dependency on experimental feature flakes (#3778)

at the request of flokli

* WPB-5312 (#3782)

* Add -U option to upload-helm-charts-s3.sh (#3784)

* [feat] nixpkgs bump (#3781)

* [feat] nixpkgs bump
* [fix] dontCheck markov-chain-usage-model because its doctests are broken
* [fix] change override of base-compat*

* [WPB-5389] Guard user connection requests by team-level federation settings (#3774)

* Define the new user connection request error

* An effect utility to check team federation

* Perform team federation checks on the calling side

* Formatting the code

* Introduce 1-1 conv test setup helpers

* Test: Migrate "Remote connections: mutual Connect - local action then remote action"

* Test: Migrate "Remote connections: mutual Connect - remote action then local action"

This test is covered by the `testConnectWithRemoteUser` test

* [feat] move testRemoteUserGetsDeleted to new integration testsuite

* Test utility to assert on connection status

* Test: Migrate "Remote connections: ignore then accept"

* Test: Migrate "Remote connections: ignore, remote cancels, then accept"

* Test: Migrate "Remote connections: block then accept"

* Test: Migrate "Remote connections: block, remote cancels, then accept"

* Test: Migrate "Remote connections: send then cancel"

* [feat] move testInternalGetConStatusesAll to new testsuite

* Include the team ID in the fed connection request

* [feat] move testConnectionLimits to new integration test suite

* Revert the generalisation of 'ensureFederatesWith'

* [fix] comment back in test that is still broken

* Test: not federating with a remote team

* Test: connection attempt under non-mutual federation

* Test: connect under allow-all mutual federation

* Test: connect under allow-dynamic mutual federation

* Test: connect under mixed federation-allow policies

* Add a changelog

* Remove an unused fed client argument in tests

* fixup! Introduce 1-1 conv test setup helpers

---------

Co-authored-by: Magnus Viernickel <[email protected]>

* federator: Do no reuse connections when talking to remotes  (#3789)

* http2-manager: Expose a function to allow single use connections

* federator: Do no reuse connections when talking to remotes

This comes with performance penalty but its required to get around this bug in
the http2 library: kazu-yamamoto/http2#102

* SQSWatcher: Ignore failures in deleting recieved messages (#3783)

* SQSWatcher: Ignore failures in deleting recieved messages

Perhaps they started getting delivered multiple times. There is code in
ElasticMQ which only allows last delivery receipt to be used for deletion.

* SQSWatcher: Better formatting for printing

* SQSWatcher: Remove unused function to fetch messages

* Use treefmt for running cabal-fmt (#3785)

* Use treefmt for running cabal-fmt

This get rid of the custom script.

* Makefile: Make lint-all fail when treefmt changes anything

* GH Actions: Remove treefmt

Concourse does this anway.

* cabal-fmt everything

* [WPB-5810] Fix the service provider endpoints that return no body (#3766)

* Fix accept header issue when resp. body is empty

* Add a changelog

* Tests: provider and service endpoints

* Test: update a service name

* Test: A provider creation helper

* [WPB-5936] Send `conversation.member-leave` events to team admins (#3790)

* Test: adapt to the requirements

* Fix a Haddock documentation reference Brig->Galley

* Send conversation.member-leave to team admins too

* Add a changelog

* WPB-4888: Implement request tracing across federation (#3765)

* TLS connections to Cassandra (#3587)

Allow the configuration of TLS-secured connections to Cassandra. TLS is used when a certificate is provided. This is either done with `--tls-ca-certificate-file` for CLI commands or the configuration attribute `cassandra.tlsCa` for services. In Helm charts, the certificate is provided as literal PEM string; either as attribute `cassandra.tlsCa` (analog to service configuration) or by a reference to a secret (`cassandra.tlsCaSecretRef`.)

k8ssandra-test-cluster now can create the needed Java KeyStores for Cassandra and a corresponding CA certificate. This certificate can be shared / synced via trust-manager to give only access to the certificate and not to other secret values (e.g. the private key.)

---------

Co-authored-by: Akshay Mankar <[email protected]>

* update diagram with recent comments

* Federation API versioning (#3762)

* Limit new MLS federation endpoints to V1

* Remove "strongly typed" Named wrapper

* Add version 0 of get-mls-clients endpoint

* Limit old MLS RPC to version 0

* Add version header to federated requests

* Propagate version header through federator

* Regenerate nix packages

* Add CHANGELOG entry

* Set latest fed API version in integration tests

* Include headers in a federator unit test

* New team feature EnforceFileDownloadLocation (#3779)

* WPB-5667: Updating integration tests to better handle comments and haddock. (#3749)

* WPB-5382 - Migrating tests from Cargohold into the new integration test suite. (#3741)

* WPB-5695 Enforce group conversation permission for external partner role (#3788)

* add optional serviceMonitor field for SFTD chart (#3770)

* Update SFTD and its nginx images used by default in the helm charts (#3768)

* fix brig's Helm template for geoip disabled (#3794)

* fix brig's Helm template for geoip disabled

* hi ci

* WPB-6001 suspend user logging (#3795)

* remove geoip (dead) code (#3792)


Co-authored-by: fisx <[email protected]>

* Update MLS section of docs in developer/reference/config-options.md (#3763)

* WPB-1436 make guest link maximum lifetime configurable (#3796)

* Migrate away from our http-client fork, use upstream. (#3736)

* Change HTTP client to a different fork branch

* migrate away from http-client fork

* Use hsopenssl for fingerprinting.

* Verify peer cert

* Bump amazonka

* Adjusted aws code

* Removed uneeded dependency

* Removed ext env from galley.

* [fix] some minor fixups

* Linted

* [chore] move the callback in 'vpCallback' to ssl-utils for reuse
- galley and brig both use the same callback, I moved it to the
  `ssl-utils` package to have it be reused

* [chore] hi github come one move your lazy ***

* Removed dead import

* hi ci

---------

Co-authored-by: Magnus Viernickel <[email protected]>

* Revert "Migrate away from our http-client fork, use upstream. (#3736)" (#3799)

This reverts commit 02a94e6.

* remove more geoip (dead) code (#3798)

* webapp: Upgrade to 2023-12-11-production.0-v0.31.17-0-1e91445 (#3803)

Beside using up-to-date versions in Helm charts is generally beneficial,
this version also provides multi-ingress support.

* Delete unused chat.py (#3804)

* WPB-6101 make feature enforceFileDownloadLocation unlockable for QA (#3805)

* make feature enforceFileDownloadLocation unlockable for QA

* changelog

* Migrate from http-client fork, use upstream. (#3801)

* WPB-6099 Bump the version of rusty-jwt-tools in wire server (#3802)

* [WPB-5883] Feature flag for a limited event fanout (#3797)

* Introduce the feature flag

This commit implements no business logic around the flag, but merely
sets up the very basics needed to use the flag.

* Document the feature flag

* Guard member deleted event fanout

* Test: Limited event fanout

This extends an existing test case that deletes a team member, but now
explicitly enabling the limited event fanout feature flag.

* Test: future-port a test from a branch from July 14, 2023

* Fix the team event fanout for the unlimited case

* Test: getting and setting the feature flag

* fix linter

* Add a changelog

* Fix more linting

* Move a test within a module

* Fix a galley-types unit test

* Fix a galley-integration test

* Make a notification push transient

* Revert the change to the billing team update notification

* Reuse a notification assertion helper

---------

Co-authored-by: Stefan Berthold <[email protected]>

* Revert brig memory setup back to 512mb (#3806)

* Revert brig memory setup back to 512mb
* Added changelog.

* fix wireapp hash (#3807)

* fix wireapp hash

* hi ci

* fix: X509 Client Identity parser (#3808)

* Add Argon2id support on top of Scrypt for password hashing (#3720)

* Add pwd verification cascading
* Added comment to default argon2id opts.
* Added test for password re-hash
* Use assertEqual

* Fix integration test collector (#3812)

* Whitespace.

* Don't let /integration/Setup.hs collect temp files.

(With patterns for emacs backup and auto-save files, but can easily be
extended.)

* WPB-6162 update x 509 verification with new client identity format (#3811)

* fix: WPB-5064 Moved namshi to ix-ai smtp image  (#3791)

Due to recent security issues, a newer version of exim4 is desired. Unfortunately, the namshi-smtp image we rely on is no longer updated. So, replace it with a more current image (ix-ai), also containing exim4.

* Various improvements around LH policy conflict detection. (#3773)

* Move integration tests from galley/lh to /integration

* Improve test coverage

* Remove optimization for corner case of self messages

* Resolve trivial FUTUREWORK

* Upload bombon bom files directly to deptrack (WPB-6142) (#3810)

This avoids cluttering our release artifact page. And, Security gets the files where they need them.

* Give underlying legal hold error instead of generic msg. (#3816)

* WPB-6012 create new API version v6 (#3815)

* [WPB-6073] cleanup haskell pins (#3814)

* [feat] removes and changes some pins, removed and changes overrides

# libraries that need investigating 
- bloodhound (immense divergence, tests don't pass)
- warp (tests don't pass)
- saml2-web-sso (tests don't pass) 
- amqp (tests don't pass)
- cql-io (tests don't pass) 
- hspec-junit-formatter (tests don't pass) 
- markov-chain-usage-model (tests don't pass) 
- openapi3 (tests don't pass) 
- quickcheck-state-machine (tests don't pass) 
- transitive-anns (tests are flaky) 
- wai-route (it has been noted to get rid of it for a while and we depend on a quite old version) 
- tasty (immense divergence) 
- there's an entire family of libraries that are made by thoralf wittner that we still have in use, commonly as a fork that may also already be years old, it doesn't seem like any of these libraries get any maintenance. Perhaps we can consider taking over maintenance for those

# removed/ updated pins
- amqp: has landed in nixpkgs
- invertible: has landed in nixpkgs
- tls: has landed in nixpkgs
- hoogle/ ghc-source-gen: directly from hackage, has not landed in nixpkgs
- polysemy: newer version on hackage
- hpack: landed in nixpkgs
- hsopenssl: newer version on hackage 
- http2: newer version on hackage 
- network-conduit-tls: landed in nixpkgs
- warp-tls: landed in nixpkgs

# removed overrides 
- kind-generics-th
- http-client
- hsaml2 
- crypton-connection
- transitive-anns
- wai-predicates
- wai-middleware-prometheus
- type-errors
- text-short
- text-icu-translit
- singletons-base
- singletons-th
- servant
- servant-client
- servant-client-core
- servant-foreign
- servant-multipart 
- servant-swagger-ui-core
- servant-swagger-ui 
- polysemy
- polysemy-plugin
- polysemy-check
- monoidal-containers
- invertible
- hashtables
- ghc-source-gen

# remaining pins and their current state
- transitive-anns: we maintain this library by ourselves 
- cryptobox-haskell: we maintain this library by ourselves
- saml2-web-sso: we maintain this library ourselves 
- bloodhound: has diverged wildly, should probably be rebased on upstream and/or merged to it, see [WPB-6168: bloodhound - switch to upstream Todo](https://wearezeta.atlassian.net/browse/WPB-6168) 
- HaskellNet-SSL: PR open, upstream seems abandoned
- hsaml2: actively maintained, should probably  be upstreamed
- hspec-wai: PR open, upstream seems abandoned 
- cql: PR open, upstream seems abandoned, maintainer (thoralf wittner) searches for other maintainers 
- cql-io: PR open, upstream seems abandoned, maintainer (thoralf wittner) searches for other maintainers 
- wai-predicates: missing upstream PR, seems likely abandoned, though
- wai-routing: we use upstream but it appears abandoned as well, mr wittner doesn’t upstream anything to hackage anymore (latest update on hackage 2016, latest commit (the one we use) 2018)
- tasty: [our PR ](UnkindPartition/tasty#351 not get accepted, we should consolidate also implementing our change to HUnit as requested or think about whether we want to continue maintaining our fork which has diverged a lot
- servant-openapi3: we have a PR open and the project seems to me more or less maintained, there hasn’t been an answer from the maintainers yet, though. Not much we can do here
- postie: PR open, our PR is missing a hackage release
- tinylog: part of the thoralf wittner zoo of libraries, probably abandoned, no PR open to test it, though
- tasty-ant-xml: PR open, the maintainer is occasionally seen, so probably not abandoned. I bumped the PR 
- text-icu-translit: project seems to be abandoned
- warp: PR is somewhat recent (1 month) and project doesn’t seem to be abandoned
- wai-route: note says we should get rid of it, currently only brig and metrics-wai depend on it
- ghc-source-gen, hoogle, safe are not yet in nixpkgs but already released on hackage

# new pins 
- safe, dependency on hoogle which we now pull from hackage instead of from the upstream git repo

* Revert "Revert brig memory setup back to 512mb" (#3819)

* WPB-6177 document steps for creating new API version (#3817)

* WPB-6181 Update rusty-jwt-tools (#3820)

* WPB-6162 update X.509 verification with new client identity format test (#3813)


Co-authored-by: Stefan Berthold <[email protected]>

* increase nginz memory limit (#3821)

We should be realistic about our memory usage to not run into surprising OOMs.

* [feat] depend on a more up to date version of tasty (#3818)

- rebase our tasty fork on top of upstream
- pin our tasty fork to the fork rebased on upstream

* Revert "Revert "Revert brig memory setup back to 512mb"" (#3822)

* Disallow changing user display name, handle in mlsE2EId-enabled teams (#3827)

* Integration tests: can not change some user data in mlsE2EId teams.

* Fix: block changes in the backend.

* Fix: lie about managed_by in `GET /self`, but only there.

---------

Co-authored-by: Leif Battermann <[email protected]>

* reactivate post-quatum cipher tests (#3836)

* remove Rust dependency on local tls_codec copy (#3837)

* fix: use correct url (#3840)

* fixing grepinclude references for docs.wire.com and adjusting nix build context; updating TLS documentation (#3839)

* fixing grepinclude references for docs.wire.com; updating TLS documentation
* Update nix build strategy for docs.wire.com

Co-authored-by: jschaul <[email protected]>

* adding local build subsection in docs readme, fixing new comments in docs build nix section
* add changelog file

---------

Co-authored-by: jschaul <[email protected]>

* Improve usage of http-manager (fixes for fingerprint verification) (#3825)

* [fix] reuse manager
* [feat] bring back no reuse of the manager for
* [fix] fresh manager for each bot

* move .envrc overriding to the end of the file (#3838)

* refactor: use GitHub forks (#3841) (#3842)

Use GitHub wireapp forks for nix dependencies

Co-authored-by: Marco <[email protected]>

* Move repository from GitLab to GitHub (#3844)


Co-authored-by: Marco Conti <[email protected]>

* WPB-4657 Disabling development versions (#3772)

* redirect Makefile to dist/run-services for integration tests (#3846)

* Replace services/run-services with dist/run-services  (#3848)

* treefmt.toml: Remove run-services from excludes of shellcheck

It is not a shell script anymore

* services/start-services-only.sh: Delete

It doesn't do anything and is not referred from any documentation. It has been
"deprecated" for quite some time.

* Replace services/run-services with dist/run-services

* add test for team settings auth (#3851)

* Use http-client fork again (#3852)

* Update http-client fork to latest upstream and use it

* Revert "Improve usage of http-manager (fixes for fingerprint verification) (#3825)"

This reverts commit 38d3398.

* Revert "Migrate from http-client fork, use upstream. (#3801)"

Except for changes to amazonka things as we're still using latest
http-client (albiet forked) which requires us to upgrade amazonka.

* Give brig more RAM in integration tests (#3856)

It seems to be OOMKilled sometimes.

* add test for team properties auth (#3862)

* WPB-5845 guests should not be able to join conversations under legalhold (#3853)

* test: team settings and propertied cannot be changeds by foreign team owner (#3866)

* [feat] update documentation on how to build `wire-server` (#3854)

* [fix] use the correct API in the integration tests (#3869)

* [fix] use the correct API in the integration tests

* WPB-6351 Use max available version for internal API calls (#3863)

* Clean up LH tests (#3830)

* Use HasTests to save a few LOC.

* Fix/extend client CRUD api.

- moved internal add from API.Brig to API.BrigInternal
- created API.BrigCommon for data structured needed in both
- added public add

* Tranlate tests: manually add/delete client.

* Fiddle with test case type abstractions.

* Remove obsolete test from integration/test/Test/Demo.hs

* Update coturn default image (#3872)

Update coturn image with bugfix to its pre-stop-hook from wireapp/coturn#10 to allow coturn pods to terminate once their traffic has drained, instead of waiting for its terminationGracePeriod (up to 24 hours).

* move a comment closer to the commented line (#3868)

* Unblock release. (#3871)

* Use runAsUser, runAsGroup in webapp/teams/account helm chart (#3826)

* replace runAsNonRoot to user group and id of 1000

* add changelog

* update topology aware annotation key for k8s 1.27+ (#3878)

* update annotation key for k8s 1.27+

* add changelog

* add backward compatability

* fix Helm pretty-printer for disabledAPIVersions (#3877)


`disabledAPIVersions` is a list which Helm would print as `[item1 item2]` into
YAML, thus, corrupting the YAML format. This can be mitigated by applying the
Helm template function `toJson` (or `toYaml`) to the list in question which
would format the list as `["item1", "item2"]`. This is no issue for scalars,
since Helm's format coincidently matches the one required by YAML.

* Introduce NotificationSubsystem (#3786)

This commit introduces the concept of Subsystems. Each of these subsystems will
represent an important part of the domain concepts in the product that will
interact with other subsystems. We will use effect systems to encode these
subsystems and test them in isolation as much as possible.

This commit consolidates all the code that spoke to gundeck from brig and galley
into the NotificationSubsystem.

https://wearezeta.atlassian.net/browse/WPB-5985

---------

Co-authored-by: Magnus Viernickel <[email protected]>
Co-authored-by: Leif Battermann <[email protected]>

* chore: [charts] Update webapp version (#3824)

Co-authored-by: Zebot <[email protected]>

* Add changelog for Release 2024-02-12

---------

Co-authored-by: Arthur Wolf <[email protected]>
Co-authored-by: Igor Ranieri Elland <[email protected]>
Co-authored-by: Mango The Fourth <[email protected]>
Co-authored-by: Paolo Capriotti <[email protected]>
Co-authored-by: Leif Battermann <[email protected]>
Co-authored-by: Marko Dimjašević <[email protected]>
Co-authored-by: Zebot <[email protected]>
Co-authored-by: Stefan Matting <[email protected]>
Co-authored-by: Florian Klink <[email protected]>
Co-authored-by: Owen Harvey <[email protected]>
Co-authored-by: Akshay Mankar <[email protected]>
Co-authored-by: Stefan Berthold <[email protected]>
Co-authored-by: jschaul <[email protected]>
Co-authored-by: Sven Tennie <[email protected]>
Co-authored-by: Stefan Matting <[email protected]>
Co-authored-by: Igor Ranieri <[email protected]>
Co-authored-by: fisx <[email protected]>
Co-authored-by: Magnus Viernickel <[email protected]>
Co-authored-by: rohan-wire <[email protected]>
Co-authored-by: Lisa Marie Maginnis <[email protected]>
Co-authored-by: Jan Schumacher <[email protected]>
Co-authored-by: Marco <[email protected]>
Co-authored-by: Amit Sagtani <[email protected]>
  • Loading branch information
2 parents 17afbcb + 90500af commit 97b2245
Show file tree
Hide file tree
Showing 765 changed files with 57,634 additions and 9,950 deletions.
22 changes: 17 additions & 5 deletions .envrc
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,24 @@ store_paths=$(echo "$nix_files" | xargs nix-store --add ./nix)
layout_dir=$(direnv_layout_dir)
env_dir=./.env

export NIX_CONFIG='extra-experimental-features = nix-command'

[[ -d "$layout_dir" ]] || mkdir -p "$layout_dir"

if [[ ! -d "$env_dir" || ! -f "$layout_dir/nix-rebuild" || "$store_paths" != $(< "$layout_dir/nix-rebuild" ) ]]; then
nix-build ./nix -A wireServer.devEnv --out-link ./.env

bcmd=nix
if command -v nom &> /dev/null; then
if [[ "${USE_NOM}" != "0" ]]; then
bcmd=nom
fi
fi
echo "🔧 Building environment"
$bcmd build -f nix wireServer.devEnv -Lv --out-link ./.env
echo "$store_paths" > "$layout_dir/nix-rebuild"
fi

PATH_add "./.env/bin"
PATH_add "./.env-hs-run/bin"
path_add "PKG_CONFIG_PATH" "./.env/lib/pkgconfig"
path_add "LIBRARY_PATH" "./.env/lib"
path_add "PYTHONPATH" "./hack/python"
Expand All @@ -32,9 +41,6 @@ PATH_add "./dist"
# e.g. "nix-shell -p foo" will get "foo" for pkgs defined in ./nix/default.nix
[[ -f "./.env/.profile" ]] && source_env "./.env/.profile"

# allow local .envrc overrides
[[ -f .envrc.local ]] && source_env .envrc.local

# Locale
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
Expand All @@ -51,3 +57,9 @@ export INTEGRATION_DYNAMIC_BACKENDS_POOLSIZE=3
export AWS_REGION="eu-west-1"
export AWS_ACCESS_KEY_ID="dummykey"
export AWS_SECRET_ACCESS_KEY="dummysecret"

# integration test suite timeout
export TEST_TIMEOUT_SECONDS=2

# allow local .envrc overrides
[[ -f .envrc.local ]] && source_env .envrc.local
18 changes: 0 additions & 18 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,6 @@ on:
branches: [master, develop]

jobs:
treefmt:
name: Run treefmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
submodules: true
- uses: cachix/install-nix-action@v20
- uses: cachix/cachix-action@v12
with:
name: wire-server
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: Install treefmt and nixpkgs-fmt (without pulling in all of dev-env)
run: nix-env -if nix/default.nix -iA pkgs.treefmt pkgs.nixpkgs-fmt pkgs.shellcheck
- name: Run treefmt
run: treefmt

build-docs:
name: Build docs
runs-on: ubuntu-latest
Expand Down
23 changes: 4 additions & 19 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,25 +40,6 @@ services/nginz/src
services/.env
tools/api-simulations/mailboxes.json
tools/api-simulations/reports
brig-schema.yaml
brig.yaml
brig.integration-aws.yaml
cannon.yaml
cannon.integration-aws.yaml
cannon2.integration-aws.yaml
cargohold.yaml
cargohold.integration-aws.yaml
galley-schema.yaml
galley.yaml
galley.integration-aws.yaml
gundeck-schema.yaml
gundeck.yaml
gundeck.integration-aws.yaml
proxy.yaml
proxy.integration-aws.yaml
spar.yaml
spar.integration-aws.yaml
integration-aws.yaml
DOCKER_ID*
swagger-ui

Expand Down Expand Up @@ -105,6 +86,7 @@ result-*
# Created by .envrc
.direnv
.env
.env-hs-run

# emacs misc
.dir-locals.el
Expand All @@ -114,3 +96,6 @@ result-*

services/nginz/third_party/headers-more-nginx-module
services/nginz/third_party/nginx-module-vts

# dumped out by running tests in kind
logs-integration
Loading

0 comments on commit 97b2245

Please sign in to comment.