Draft: TUF Implementation

[SniperSister]

Summary of Changes

This pull requests changes the way how Joomla retrieves update information for Joomla core.

So far, the information about available updates has been retrieved using an XML file hosted on the Joomla.org CDN. Whatever information was written in that XML file was trusted and there was no way for a Joomla installation of that update XML actually is a legit file distributed by the project.

This makes the project vulnerable to supply chain attacks, where an attacker, once he gains access to the update XML file, might be able to distribute malicious update packages. The already implemented security measure of package hashes is no proper mitigation for that scenario as the package URL und the package hashes are stored in the same XML.

In order to succesfully mitigate such attacks, we would like to use "The Update Framework" (short "TUF") to the Joomla core updater. We are not going to introduce the general concepts of TUF in this PR as it's very extensively documented at https://theupdateframework.io/

The main changes in this PR are:

• Inclusion of the PHP-TUF client
• Inclusion of new library classes to connect the TUF client with the CMS
• Addition of a new service provider for the HTTP Factory, which allows us to mock it as a dependency in our unit tests
• Various changes to the existing Update and Updater classes to add TUF repos as a potential update source next to the existing XML mechanism
• An additional check in com_joomlaupdate to verify that the package version that shall be installed is actually the package version that users confirmed to install - that fixes an existing bug, where the re-retrieval of update information before the package download might cause a different version to be installed than the version that user saw on the update information page

Testing Instructions

Scenario 1: successful retrieval of a legit core update via TUF

• Apply the patch
• Execute the DB changes by navigating to System > Maintenance > Database and hit "Update Structure"
• Execute the "valid production metadata" query from the test queries section below using a DB client of your choice
• Navigate to System > Update > Joomla, fetch the available updates
• Expected Result: Fetching succeeds, no update being offered

Scenario 2: blocked retrieval of a malicious core update via TUF

• Apply the patch
• Execute the DB changes by navigating to System > Maintenance > Database and hit "Update Structure"
• Execute the "invalid test metadata" query from the test queries section below using a DB client of your choice
• Navigate to System > Update > Joomla, fetch the available updates
• Expected Result: The message "Update not possible because the offered update does not have enough signatures" is shown.

Scenario 3: successful retrieval of a core update via a custom XML server

• Apply the patch
• Execute the DB changes by navigating to System > Maintenance > Database and hit "Update Structure"
• Execute the "valid production metadata" query from the test queries section below using a DB client of your choice
• Navigate to System > Update > Joomla, hit "Options"
• Set the Update Channel to "Custom" and use https://update.joomla.org/core/sts/list_sts.xml as an update URL
• Hit save
• Fetch the available updates
• Expected Result: "No update available" is shown

Scenario 4: successful retrieval of an extension update via the existing XML mechanisms

• Apply the patch
• Execute the DB changes by navigating to System > Maintenance > Database and hit "Update Structure"
• Execute the "valid production metadata" query from the test queries section below using a DB client of your choice
• Install an outdated version of an extension of choice
• Fetch and install the extension update

Scenario 5: successful installation of a core update

• Apply the patch
• Execute the DB changes by navigating to System > Maintenance > Database and hit "Update Structure"
• Execute the "valid production metadata" query from the test queries section below using a DB client of your choice
• Open the file administrator/components/com_joomlaupdate/src/Model/UpdateModel.php and change line 119 from $updateURL = 'https://update.joomla.org/cms/'; to $updateURL = 'https://update.joomla.org/alpha/';
• Navigate to System > Update > Joomla, fetch the available updates
• Fetching succeeds, update to 5.1.1 being offered
• Update to 5.1.99 can be performed

Test Queries

Valid production metadata

UPDATE `#__tuf_metadata` SET `root` = '{"signed":{"_type":"root","spec_version":"1.0","version":2,"expires":"2025-03-02T11:22:17Z","keys":{"07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"9b2af2d9b9727227735253d795bd27ea8f0e294a5f3603e822dc5052b44802b9"}},"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"a18e5ebabc19d5d5984b601a292ece61ba3662ab2d071dc520da5bd4f8948799"}},"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"cb0a7a131961a20edea051d6dc2b091fb650bd399bd8514adb67b3c60db9f8f9"}},"31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"589d029a68b470deff1ca16dbf3eea6b5b3fcba0ae7bb52c468abc7fb058b2a2"}},"9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"6043c8bacc76ac5c9750f45454dd865c6ca1fc57d69e14cc192cfd420f6a66a9"}}},"roles":{"root":{"keyids":["1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"snapshot":{"keyids":["07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"targets":{"keyids":["31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3"],"threshold":1},"timestamp":{"keyids":["9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc"],"threshold":1}},"consistent_snapshot":true},"signatures":[{"keyid":"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e","sig":"2a225a560ec0837b721d4c5e379fedbd3c7c9079a94e6b31e47e0184c8b95421b6036b4286c5d90f29ab4c468d79a712fdb65e96511394ceb3aa8e2b3983a501"},{"keyid":"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","sig":"8ce0b2a7bdc1e6dcba12081f440510df0a593c072dcf591631c2dd0f456844a7da63be8e8ac31ffbddf42641fde84dc733a336031d182c2163b4c1eaf2117005"}]}', `targets` = NULL, `snapshot` = null, `timestamp` = null;

Invalid test metadata

UPDATE `#__tuf_metadata` SET `root` = '{"signed":{"_type":"root","spec_version":"1.0","version":2,"expires":"2025-03-02T11:22:17Z","keys":{"07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"9b2af2d9b9727227735253d795bd27ea8f0e294a5f3603e822dc5052b44802b9"}},"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"a18e5ebabc19d5d5984b601a292ece61ba3662ab2d071dc520da5bd4f8948799"}},"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"cb0a7a131961a20edea051d6dc2b091fb650bd399bd8514adb67b3c60db9f8f9"}},"31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"589d029a68b470deff1ca16dbf3eea6b5b3fcba0ae7bb52c468abc7fb058b2a2"}},"9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"6043c8bacc76ac5c9750f45454dd865c6ca1fc57d69e14cc192cfd420f6a66a9"}}},"roles":{"root":{"keyids":["1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"snapshot":{"keyids":["07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"targets":{"keyids":["31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3"],"threshold":1},"timestamp":{"keyids":["9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc"],"threshold":1}},"consistent_snapshot":true},"signatures":[{"keyid":"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e","sig":"2a225a560ec0837b721d4c5e379fedbd3c7c9079a94e6b31e47e0184c8b95421b6036b4286c5d90f29ab4c468d79a712fdb6wee96511394ceb3aa8e2b3983a501"},{"keyid":"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","sig":"8ce0b2a7bdc1e6dcba12081f440510df0a593c072dcf591631c2dd0f456844a7da63be8e8ac31ffbddf42641fde84dc733a33f031d182c2163b4c1eaf2117005"}]}', `targets` = NULL, `snapshot` = null, `timestamp` = null;

Link to documentations

Please select:

• No documentation changes for docs.joomla.org needed
• No documentation changes for manual.joomla.org needed
• Link to internal documentation: https://internal.joomla.org/docs/production/update/infrastructure/expired-metadata

Kudos

This is not my personal work, a ton of people helped creating this feature and I would like to thank Harald, Benjamin, Niels, Martina, Hannes, Magnus, Tobias, Franciska, Timo, Stefan and Elias for their time and contributions!

[SniperSister]

@todo change root data

[SniperSister]

@todo change root data

[SniperSister]

See joomla/joomla-cms#42799