Skip to content

Commit

Permalink
Implement token-exchange to get scoped token for MinIO (#41 #10 #23 #3).
Browse files Browse the repository at this point in the history
  • Loading branch information
chenkins committed Jan 10, 2024
1 parent d108df4 commit e3287f1
Show file tree
Hide file tree
Showing 11 changed files with 198 additions and 143 deletions.
75 changes: 39 additions & 36 deletions backend/CIPHERDUCK.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,53 +12,65 @@ Documentation
* [MinIO Client Reference `mc idp openid`](https://min.io/docs/minio/linux/reference/minio-mc/mc-idp-openid.html)
* [MinIO Security Token Service `AssumeRoleWithWebIdentity](https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html)

#### OIDC provider
```shell
export MINIO_IDENTITY_OPENID_CONFIG_URL=https://login1.staging.cryptomator.cloud/realms/cipherduck/.well-known/openid-configuration
export MINIO_IDENTITY_OPENID_CLIENT_ID=cryptomator
export MINIO_IDENTITY_OPENID_CLAIM_NAME=amr
minio server tmp_data --console-address :9001
```

TODO https://github.com/chenkins/cipherduck-hub/issues/41 with scoped token, this will become:


#### Frontend role
Add role for creating buckets with prefix `cipherduck` and uploading `vault.cryptomator`,
#### Policy and OIDC provider for MinIO
Add role for creating buckets with prefix `cipherduck` and uploading `vault.cryptomator`, as well as RW to access to buckets through `aud` claim in JWT token:
see
[createbucketpermissionpolicy.json](src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Fminio%2Fcreatebucketpermissionpolicy.json).
[cipherduck.json](src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Fminio%2Fcipherduckpolicy.json).

Side-note: MinIO does not allow for multiple OIDC providers with the same client ID:

> mc: <ERROR> Unable to add OpenID IDP config to server. Client ID XYZ is present with multiple OpenID configurations.
This is not a problem as we leave the claim specifying the vault unset or pointing to a non-existing vault.




```shell
mc alias set myminio http://127.0.0.1:9000 minioadmin minioadmin
mc admin policy create myminio cipherduck-createbucket src/main/resources/cipherduck/setup/minio/createbucketpermissionpolicy.json
mc admin policy create myminio cipherduck src/main/resources/cipherduck/setup/minio/cipherduckpolicy.json
```

Add a new OIDC provider using the policy:
```shell
mc idp openid add myminio cryptomator \
config_url="https://login1.staging.cryptomator.cloud/realms/cipherduck/.well-known/openid-configuration" \
config_url="http://keycloak:8180/realms/cryptomator/.well-known/openid-configuration" \
client_id="cryptomator" \
client_secret="ignore-me" \
role_policy="cipherduck-createbucket"
role_policy="cipherduck"
mc idp openid add myminio cryptomatorhub \
config_url="http://keycloak:8180/realms/cryptomator/.well-known/openid-configuration" \
client_id="cryptomatorhub" \
client_secret="ignore-me" \
role_policy="cipherduck"
mc admin service restart myminio
```

Or use environment variables on the default OIDC provider:
```shell
export MINIO_IDENTITY_OPENID_CONFIG_URL=https://login1.staging.cryptomator.cloud/realms/cipherduck/.well-known/openid-configuration
export MINIO_IDENTITY_OPENID_CLIENT_ID=cryptomator
export MINIO_IDENTITY_OPENID_ROLE_POLICY=cipherduck-createbucket
minio server tmp_data --console-address :9001

mc idp openid add myminio cryptomator \
config_url="https://login1.staging.cryptomator.cloud/realms/cipherduck/.well-known/openid-configuration" \
client_id="cryptomator" \
client_secret="ignore-me" \
role_policy="cipherduck"
mc idp openid add myminio cryptomatorhub \
config_url="https://login1.staging.cryptomator.cloud/realms/cipherduck/.well-known/openid-configuration" \
client_id="cryptomatorhub" \
client_secret="ignore-me" \
role_policy="cipherduck"
mc admin service restart myminio
```

Extract the policy ARN:
```shell
mc idp openid ls myminio feature/cipherduck
╭──────────────────────────────────────────────────────────────────╮
│ On? Name RoleARN │
│ 🟢 (default) arn:minio:iam:::role/IqZpDC5ahW_DCAvZPZA4ACjEnDE │
╰──────────────────────────────────────────────────────────────────╯
╭───────────────────────────────────────────────────────────────────────╮
│ On? Name RoleARN │
│ 🔴 (default) │
│ 🟢 cryptomator arn:minio:iam:::role/IqZpDC5ahW_DCAvZPZA4ACjEnDE │
│ 🟢 cryptomatorhub arn:minio:iam:::role/HGKdlY4eFFsXVvJmwlMYMhmbnDE │
╰───────────────────────────────────────────────────────────────────────╯


mc idp openid info myminio feature/cipherduck
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
Expand All @@ -70,15 +82,6 @@ mc idp openid info myminio
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

TODO https://github.com/chenkins/cipherduck-hub/issues/41 add scopedvaultrole - probably we need to include in the same policy as MinIO
uses issuer and client id to decide which policy to evaluate:
> mc: <ERROR> Unable to add OpenID IDP config to server. Client ID cryptomator is present with multiple OpenID configurations.
This is not a problem if we can leave the claim specifying the vault unset or pointing to a non-existing vault.

###
TODO https://github.com/chenkins/cipherduck-hub/issues/41
[scopedvaultaccesspermissionpolicy.json](src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Fminio%2Fscopedvaultaccesspermissionpolicy.json)

### Hub configuration

Expand Down Expand Up @@ -133,7 +136,7 @@ Add role for creating buckets with prefix `cipherduck` and uploading `vault.cryp
see
[createbucketpermissionpolicy.json](src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Faws%2Fcreatebucketpermissionpolicy.json)
and
[createbuckettrustpolicy.json](src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Faws%2Fcreatebuckettrustpolicy.json).
[createbuckettrustpolicy.json](./src%2Fmain%2Fresources%2Fcipherduck%2Fsetup%2Faws%2Fcreatebuckettrustpolicy.json).

```shell
aws iam create-role --role-name cipherduck-createbucket --assume-role-policy-document file://src/main/resources/cipherduck/setup/createbuckettrustpolicy.json
Expand Down
15 changes: 13 additions & 2 deletions backend/src/main/java/org/cryptomator/hub/api/VaultResource.java
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import org.cryptomator.hub.validation.OnlyBase64Chars;
import org.cryptomator.hub.validation.ValidId;
import org.cryptomator.hub.validation.ValidJWS;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.eclipse.microprofile.jwt.JsonWebToken;
import org.eclipse.microprofile.openapi.annotations.Operation;
import org.eclipse.microprofile.openapi.annotations.enums.ParameterIn;
Expand Down Expand Up @@ -88,6 +89,16 @@ public class VaultResource {
@Inject
LicenseHolder license;

// / start cipherduck extension
@Inject
@ConfigProperty(name = "quarkus.oidc.client-id", defaultValue = "")
String keycloakClientIdHub;

@Inject
@ConfigProperty(name = "hub.keycloak.oidc.cryptomator-client-id", defaultValue = "")
String keycloakClientIdCryptomator;
// \ end cipherduck extension

@GET
@Path("/accessible")
@RolesAllowed("user")
Expand Down Expand Up @@ -175,7 +186,7 @@ public Response addUser(@PathParam("vaultId") UUID vaultId, @PathParam("userId")
}

// / start cipherduck extension
keycloakGrantAccessToVault(syncerConfig, vaultId.toString(), userId);
keycloakGrantAccessToVault(syncerConfig, vaultId.toString(), userId, List.of(keycloakClientIdHub, keycloakClientIdCryptomator));
// \ end cipherduck extension

return addAuthority(vault, user, role);
Expand Down Expand Up @@ -365,7 +376,7 @@ public Response grantAccess(@PathParam("vaultId") UUID vaultId, @NotEmpty Map<St

// / start cipherduck extension
// TODO check remove upon DELETE operations?
keycloakGrantAccessToVault(syncerConfig, vaultId.toString(), userId);
keycloakGrantAccessToVault(syncerConfig, vaultId.toString(), userId, List.of(keycloakClientIdHub, keycloakClientIdCryptomator));
// \ end cipherduck extension

AuditEventVaultAccessGrant.log(jwt.getSubject(), vaultId, userId);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ public interface StorageConfig {
@JsonProperty("bucketPrefix")
String bucketPrefix();

// TODO obsolete?
@JsonProperty("oidcProvider")
Optional<String> oidcProvider();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ public class StorageResource {
@Transactional
@Operation(summary = "creates bucket and policy", description = "creates an S3 bucket and uploads policy for it.")
@APIResponse(responseCode = "200", description = "uploaded storage configuration")
@APIResponse(responseCode = "400", description = "frischfrommfrei")
@APIResponse(responseCode = "400", description = "frischfrommfroehlichfrei")
public Response createBucket(StorageDto dto) {

Map<String, StorageConfig> storageConfigs = backendsConfig.backends().stream().collect(Collectors.toMap(StorageConfig::id, Function.identity()));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,6 @@ public record VaultJweBackendDto(
Optional<String> hostname,
Optional<String> scheme,
Optional<Integer> port,
// String defaultPath,
// String username,
// String password,
Optional<String> region,
Optional<String> stsEndpoint,
Optional<String> stsRoleArn,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,43 +1,120 @@
package org.cryptomator.hub.cipherduck;

import jakarta.ws.rs.core.Response;
import org.cryptomator.hub.SyncerConfig;
import org.jboss.logging.Logger;
import org.keycloak.admin.client.Keycloak;
import org.keycloak.admin.client.resource.RealmResource;
import org.keycloak.admin.client.resource.UserResource;
import org.keycloak.representations.idm.ClientRepresentation;
import org.keycloak.representations.idm.ClientScopeRepresentation;
import org.keycloak.representations.idm.ProtocolMapperRepresentation;
import org.keycloak.representations.idm.RoleRepresentation;
import org.keycloak.representations.idm.UserRepresentation;

import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class KeycloakGrantAccessToVault {
private static final Logger LOG = Logger.getLogger(KeycloakGrantAccessToVault.class);

public static void keycloakGrantAccessToVault(final SyncerConfig syncerConfig, final String vaultId, final String userId) {
LOG.info("keycloakURL=" + syncerConfig.getKeycloakUrl());
// N.B. quarkus has no means to provide empty string as value, interpreted as no value, see https://github.com/quarkusio/quarkus/issues/2765
// TODO better solution than using sentinel string "empty"?
if ("empty".equals(syncerConfig.getKeycloakUrl())) {
LOG.error(String.format("Could not grant access to vault %s for user %s as keycloak URL is not defined.", vaultId, userId));
return;
}
try (final Keycloak keycloak = Keycloak.getInstance(syncerConfig.getKeycloakUrl(), syncerConfig.getKeycloakRealm(), syncerConfig.getUsername(), syncerConfig.getPassword(), syncerConfig.getKeycloakClientId())) {

final RealmResource realm = keycloak.realm(syncerConfig.getKeycloakRealm());
final UserResource userResource = realm.users().get(userId);
final UserRepresentation ur = userResource.toRepresentation();

// TODO https://github.com/chenkins/cipherduck-hub/issues/4 do we want to use user attributes? Or should we use groups/....? What happens in federation setting - do attributes come from AD etc. and will there be no conflict?
Map<String, List<String>> attributes = ur.getAttributes();
if (attributes == null) {
attributes = new HashMap<>();
}
attributes.put("vault", Stream.concat(attributes.getOrDefault("vault", Collections.EMPTY_LIST).stream(), Stream.of(vaultId)).toList());
ur.setAttributes(attributes);
userResource.update(ur);
}
}
private static final Logger LOG = Logger.getLogger(KeycloakGrantAccessToVault.class);


public static void keycloakGrantAccessToVault(final SyncerConfig syncerConfig, final String vaultId, final String userId, final List<String> clientIds) {
LOG.info("keycloakURL=" + syncerConfig.getKeycloakUrl());
// N.B. quarkus has no means to provide empty string as value, interpreted as no value, see https://github.com/quarkusio/quarkus/issues/2765
// TODO better solution than using sentinel string "empty"?
if ("empty".equals(syncerConfig.getKeycloakUrl())) {
LOG.error(String.format("Could not grant access to vault %s for user %s as keycloak URL is not defined.", vaultId, userId));
return;
}
try (final Keycloak keycloak = Keycloak.getInstance(syncerConfig.getKeycloakUrl(), syncerConfig.getKeycloakRealm(), syncerConfig.getUsername(), syncerConfig.getPassword(), syncerConfig.getKeycloakClientId())) {

// https://www.keycloak.org/docs-api/21.1.1/rest-api
final RealmResource realm = keycloak.realm(syncerConfig.getKeycloakRealm());

final UserResource userResource = realm.users().get(userId);
final UserRepresentation ur = userResource.toRepresentation();

// TODO https://github.com/chenkins/cipherduck-hub/issues/4 do we want to use user attributes? Or should we use groups/....? What happens in federation setting - do attributes come from AD etc. and will there be no conflict?
Map<String, List<String>> attributes = ur.getAttributes();
if (attributes == null) {
attributes = new HashMap<>();
}
attributes.put("vault", Stream.concat(attributes.getOrDefault("vault", Collections.EMPTY_LIST).stream(), Stream.of(vaultId)).toList());
ur.setAttributes(attributes);
userResource.update(ur);


// create client scope <vaultId> (if necessary)
if (realm.clientScopes().findAll().stream().filter(clientScopeRepresentation -> clientScopeRepresentation.getId().equals(vaultId)).collect(Collectors.toList()).isEmpty()) {

ClientScopeRepresentation vaultClientScope = new ClientScopeRepresentation();
vaultClientScope.setId(vaultId);
vaultClientScope.setName(vaultId);
vaultClientScope.setDescription(String.format("Client scope for vault %s", vaultId));
vaultClientScope.setAttributes(new HashMap<>());
vaultClientScope.setProtocol("openid-connect");


ProtocolMapperRepresentation protocolMapper = new ProtocolMapperRepresentation();
protocolMapper.setName(String.format("Hard-coded mapper for vault %s", vaultId));
protocolMapper.setProtocolMapper("oidc-hardcoded-claim-mapper");
protocolMapper.setProtocol("openid-connect");

Map<String, String> config = new HashMap<>();
config.put("jsonType.label", "String");

// TODO https://github.com/chenkins/cipherduck-hub/issues/41 do we need only access token?
config.put("userinfo.token.claim", "true");
config.put("id.token.claim", "true");
config.put("access.token.claim", "true");
config.put("access.tokenResponse.claim", "false");

// TODO https://github.com/chenkins/cipherduck-hub/issues/41 support for AWS etc. - we need storage type info, however, we currently do not have it in access grant - should we make it a separate REST endpoint in StorageResource?
config.put("claim.name", "aud");
config.put("claim.value", vaultId);

protocolMapper.setConfig(config);
vaultClientScope.setProtocolMappers(Collections.singletonList(protocolMapper));

Response response = realm.clientScopes().create(vaultClientScope);
if (response.getStatus() != 201) {
throw new RuntimeException(String.format("Failed to create client for vault %s. %s", vaultId, response.getStatusInfo().getReasonPhrase()));
}
}

// add client scope to "cryptomator" and cryptomatorhub" clients
for (String clientId : clientIds) {
List<ClientRepresentation> byClientId = realm.clients().findByClientId(clientId);
if (byClientId.size() != 1) {
throw new RuntimeException(String.format("There are %s clients with clientId %s, expected to found exactly one.", byClientId.size(), clientId));
}
ClientRepresentation cryptomatorClient = byClientId.get(0);
realm.clients().get(cryptomatorClient.getId()).addOptionalClientScope(vaultId);
}


// create role <vaultId> (if necessary)
if (realm.roles().list().stream().filter(role -> role.getName().equals(vaultId)).collect(Collectors.toList()).isEmpty()) {
RoleRepresentation vaultRole = new RoleRepresentation();
vaultRole.setName(vaultId);
vaultRole.setDescription(String.format("Role for vault %s", vaultId));
realm.roles().create(vaultRole);
}

// scope the client scope to the vault role
realm.clientScopes().get(vaultId).getScopeMappings().realmLevel().add(Collections.singletonList(realm.roles().get(vaultId).toRepresentation()));

// add role to user
realm.users().get(userId).roles().realmLevel().add(Collections.singletonList(realm.roles().get(vaultId).toRepresentation()));


// TODO https://github.com/chenkins/cipherduck-hub/issues/41 which permissions required for syncer -> minimal set
// TODO https://github.com/chenkins/cipherduck-hub/issues/41 proof of access to vault in grantAccess()?
}
}
}
9 changes: 9 additions & 0 deletions backend/src/main/resources/application.properties
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ hub.keycloak.oidc.cryptomator-client-id=cryptomator
%dev.quarkus.keycloak.devservices.port=8180
%dev.quarkus.keycloak.devservices.service-name=quarkus-cryptomator-hub
%dev.quarkus.keycloak.devservices.image-name=ghcr.io/cryptomator/keycloak:22.0.5
# TODO https://github.com/chenkins/cipherduck-hub/issues/41 add to Dockerfile as well?
# https://github.com/quarkusio/quarkus/blob/596d9ae7a76cf529d24594a82b7c540030799dac/extensions/oidc/deployment/src/main/java/io/quarkus/oidc/deployment/devservices/keycloak/DevServicesConfig.java#L30
# https://github.com/quarkusio/quarkus/blob/main/extensions/oidc/deployment/src/main/java/io/quarkus/oidc/deployment/devservices/keycloak/KeycloakDevServicesProcessor.java#L110
# https://github.com/cryptomator/cryptomator.github.io/blob/52ac36a1db04ce1aa6db41f0aeb0a0f2b76b68b5/assets/js/hubsetup.js#L112
# https://github.com/keycloak/keycloak/issues/20582
%dev.quarkus.keycloak.devservices.start-command=start-dev --import-realm --features=token-exchange
%dev.quarkus.oidc.devui.grant.type=code
# OIDC will be mocked during unit tests. Use fake auth url to prevent dev services to start:
%test.quarkus.oidc.auth-server-url=http://localhost:43210/dev/null
Expand Down Expand Up @@ -82,6 +88,9 @@ quarkus.hibernate-orm.database.globally-quoted-identifiers=true
quarkus.flyway.migrate-at-start=true
quarkus.flyway.locations=classpath:org/cryptomator/hub/flyway
%dev.quarkus.flyway.ignore-missing-migrations=true
# https://quarkus.io/guides/databases-dev-services
# https://stackoverflow.com/questions/44654216/correct-way-to-install-psql-without-full-postgres-on-macos
# psql -h localhost -p 54082 -U quarkus -d quarkus

# log Hibernate SQL statements including values, for dev-purpose only
%dev.quarkus.log.min-level=TRACE
Expand Down
Loading

0 comments on commit e3287f1

Please sign in to comment.