Network Policy for Google Cloud Storage #28
Comments
jeunii
changed the title
|
That is a great question. Not certain we do, but let me reach out to someone. |
|
@chrislovecnm Thanks. I added a bit of logs too to my original post. |
|
Hi, I run in the same problem. Looks like the address for storage.googleapis.com is extremely unpredictable and volatile. Would like to see a predictable set of cidrs for this. nslookup presents a single IP, and it is different all the time. We need help for this, otherwise we cannot use buckets in security critical environments gsutil -D ls ***************************** WARNING ***************************** *** You are running gsutil with debug output enabled. *** Be aware that debug output includes authentication credentials. *** Make sure to remove the value of the Authorization header for *** each HTTP request printed to the console prior to posting to *** a public medium such as a forum post or Stack Overflow. ***************************** WARNING ***************************** gsutil version: 4.44 checksum: e2af2d34cc25b33a166680f83089faee (OK) boto version: 2.49.0 python version: 2.7.16 (default, May 6 2019, 19:28:45) [GCC 8.3.0] OS: Linux 4.14.137+ multiprocessing available: False using cloud sdk: True pass cloud sdk credentials to gsutil: True config path(s): /tmp/.config/gcloud/legacy_credentials/xxxxxxxxxxxxxxxxx/.boto gsutil path: /google-cloud-sdk/bin/gsutil compiled crcmod: True installed via package manager: False editable install: False Command being run: /google-cloud-sdk/platform/gsutil/gsutil -o GSUtil:default_project_id=zz-muster -D ls config_file_list: ['/tmp/.config/gcloud/legacy_credentials/xxxxxxxxxxxxxxxxxx/.boto'] config: [('debug', '0'), ('working_dir', '/mnt/pyami'), (u'https_validate_certificates', 'true'), ('debug', '0'), ('working_dir', '/mnt/pyami'), (u'default_project_id', u'zz-muster')] DEBUG 1015 15:08:05.735010 multiprocess_file_storage.py] Read credential file INFO 1015 15:08:05.735191 client.py] access_token is expired. Now: 2019-10-15 15:08:05.735172, token_expiry: 2019-10-15 14:33:29 DEBUG 1015 15:08:05.735759 multiprocess_file_storage.py] Read credential file INFO 1015 15:08:05.736997 base_api.py] Calling method storage.buckets.list with StorageBucketsListRequest: <StorageBucketsListRequest maxResults: 1000 project: u'zz-muster' projection: ProjectionValueValuesEnum(noAcl, 1)> INFO 1015 15:08:05.738046 base_api.py] Making http GET to https://storage.googleapis.com/storage/v1/b?project=zz-muster&fields=nextPageToken%2Citems%2Fid&alt=json&projection=noAcl&maxResults=1000 INFO 1015 15:08:05.738615 base_api.py] Headers: {'accept': 'application/json', 'accept-encoding': 'gzip, deflate', 'content-length': '0', 'user-agent': 'apitools gsutil/4.44 Python/2.7.16 (linux2) google-cloud-sdk/266.0.0 analytics/disabled'} INFO 1015 15:08:05.738781 base_api.py] Body: (none) connect: (storage.googleapis.com, 443) send: 'GET /storage/v1/b?project=zz-muster&fields=nextPageToken%2Citems%2Fid&alt=json&projection=noAcl&maxResults=1000 HTTP/1.1\r\nHost: storage.googleapis.com\r\ncontent-length: 0\r\nauthorization: Bearer ya29.ImWbB2JsJbmHr5l9CMGrVmuNGU9L4gJnWS96sbuCtFPO9pwlJFc97UgkaopYX2_hbQ01hKOemgvAFJKnn8-SlbEHVz_HVpSDPEBUig0jnvOk1WF5dCDx01Af0SkzL8iwLz7uaDR4uw\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: apitools gsutil/4.44 Python/2.7.16 (linux2) google-cloud-sdk/266.0.0 analytics/disabled\r\n\r\n' reply: 'HTTP/1.1 401 Unauthorized\r\n' header: X-GUploader-UploadID: AEnB2UqkJG3pZO51HvYEZegDdXMeFPQMGFusODqDX0koCJQp_GmQqzo97q3ihXLIosJeiw6NBl4uOA139BR4UuBZN2ODcqOridYYeAuZONoGktqrzmxVPb0 header: Content-Type: application/json; charset=UTF-8 header: Date: Tue, 15 Oct 2019 15:08:05 GMT header: Vary: Origin header: Vary: X-Origin header: WWW-Authenticate: Bearer realm="https://accounts.google.com/", error=invalid_token header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate header: Expires: Mon, 01 Jan 1990 00:00:00 GMT header: Pragma: no-cache header: Content-Length: 285 header: Server: UploadServer INFO 1015 15:08:05.943315 transport.py] Refreshing due to a 401 (attempt 1/2) DEBUG 1015 15:08:05.944178 multiprocess_file_storage.py] Read credential file INFO 1015 15:08:05.944321 client.py] access_token is expired. Now: 2019-10-15 15:08:05.944308, token_expiry: 2019-10-15 14:33:29 DEBUG 1015 15:08:05.944707 multiprocess_file_storage.py] Read credential file INFO 1015 15:08:05.944816 reauth_creds.py] Refreshing access_token connect: (oauth2.googleapis.com, 443) send: u'POST /token HTTP/1.1\r\nHost: oauth2.googleapis.com\r\nContent-Length: 174\r\ncontent-type: application/x-www-form-urlencoded\r\naccept-encoding: gzip, deflate\r\nuser-agent: Python-httplib2/0.11.3 (gzip)\r\n\r\nclient_secret=ZmssLNjJy2998hD4CTg2ejr2&grant_type=refresh_token&refresh_token=1%2FxLAid1pO_W2MrKG_kRoxq-Q0sAlcM8TmiRTHuaa3tJg&client_id=32555940559.apps.googleusercontent.com' reply: 'HTTP/1.1 200 OK\r\n' header: Content-Type: application/json; charset=utf-8 header: Vary: Origin header: Vary: X-Origin header: Vary: Referer header: Content-Encoding: gzip header: Date: Tue, 15 Oct 2019 15:08:06 GMT header: Server: scaffolding on HTTPServer2 header: Cache-Control: private header: X-XSS-Protection: 0 header: X-Frame-Options: SAMEORIGIN header: X-Content-Type-Options: nosniff header: Transfer-Encoding: chunked DEBUG 1015 15:08:06.164385 multiprocess_file_storage.py] Read credential file DEBUG 1015 15:08:06.164941 multiprocess_file_storage.py] Wrote credential file /tmp/.gsutil/credstore2. send: 'GET /storage/v1/b?project=zz-muster&fields=nextPageToken%2Citems%2Fid&alt=json&projection=noAcl&maxResults=1000 HTTP/1.1\r\nHost: storage.googleapis.com\r\ncontent-length: 0\r\nauthorization: Bearer ya29.ImWbB-OmTvbo6W0_xtzW9GRJcsTiHxVNdFzoLAagWpUdREGfEBe-UUwl2M5u_5WzNYIQl5NnzhhOeZUkSxHXOhzx5OsbFX_MarSj9Rn8lT6YhCHLwWT5lg-38MIqLRpIDdisgeKIYw\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: apitools gsutil/4.44 Python/2.7.16 (linux2) google-cloud-sdk/266.0.0 analytics/disabled\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-GUploader-UploadID: AEnB2UpbgTsTeSKw-ayr8DAghbX214RHwNrU61XTbwfvyOC1nuUhOiNSXbTaEmU6eMiFS-pKdbIviWxrt4c3k5ftg3_2ZvTSmWkblgyAnzCgsE0wabkl2M0 header: Content-Type: application/json; charset=UTF-8 header: Date: Tue, 15 Oct 2019 15:08:06 GMT header: Vary: Origin header: Vary: X-Origin header: Cache-Control: private, max-age=0, must-revalidate, no-transform header: Expires: Tue, 15 Oct 2019 15:08:06 GMT header: Content-Length: 64 header: Server: UploadServer INFO 1015 15:08:06.419345 base_api.py] Response of type Buckets: <Buckets items: [<Bucket acl: [] cors: [] defaultObjectAcl: [] id: u'test_bucket_spon'>]> gs://test_bucket_spon/ bash-5.0$ nslookup storage.googleapis.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.googleapis.com Address 1: 209.85.234.128 jn-in-f128.1e100.net Address 2: 2607:f8b0:4001:c12::80 bash-5.0$ gsutil -D ls ***************************** WARNING ***************************** *** You are running gsutil with debug output enabled. *** Be aware that debug output includes authentication credentials. *** Make sure to remove the value of the Authorization header for *** each HTTP request printed to the console prior to posting to *** a public medium such as a forum post or Stack Overflow. ***************************** WARNING ***************************** gsutil version: 4.44 checksum: e2af2d34cc25b33a166680f83089faee (OK) boto version: 2.49.0 python version: 2.7.16 (default, May 6 2019, 19:28:45) [GCC 8.3.0] OS: Linux 4.14.137+ multiprocessing available: False using cloud sdk: True pass cloud sdk credentials to gsutil: True config path(s): /tmp/.config/gcloud/legacy_credentials/xxxxxxxxxx/.boto gsutil path: /google-cloud-sdk/bin/gsutil compiled crcmod: True installed via package manager: False editable install: False Command being run: /google-cloud-sdk/platform/gsutil/gsutil -o GSUtil:default_project_id=zz-muster -D ls config_file_list: ['/tmp/.config/gcloud/legacy_credentials/xxxxxxxxxxxxx/.boto'] config: [('debug', '0'), ('working_dir', '/mnt/pyami'), (u'https_validate_certificates', 'true'), ('debug', '0'), ('working_dir', '/mnt/pyami'), (u'default_project_id', u'zz-muster')] DEBUG 1015 15:09:40.532640 multiprocess_file_storage.py] Read credential file INFO 1015 15:09:40.534110 base_api.py] Calling method storage.buckets.list with StorageBucketsListRequest: <StorageBucketsListRequest maxResults: 1000 project: u'zz-muster' projection: ProjectionValueValuesEnum(noAcl, 1)> INFO 1015 15:09:40.535085 base_api.py] Making http GET to https://storage.googleapis.com/storage/v1/b?project=zz-muster&fields=nextPageToken%2Citems%2Fid&alt=json&projection=noAcl&maxResults=1000 INFO 1015 15:09:40.535679 base_api.py] Headers: {'accept': 'application/json', 'accept-encoding': 'gzip, deflate', 'content-length': '0', 'user-agent': 'apitools gsutil/4.44 Python/2.7.16 (linux2) google-cloud-sdk/266.0.0 analytics/disabled'} INFO 1015 15:09:40.535859 base_api.py] Body: (none) connect: (storage.googleapis.com, 443) send: 'GET /storage/v1/b?project=zz-muster&fields=nextPageToken%2Citems%2Fid&alt=json&projection=noAcl&maxResults=1000 HTTP/1.1\r\nHost: storage.googleapis.com\r\ncontent-length: 0\r\nauthorization: Bearer ya29.ImWbB-OmTvbo6W0_xtzW9GRJcsTiHxVNdFzoLAagWpUdREGfEBe-UUwl2M5u_5WzNYIQl5NnzhhOeZUkSxHXOhzx5OsbFX_MarSj9Rn8lT6YhCHLwWT5lg-38MIqLRpIDdisgeKIYw\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nuser-agent: apitools gsutil/4.44 Python/2.7.16 (linux2) google-cloud-sdk/266.0.0 analytics/disabled\r\n\r\n' reply: 'HTTP/1.1 200 OK\r\n' header: X-GUploader-UploadID: AEnB2UrjQ570DSiLuf8Ozl4_sI8CW0xgxoTrWVpaIhAHrTtjRd_8byViAkRVEBzE4rgtdvla-BiRjapDxR4hZKCgrAQ3Kzr9ZoRpC7oLswNATif_bkcEexE header: Vary: Origin header: Vary: X-Origin header: Content-Type: application/json; charset=UTF-8 header: Expires: Tue, 15 Oct 2019 15:09:41 GMT header: Date: Tue, 15 Oct 2019 15:09:41 GMT header: Cache-Control: private, max-age=0, must-revalidate, no-transform header: Content-Length: 55 header: Server: UploadServer INFO 1015 15:09:41.017758 base_api.py] Response of type Buckets: <Buckets items: [<Bucket acl: [] cors: [] defaultObjectAcl: [] id: u'test_bucket_spon'>]> gs://test_bucket_spon/ bash-5.0$ nslookup storage.googleapis.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.googleapis.com Address 1: 209.85.234.128 jn-in-f128.1e100.net Address 2: 2607:f8b0:4001:c12::80 bash-5.0$ nslookup storage.googleapis.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.googleapis.com Address 1: 172.217.212.128 Address 2: 2607:f8b0:4001:c03::80 ie-in-x80.1e100.net bash-5.0$ nslookup storage.l.googleusercontent.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.l.googleusercontent.com Address 1: 209.85.234.128 jn-in-f128.1e100.net Address 2: 2607:f8b0:4001:c1c::80 iz-in-x80.1e100.net bash-5.0$ nslookup storage.l.googleusercontent.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.l.googleusercontent.com Address 1: 209.85.234.128 jn-in-f128.1e100.net Address 2: 2607:f8b0:4001:c1c::80 iz-in-x80.1e100.net bash-5.0$ nslookup storage.l.googleusercontent.com nslookup: can't resolve '(null)': Name does not resolve Name: storage.l.googleusercontent.com Address 1: 172.217.214.128 Address 2: 2607:f8b0:4001:c07::80 bash-5.0$ ------------- networkpolicy --------------------- kind: NetworkPolicy apiVersion: extensions/v1beta1 metadata: generation: 1 labels: name: gcloud namespace: dev-zz-muster spec: podSelector: policyTypes:
egress:
|
|
Here is an explanation of what IP ranges / hostnames / Ports should be added to network policy to whitelist Google Cloud Storage: |
Hello
This is more of a general question. Would you have an example of a Network Policy that allows access to Storage buckets in the same Project ? I have a GKE cluster running that hosts an app that needs access to the storage buckets. The current Network Policy is very restrictive and I cannot access the buckets at all unless I take down the Network Policy. Would you know how I can find the relevant Url OR IPs/Port so that I can whitelist them ?
EDIT:
With my policy in place, if I login to my Pod and run gsutil ls, I get
But the moment I take down the Network Policy, I can see all my buckets.
Now Im unsure of how can I specify the Network Policy rules. Meaning that Im unaware of what IP ranges / hostnames / Ports should I add to my network policy to whitelist Google Cloud Storage.
The text was updated successfully, but these errors were encountered: