Invalid request (Google::Apis::ClientError) 404 #6103

shiratsu · 2021-08-13T01:31:17Z

hi.

we are using this api for getting spreadsheet datas.

Until 2 days ago
we could get data from spreadsheet by using this api
but since yesterday i can't.

then to investigate this problem,
i put some debug log to http_command.rb#check_status at my local.

it says

---check_status----
10:07:20 web.1           | D, [2021-08-13T10:07:20.929125 #71084] DEBUG -- : 404
10:07:20 web.1           | D, [2021-08-13T10:07:20.929163 #71084] DEBUG -- : #<HTTP::Message::Headers:0x00007ff3d5c60360 @http_version="1.1", @body_size=0, @chunked=false, @request_method="GET", @request_uri=#<Addressable::URI:0x3ff9eae30e1c URI:https://spreadsheets.google.com/feeds/worksheets/1IQiTTQGp2mJnBC_12IFWzfluDIVbO61E57ZifPBZcMA/private/full?>, @request_query=nil, @request_absolute_uri=nil, @status_code=404, @reason_phrase="Not Found", @body_type=nil, @body_charset=nil, @body_date=nil, @body_encoding=#<Encoding:UTF-8>, @is_request=false, @header_item=[["Cache-Control", "no-cache, no-store, max-age=0, must-revalidate"], ["Pragma", "no-cache"], ["Expires", "Mon, 01 Jan 1990 00:00:00 GMT"], ["Date", "Fri, 13 Aug 2021 01:07:20 GMT"], ["Content-Type", "text/html; charset=utf-8"], ["Content-Encoding", "gzip"], ["P3P", "CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\""], ["x-chromium-appcache-fallback-override", "disallow-fallback"], ["X-Content-Type-Options", "nosniff"], ["X-XSS-Protection", "1; mode=block"], ["Server", "GSE"], ["Set-Cookie", "NID=221=rqH2hzFR3iFWy-QWMGVoa8dH7hjJzmg2N5ulckb26slyJbGJL2c7ftBeWYC3iUv7Wa6w0xhVO-pmZHGhkLPeOWgZo9PcoBi1z-qr71ecMIwCOyHoJgCbnx1iJphjJsfgwNGxrLHaQ58p4gfc260wq4KmVabCRH8idZe5Z62iLOc; expires=Sat, 12-Feb-2022 01:07:20 GMT; path=/; domain=.google.com; HttpOnly"], ["Alt-Svc", "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-T051=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""], ["Transfer-Encoding", "chunked"]], @dumped=false>
10:07:20 web.1           | D, [2021-08-13T10:07:20.929357 #71084] DEBUG -- : <!DOCTYPE html><html lang="ja"><head><meta name="description" content="ウェブ ワープロ、プレゼンテーション、スプレッドシート"><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0, user-scalable=0"><link rel="shortcut icon" href="//docs.google.com/favicon.ico"><title>ページが見つかりません</title><meta name="referrer" content="origin"><link href="//fonts.googleapis.com/css?family=Product+Sans" rel="stylesheet" type="text/css" nonce="C5LZHC1OaGxvub1TlKvZ5g"><style nonce="C5LZHC1OaGxvub1TlKvZ5g">/* Copyright 2021 Google Inc. All Rights Reserved. */

what happened?

api limitation?

The text was updated successfully, but these errors were encountered:

shiratsu · 2021-08-13T02:56:48Z

we use '0.17.2'...
it's too old?

h-z · 2021-08-13T09:30:22Z

Hi!

We have the exact same thing, on 2021-08-11 it was working fine, but it stopped working on 2021-08-12.
We are using version 0.21.2:

The results are the same, getting 404 with this error message:

Google::Apis::ClientError: Invalid request/app/vendor/bundle/ruby/2.6.0/gems/google-api-client-0.21.2/lib/google/apis/core/http_command.rb:218:in `check_status'
[...]

dazuma · 2021-08-13T15:44:09Z

Exactly which call (which class/method) is failing?

shiratsu · 2021-08-13T15:54:15Z

http_command.rb#excute_once

# Execute the command once.
        #
        # @private
        # @param [HTTPClient] client
        #   HTTP client
        # @return [Object]
        # @raise [Google::Apis::ServerError] An error occurred on the server and the request can be retried
        # @raise [Google::Apis::ClientError] The request is invalid and should not be retried without modification
        # @raise [Google::Apis::AuthorizationError] Authorization is required
        def execute_once(client)
          Rails.logger.debug("----execute_once----")
          Rails.logger.debug(client)
          Rails.logger.debug(body)
          body.rewind if body.respond_to?(:rewind)
          Rails.logger.debug("----go to next-----")
          begin
            logger.debug { sprintf('Sending HTTP %s %s', method, url) }
            request_header = header.dup
            apply_request_options(request_header)

            http_res = client.request(method.to_s.upcase,
                                      url.to_s,
                                      query: nil,
                                      body: body,
                                      header: request_header,
                                      follow_redirect: true)
            logger.debug { http_res.status }
            logger.debug { http_res.inspect }
            response = process_response(http_res.status.to_i, http_res.header, http_res.body)
            Rails.logger.debug("----response-------")
            Rails.logger.debug(response)
            success(response)
          rescue => e
            Rails.logger.debug("----Caught error----")
            Rails.logger.debug(e.message)

            logger.debug { sprintf('Caught error %s', e) }
            error(e, rethrow: true)
          end
        end

out system use old library with this(https://github.com/gimite/google-drive-ruby).
it pass spreadsheet url like this

https://spreadsheets.google.com/feeds/worksheets/1IQiTTQGp2mJnBC_12IFWzfluDIVbO61E57ZifPBZcMA/private/full

but current google-api-ruby-client handle a below-url.

https://sheets.googleapis.com/v4/spreadsheets/1IQiTTQGp2mJnBC_12IFWzfluDIVbO61E57ZifPBZcMA?fields=sheets.properties

i think is's a reason why it says 404.

is that right?

dazuma · 2021-08-13T17:37:04Z

Sorry, I wanted to know which method in the spreadsheets client class you are calling directly (not which low-level method in the core library is being called).

Also, are you using the google-api-ruby-client gem? That gem is deprecated and is not being updated, so it's possible it's trying to hit an old endpoint. Consider using the google-apis-sheets_v4 gem.

According to the official documentation, the correct URLs are of the form https://sheets.googleapis.com/v4/.... So I don't think that's the reason.

If you are still getting 404s, file a bug with the sheets API team: go to https://developers.google.com/sheets/api/reference/rest and click the "send feedback" button and choose "product feedback". That will go to the bug form directly to the sheets team. Let them know the URL you are trying to hit. I don't think we can do much in these open source client libraries.

shiratsu · 2021-08-14T00:10:07Z

@dazuma

i think this is a 404 reason that our system use spreadsheet api which is https://spreadsheets.google.com/feeds/worksheets/~.
because current spreadsheet url is https://sheets.googleapis.com/v4/spreadsheets.

we are using old? wrong? url(https://spreadsheets.google.com/feeds/worksheets~).

so i need to change url.

but you mean you think different reason?

shiratsu · 2021-08-14T07:24:50Z

Sorry, I wanted to know which method in the spreadsheets client class you are calling directly (not which low-level method in the core library is being called).

i am going to post it later

bonyii · 2021-08-16T14:59:45Z

For us, updating the google_drive gem to 3.0.7, updating service accounts, and allowing them to access the G-suite including the drive solved the issue.

https://developers.google.com/identity/protocols/oauth2/service-account#creatinganaccount
https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority

AndrewRayCode · 2021-08-18T00:00:04Z

We're hitting the same issue and we're unable to update the google_drive gem because of dependency hell in a large project. We can't use the raw api library out of the box, because of more dependency hell

  In Gemfile:
    google-apis-sheets_v4 was resolved to 0.6.0, which depends on
      google-apis-core (~> 0.1) was resolved to 0.4.1, which depends on
        representable (~> 3.0)

    roar was resolved to 1.0.4, which depends on
      representable (< 2.4.0, >= 2.0.1)

Does anyone know why Google chose to mess with everyone without warning?

AndrewRayCode · 2021-08-18T14:52:35Z

See also #6103

@dazuma could you share any inside information you have on why Google changed their APIs without warning within the last few days? Usually Google is good about deprecation notices

dazuma · 2021-08-18T21:56:23Z

It's possible that the https://spreadsheets.google.com/feeds/worksheets/... URLs, used by older versions of the google_drive gem, are no longer working, and that you just need to update google_drive to a newer version as @bonyii suggests. Unfortunately, I don't have enough visibility into the Sheets/GSuite team to be able to confirm whether or not the endpoint has changed. I do know that the client library has pretty much always been using https://sheets.googleapis.com/v4/spreadsheets/.... Even the old gem version (google-api-client 0.17.2 from 2017) was using the correct URL, as far as I can tell.

If you can determine where the https://spreadsheets.google.com/feeds/worksheets/... URLs are coming from, that might help. I don't think the client library itself should be setting that URL, but maybe you can show me otherwise.

It's also possible that something else is going on. @bonyii said they updated their service accounts and auth setup, so I suppose it's also possible something changed in the auth setup.

Did anything change in your Gemfile related to auth (for example, new version of the googleauth gem) when this started happening?
Maybe try just updating your credentials as @bonyii describes, and see if that has any effect.

AndrewRayCode · 2021-08-26T23:33:45Z

This likely broke for all of us consumers because of Google's "deprecation" (aka removal) of the v3 API: https://cloud.google.com/blog/products/g-suite/migrate-your-apps-use-latest-sheets-api

Did anyone else get any notifications from Google about this? I'm trying to figure out if there's any channels we should have caught this on.

jpriollaud · 2022-06-06T19:55:03Z

@yoshi-automation It appears to be authentication related. Hence for both read (@shiratsu ) and write (others) this error is observed. A clearer error message, probably related to credentials refresh, etc., and a more appropriate error code, from Google would be appreciated.

jpriollaud · 2022-06-06T19:58:21Z

@yoshi-automation also note that @bonyii made authentication-related changes, which may have solved the problem, irrespective of gem changes, which are less germane.

Hupdy · 2023-07-05T15:29:49Z

😒

shiratsu changed the title ~~http_command says 404~~ Invalid request (Google::Apis::ClientError) 404 Aug 13, 2021

dazuma added api: sheets Issues related to the Sheets API API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Aug 18, 2021

zdavis mentioned this issue Aug 23, 2021

Google Drive session not fetching files - 404 'Invalid Request' gimite/google-drive-ruby#382

Open

yoshi-automation added 🚨 This issue needs some love. and removed 🚨 This issue needs some love. labels Nov 24, 2021

yoshi-automation added the 🚨 This issue needs some love. label Feb 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Invalid request (Google::Apis::ClientError) 404 #6103

Invalid request (Google::Apis::ClientError) 404 #6103

shiratsu commented Aug 13, 2021

shiratsu commented Aug 13, 2021

h-z commented Aug 13, 2021

dazuma commented Aug 13, 2021

shiratsu commented Aug 13, 2021 •

edited

Loading

dazuma commented Aug 13, 2021 •

edited

Loading

shiratsu commented Aug 14, 2021 •

edited

Loading

shiratsu commented Aug 14, 2021

bonyii commented Aug 16, 2021

AndrewRayCode commented Aug 18, 2021

AndrewRayCode commented Aug 18, 2021

dazuma commented Aug 18, 2021

AndrewRayCode commented Aug 26, 2021

jpriollaud commented Jun 6, 2022

jpriollaud commented Jun 6, 2022

Hupdy commented Jul 5, 2023

Invalid request (Google::Apis::ClientError) 404 #6103

Invalid request (Google::Apis::ClientError) 404 #6103

Comments

shiratsu commented Aug 13, 2021

shiratsu commented Aug 13, 2021

h-z commented Aug 13, 2021

dazuma commented Aug 13, 2021

shiratsu commented Aug 13, 2021 • edited Loading

dazuma commented Aug 13, 2021 • edited Loading

shiratsu commented Aug 14, 2021 • edited Loading

shiratsu commented Aug 14, 2021

bonyii commented Aug 16, 2021

AndrewRayCode commented Aug 18, 2021

AndrewRayCode commented Aug 18, 2021

dazuma commented Aug 18, 2021

AndrewRayCode commented Aug 26, 2021

jpriollaud commented Jun 6, 2022

jpriollaud commented Jun 6, 2022

Hupdy commented Jul 5, 2023

shiratsu commented Aug 13, 2021 •

edited

Loading

dazuma commented Aug 13, 2021 •

edited

Loading

shiratsu commented Aug 14, 2021 •

edited

Loading