Fix issue 1592 (Online tests can no longer be run with a service account)

.github/CONTRIBUTING.md

@@ -25,6 +25,21 @@ To run tests, add your credentials to `tests/creds.json` and run
 GS_CREDS_FILENAME="tests/creds.json" GS_RECORD_MODE="all" tox -e py -- -k "<specific test to run>"
 ```
 
+On Windows (PowerShell), set the environment variables first — PowerShell has no inline `VAR=value command` prefix:
+
+```powershell
+$env:GS_CREDS_FILENAME = "tests/creds.json"
+$env:GS_RECORD_MODE = "all"
+tox -e py -- -k "<specific test to run>"
+```
+
+The variables stay set for the rest of the PowerShell session. To clear them again, use
+`Remove-Item Env:GS_CREDS_FILENAME, Env:GS_RECORD_MODE` — this matters in particular for
+`GS_RECORD_MODE`, since leaving it set will keep later runs online.
+
+The rest of this guide shows the commands in `bash` form; on Windows, apply the same translation
+(export the variables with `$env:NAME = "value"`, then run the command on its own line).
+
 For more information on tests, see below.
 
 ## CI checks
@@ -59,11 +74,23 @@ The CI uses tox. For faster local development, you can set up an environment and
 
 ```bash
 python -m venv env
-source /env/bin/activate
-pip install test-requirements.txt
+source env/bin/activate
+pip install -r test-requirements.txt
 pytest -k TEST_NAME
 ```
 
+On Windows (PowerShell), the activation script lives elsewhere:
+
+```powershell
+python -m venv env
+.\env\Scripts\Activate.ps1
+pip install -r test-requirements.txt
+pytest -k TEST_NAME
+```
+
+If PowerShell refuses to run the activation script, allow local scripts for the current session with
+`Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned`.
+
 ## Format
 
 ```bash
@@ -92,9 +119,27 @@ gspread uses [vcrpy](https://github.com/kevin1024/vcrpy) to record and replay HT
 
 ### `GS_CREDS_FILENAME` environment variable
 
-You must provide service account credentials using the `GS_CREDS_FILENAME` environment variable in order to make HTTP requests to the Sheets API.
+You must provide **Service account** or **OAuth user credentials** using the `GS_CREDS_FILENAME` environment variable
+in order to make HTTP requests to the Sheets API. Note, that  **service account** can only be used in the
+Google Workspace environment, as otherwise service accounts have no Drive storage quota and can't own files, which is
+required to run tests.
+
+#### To obtain the **Service Account credentials**
+
+* Follow the [Using Service Account](https://docs.gspread.org/en/latest/oauth2.html#for-bots-using-service-account) procedure
+
+* Point  `GS_CREDS_FILENAME` at the key file you download from the Google
+Cloud Console (the one containing `private_key` and `client_email`).
+
+####  To obtain the **OAuth user credentials** 
+
+* Follow the [Using OAuth Client ID](https://docs.gspread.org/en/latest/oauth2.html#for-end-users-using-oauth-client-id) procedure
+
+
+* Point `GS_CREDS_FILENAME` at the `authorized_user.json` produced by the browser flow — the file
+containing `refresh_token` — **not** the OAuth client-secrets file, which the linked guide calls
+`credentials.json`.
 
-[Obtain service account credentials from Google Developers Console](https://docs.gspread.org/en/latest/oauth2.html#for-bots-using-service-account).
 
 ### `GS_RECORD_MODE` environment variable
 
@@ -110,6 +155,20 @@ In the following cases, you must record new HTTP requests:
 - an existing test is updated and does a new HTTP request
 - gspread is updated and does a new HTTP request
 
+### Run tests online, but not capture any HTTP requests
+
+```bash
+GS_CREDS_FILENAME=<./YOUR_CREDS.json> tox -e py -- --disable-vcr
+```
+
+```powershell
+$env:GS_CREDS_FILENAME = "./YOUR_CREDS.json"
+tox -e py -- --disable-vcr
+```
+
+`--disable-vcr` runs every request against the live API and leaves `tests/cassettes/` untouched.
+`GS_RECORD_MODE` is ignored in this mode.
+
 ### Run test, capturing *all* HTTP requests
 
 In some cases if the test suite can't record new episodes, or it can't replay them offline, you can run a complete update of the cassettes.
@@ -118,6 +177,12 @@ In some cases if the test suite can't record new episodes, or it can't replay th
 GS_CREDS_FILENAME=<./YOUR_CREDS.json> GS_RECORD_MODE=all tox -e py
 ```
 
+```powershell
+$env:GS_CREDS_FILENAME = "./YOUR_CREDS.json"
+$env:GS_RECORD_MODE = "all"
+tox -e py
+```
+
 ### Run test, capturing *only new* HTTP requests
 
 To record new HTTP requests:
@@ -133,10 +198,18 @@ To record new HTTP requests:
 GS_CREDS_FILENAME=<./YOUR_CREDS.json> GS_RECORD_MODE=new_episodes tox -e py
 ```
 
+```powershell
+$env:GS_CREDS_FILENAME = "./YOUR_CREDS.json"
+$env:GS_RECORD_MODE = "new_episodes"
+tox -e py
+```
+
 This will mostly result in a lot of updated files in `tests/cassettes/`. Don't forget to add them in your PR.
 Please add them in a dedicated commit, in order to make the review process easier.
 
 Afterwards, remember to [run the tests in offline mode](#run-tests-offline) to make sure you have recorded everything correctly.
+On Windows, clear `GS_RECORD_MODE` first (`Remove-Item Env:GS_RECORD_MODE`), otherwise the offline run
+will still try to record.
 
 ## Release
 

docs/oauth2.rst

@@ -208,19 +208,24 @@ For End Users: Using OAuth Client ID
 This is the case where your application or a script is accessing spreadsheets on behalf of an end user. When you use this scenario, your application or a script will ask the end user (or yourself if you're running it) to grant access to the user's data.
 
 1. :ref:`enable-api-access` if you haven't done it yet.
-#. Go to "APIs & Services > OAuth Consent Screen." Click the button for "Configure Consent Screen".
+#. Configure the consent screen. Go to "APIs & Services > OAuth consent screen" (this opens the "Google Auth Platform"). If the project doesn't have a consent screen yet, click "Get started".
 
-  a. In the "1 OAuth consent screen" tab, give your app a name and fill the "User support email" and "Developer contact information". Click "SAVE AND CONTINUE".
-  #. There is no need to fill in anything in the tab "2 Scopes", just click "SAVE AND CONTINUE".
-  #. In the tab "3 Test users", add the Google account email of the end user, typically your own Google email. Click "SAVE AND CONTINUE".
-  #. Double check the "4 Summary" presented and click "BACK TO DASHBOARD".
+  a. Under "App Information", give your app a name and select a "User support email". Click "Next".
+  #. Under "Audience", select "External". Click "Next".
+  #. Under "Contact Information", enter your email address. Click "Next".
+  #. Agree to the user data policy and click "Create".
 
-3. Go to "APIs & Services > Credentials"
-#. Click "+ Create credentials" at the top, then select "OAuth client ID".
-#. Select "Desktop app", name the credentials and click "Create". Click "Ok" in the "OAuth client created" popup.
-#. Download the credentials by clicking the Download JSON button in "OAuth 2.0 Client IDs" section.
+3. Add yourself as a test user. Go to "Audience", and under "Test users" click "Add users". Add the Google account you will authorize with, typically your own.
+#. Create the OAuth client. Go to "Clients" and click "Create client".
+#. Select "Desktop app" as the "Application type", name the client and click "Create".
+#. Download the JSON file for the client you just created (the download icon in the "OAuth 2.0 Client IDs" list).
 #. Move the downloaded file to ``~/.config/gspread/credentials.json``. Windows users should put this file to ``%APPDATA%\gspread\credentials.json``.
 
+.. NOTE::
+    Only accounts listed under "Test users" can authorize the app while its publishing status is "Testing". 
+    Signing in with any other account fails with ``Error 403: access_denied``. If you are signed in to several Google 
+    accounts, make sure the browser uses the one you added above.
+
 Create a new Python file with this code:
 
 ::
@@ -233,8 +238,21 @@ Create a new Python file with this code:
 
     print(sh.sheet1.get('A1'))
 
-When you run this code, it launches a browser asking you for authentication. Follow the instruction on the web page. Once finished, gspread stores authorized credentials in the config directory next to `credentials.json`.
-You only need to do authorization in the browser once, following runs will reuse stored credentials.
+When you run this code, it launches a browser asking you for authentication. Follow the instruction on the web page. Once finished, gspread stores the authorized credentials in ``authorized_user.json``, in the config directory next to ``credentials.json`` — i.e. ``~/.config/gspread/authorized_user.json``, or ``%APPDATA%\gspread\authorized_user.json`` on Windows.
+This file contains a refresh token, so following runs reuse it: you only need to do authorization in the browser once.
+
+.. NOTE::
+    Google shows a "Google hasn't verified this app" warning during authorization. This is expected: the app requests sensitive scopes and has not gone through Google's verification process, which is only required to distribute it to other people. To continue, click "Advanced", then "Go to <your app name> (unsafe)".
+
+.. WARNING::
+    While the publishing status of your app is "Testing", Google expires the refresh token after **7 days**. 
+    Once it expires, gspread fails with ``google.auth.exceptions.RefreshError: invalid_grant``.
+
+    To authorize again, **delete** ``authorized_user.json`` and re-run your code. gspread reuses an existing ``authorized_user.json`` 
+    without checking whether it is still valid, so the browser flow does not start again while the stale file is in place.
+
+    To stop the token from expiring, publish the app: go to "APIs & Services → OAuth consent screen → Audience" and 
+    click "Publish app". Read more `here <https://support.google.com/cloud/answer/15549945>`_ about publishing an app.
 
 .. NOTE::
     If you want to store the credentials file somewhere else, specify the path to `credentials.json` and `authorized_user.json` in :meth:`~gspread.oauth`:

tests/conftest.py

@@ -1,5 +1,6 @@
 import io
 import itertools
+import json
 import os
 import unittest
 from typing import Any, Dict, Generator, Optional, Tuple
@@ -29,6 +30,19 @@
 
 
 def read_credentials(filename: str) -> Credentials:
+    # Support both a service-account key and an OAuth "authorized_user" file.
+    # Unless in Google Workspace environment, service accounts have no Drive
+    # storage quota and can't own files, so running tests online
+    # (which create spreadsheets) requires OAuth user creds.
+    # An OAuth user file has a "refresh_token"; a service-account key has a
+    # "private_key" (and gspread's stored user file has no "type" field).
+    with open(filename) as file:
+        info = json.load(file)
+
+    if "refresh_token" in info:
+        # Scopes are taken from the stored token to avoid scope-mismatch errors.
+        return UserCredentials.from_authorized_user_info(info)
+
     return ServiceAccountCredentials.from_service_account_file(filename, scopes=SCOPE)