-
Notifications
You must be signed in to change notification settings - Fork 123
docs: add guides for occ, configure_client, and troubleshooting #5441
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 5 commits
cb25b07
384d797
f403103
ad407be
9236399
151991d
7c09af5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // https://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| pub mod set_iam_policy; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // https://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| // [BEGIN rust_occ_loop] | ||
| use google_cloud_gax::error::rpc::Code; | ||
| use google_cloud_iam_v1::model::Binding; | ||
| use google_cloud_iam_v1::model::Policy; | ||
| use google_cloud_secretmanager_v1::client::SecretManagerService; | ||
| use google_cloud_wkt::FieldMask; | ||
|
|
||
| /// Executes an Optimistic Concurrency Control (OCC) loop to safely update a resource. | ||
| /// | ||
| /// This function demonstrates the core Read-Modify-Write-Retry pattern. It uses the secret manager | ||
| /// service and a hard-coded role. The principles apply to any other service or role. | ||
| /// | ||
| /// # Parameters | ||
| /// * `project_id` The Google Cloud Project ID (e.g., "my-project-123"). | ||
| /// * `secret_id` The Google Cloud Project ID (e.g., "my-secret"). | ||
| /// * `member` The member to add (e.g., "user:user@example.com"). | ||
| /// | ||
| /// # Returns | ||
| /// The new IAM policy. | ||
| pub async fn sample(project_id: &str, secret_id: &str, member: &str) -> anyhow::Result<Policy> { | ||
| // ANCHOR: occ-loop | ||
| const ROLE: &str = "roles/secretmanager.secretAccessor"; | ||
| const ATTEMPTS: u32 = 5; | ||
|
|
||
| let secret_name = format!("projects/{project_id}/secrets/{secret_id}"); | ||
| let client = SecretManagerService::builder().build().await?; | ||
| for _attempt in 0..ATTEMPTS { | ||
| let mut current = client | ||
| .get_iam_policy() | ||
| .set_resource(&secret_name) | ||
| .send() | ||
| .await?; | ||
|
|
||
| match current.bindings.iter_mut().find(|b| b.role == ROLE) { | ||
| None => current | ||
| .bindings | ||
| .push(Binding::new().set_role(ROLE).set_members([member])), | ||
| Some(b) => { | ||
| if b.members.iter().find(|m| *m == member).is_some() { | ||
| return Ok(current); | ||
| } | ||
| b.members.push(member.to_string()); | ||
| } | ||
| }; | ||
| let updated = client | ||
| .set_iam_policy() | ||
| .set_resource(&secret_name) | ||
| .set_policy(current) | ||
| .set_update_mask(FieldMask::default().set_paths(["bindings"])) | ||
| .send() | ||
| .await; | ||
| match updated { | ||
| Ok(p) => return Ok(p), | ||
| Err(e) | ||
| if e.status().is_some_and(|s| { | ||
| s.code == Code::Aborted || s.code == Code::FailedPrecondition | ||
| }) => | ||
| { | ||
| continue; | ||
| } | ||
| Err(e) => return Err(e.into()), | ||
| } | ||
| } | ||
| anyhow::bail!("could not set IAM policy after {ATTEMPTS} attempts") | ||
| // ANCHOR_END: occ-loop | ||
| } | ||
| // [END rust_occ_loop] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| # How to configure a client | ||
|
|
||
| The Google Cloud Rust Client Libraries let you configure client behavior | ||
| using a configuration object passed to the client constructor. This configuration | ||
| is typically handled by a `ClientConfig` or `Config` struct provided by the | ||
| specific service crate. | ||
|
|
||
| ## 1. Customizing the API endpoint | ||
|
|
||
| See [Override the default endpoint][override-default-endpoint]. | ||
|
|
||
| ## 2. Authentication configuration | ||
|
|
||
| While the client attempts to find [Application Default Credentials (ADC)][adc] | ||
| automatically, you can explicitly provide them using the `with_auth` or | ||
| `with_api_key` methods on the configuration object. See | ||
| [`Override the default authentication method`][authentication] for details and | ||
| examples. | ||
|
|
||
| ## 3. Logging | ||
|
|
||
| Logging is handled through the `tracing` ecosystem. You can configure a | ||
| subscriber to capture logs and traces from the client libraries. | ||
| See [Troubleshooting](/troubleshooting.md) for a comprehensive guide. | ||
|
|
||
| ## 3. Configuring a proxy | ||
|
|
||
| The configuration method depends on whether you are using a gRPC or REST-based | ||
| transport. | ||
|
|
||
| ### Proxy with gRPC | ||
|
|
||
| When using the gRPC transport (standard for most services), the client library | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nope, they almost all use REST. |
||
| respects the [standard environment variables][envvars]. You don't need to | ||
| configure this in the Rust code itself. | ||
|
|
||
| Set the following environment variables in your shell or container: | ||
|
|
||
| ```bash | ||
| export http_proxy="http://proxy.example.com:3128" | ||
| export https_proxy="http://proxy.example.com:3128" | ||
|
Comment on lines
+40
to
+41
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume this is true, but frankly I have not tested this feature. |
||
| ``` | ||
|
|
||
| **Handling self-signed certificates (gRPC):** If your proxy uses a self-signed | ||
| certificate (Deep Packet Inspection), you cannot "ignore" verification in gRPC. | ||
| You must provide the path to the proxy's CA certificate bundle. | ||
|
|
||
| ```bash | ||
| # Point gRPC to a CA bundle that includes your proxy's certificate | ||
| export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH="/path/to/roots.pem" | ||
| ``` | ||
|
Comment on lines
+48
to
+51
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nothing in Rust uses that environment variable. |
||
|
|
||
| ### Proxy with REST | ||
|
|
||
| If you're using a library that supports REST transport, you can configure the | ||
| proxy by providing a custom `reqwest` or `hyper` client to the configuration, | ||
| depending on the specific implementation of the crate. | ||
|
|
||
| ```rust | ||
| use google_cloud_secret_manager::v1::client::{SecretManagerClient, ClientConfig}; | ||
|
|
||
| async fn run() -> Result<(), Box<dyn std::error::Error>> { | ||
| // Configure a proxy using standard environment variables or a custom connector | ||
| let proxy = reqwest::Proxy::all("http://user:password@proxy.example.com")?; | ||
| let http_client = reqwest::Client::builder() | ||
| .proxy(proxy) | ||
| .build()?; | ||
|
|
||
| let config = ClientConfig::default() | ||
| .with_http_client(http_client); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not an API. Which is why we compile our sample code. |
||
|
|
||
| let client = SecretManagerClient::new(config).await?; | ||
| Ok(()) | ||
| } | ||
| ``` | ||
|
Comment on lines
+59
to
+75
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please move this to a code sample so we can compile it, it won't compile |
||
|
|
||
| ## 4. Configuring retries | ||
|
|
||
| See [Configuring retry policies](/configuring_retry_policies.md) | ||
|
|
||
| ## 5. Logging | ||
|
|
||
| You can use the `tracing` crate to capture client logs. By initializing a | ||
| subscriber, you can debug request metadata, status codes, and events. | ||
|
|
||
| ```rust | ||
| use tracing_subscriber; | ||
|
|
||
| fn main() { | ||
| // Initialize tracing subscriber to see debug output from the client | ||
| tracing_subscriber::fmt() | ||
| .with_max_level(tracing::Level::DEBUG) | ||
| .init(); | ||
| } | ||
| ``` | ||
|
Comment on lines
+81
to
+95
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we can just reference this doc instead: |
||
|
|
||
| ## 6. Other common configuration options | ||
|
|
||
| The following options can be passed to the configuration builder of most | ||
| clients. | ||
|
|
||
| | Option | Type | Description | | ||
| | ----- | ----- | ----- | | ||
| | `credentials` | `Option<Credentials>` | Explicit credentials object for authentication. | | ||
| | `endpoint` | `String` | The address of the API remote host. Used for Regional Endpoints (e.g., `https://us-central1-pubsub.googleapis.com:443`) or Private Service Connect. | | ||
|
|
||
| To use API Keys [override the default credentials with API keys][override-api-keys]. | ||
|
Comment on lines
+97
to
+107
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
|
||
| [adc]: https://cloud.google.com/docs/authentication/application-default-credentials | ||
| [authentication]: https://docs.cloud.google.com/rust/override-default-authentication | ||
| [envvars]: https://grpc.github.io/grpc/core/md_doc_environment_variables.html | ||
| [override-default-endpoint]: https://docs.cloud.google.com/rust/override-default-endpoint | ||
| [override-api-keys]: https://docs.cloud.google.com/rust/override-default-authentication#override_the_default_credentials_api_keys | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| <!-- | ||
| Copyright 2025 Google LLC | ||
|
|
||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
|
|
||
| https://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| --> | ||
|
|
||
| # How to work with Optimistic Concurrency Control (OCC) | ||
|
|
||
| Optimistic Concurrency Control (OCC) is a strategy used to manage shared | ||
| resources and prevent "lost updates" or race conditions when multiple users or | ||
| processes attempt to modify the same resource simultaneously. | ||
|
|
||
| As an example, consider systems like Google Cloud IAM, where | ||
| the shared resource is an **IAM Policy** applied to a resource (like a Project, | ||
| Bucket, or Service). To implement OCC, systems typically use a version number or | ||
| an `etag` (entity tag) field on the resource struct. | ||
|
|
||
| ## Introduction to OCC | ||
|
|
||
| Imagine two processes, A and B, try to update a shared resource at the same | ||
| time: | ||
|
|
||
| 1. Process **A** reads the current state of the resource. | ||
|
|
||
| 2. Process **B** reads the *same* current state. | ||
|
|
||
| 3. Process **A** modifies its copy and writes it back to the server. | ||
|
|
||
| 4. Process **B** modifies its copy and writes it back to the server. | ||
|
|
||
| Because Process **B** overwrites the resource *without* knowing that Process | ||
| **A** already changed it, Process **A**'s updates are **lost**. | ||
|
|
||
| OCC solves this by introducing a unique fingerprint which changes every time an | ||
| entity is modified. In many systems (like IAM), this is done | ||
| using an `etag`. The server checks this tag on every write: | ||
|
|
||
| 1. When you read the resource, the server returns an `etag` (a unique | ||
| fingerprint). | ||
|
|
||
| 2. When you send the modified resource back, you must include the original | ||
| `etag`. | ||
|
|
||
| 3. If the server finds that the stored `etag` does **not** match the `etag` you | ||
| sent (meaning someone else modified the resource since you read it), the write | ||
| operation fails with an `ABORTED` or `FAILED_PR1ECONDITION` error. | ||
|
bshaffer marked this conversation as resolved.
Outdated
|
||
|
|
||
| This failure forces the client to **retry** the entire process—re-read the *new* | ||
| state, re-apply the changes, and try the write again with the new `etag`. | ||
|
|
||
| ## Implementing the OCC loop | ||
|
|
||
| The core of the OCC implementation is a loop that handles the retry logic. You | ||
| should set a reasonable maximum number of retries to prevent infinite loops in | ||
| cases of high contention. | ||
|
|
||
| ### Steps of the loop: | ||
|
|
||
| | **Step** | **Action** | **Implementation example** | | ||
| | --- | --- | --- | | ||
| | **Read** | Fetch the current resource state, including the `etag`. | `let mut policy = client.get_iam_policy(request).await?;` | | ||
| | **Modify** | Apply the changes to the local struct. | `policy.bindings.push(new_binding);` | | ||
| | **Write/Check** | Attempt to save the modified resource using the old `etag`. This action is checked for specific error codes. | `match client.set_iam_policy(request).await { Ok(p) => return Ok(p), Err(e) => { /* retry logic */ } }` | | ||
| | **Success/Retry** | If the write succeeds, exit the loop. If it fails with a concurrency error, increment the retry counter and continue the loop (go back to the Read step). | | | ||
|
|
||
| The following code provides an example of how to implement the OCC loop using an | ||
| IAM policy on a Project resource as the target. | ||
|
|
||
| **Note**: This example assumes the use of the Secret Manager client, but the | ||
| same OCC pattern applies to any service or database that implements versioned | ||
| updates. | ||
|
|
||
| ### Example | ||
|
|
||
| As usual with Rust, you must declare the dependency in your `Cargo.toml` file: | ||
|
|
||
| ```shell | ||
| cargo add google-cloud-secretmanager-v1 | ||
| ``` | ||
|
|
||
| ```rust,ignore | ||
| {{#include ../samples/src/occ/set_iam_policy.rs:occ-loop}} | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,68 @@ | ||||||
| # Troubleshoot the Google Cloud Rust Client Library | ||||||
|
|
||||||
| {% block body %} | ||||||
|
|
||||||
| ## Debug logging | ||||||
|
|
||||||
| The best way to troubleshoot is by enabling logging. See | ||||||
| [Enabling Logging][enable-logging] for more | ||||||
| information. | ||||||
|
|
||||||
| ## How can I trace gRPC issues? | ||||||
|
|
||||||
| When working with libraries that use gRPC, you can use the underlying gRPC | ||||||
| environment variables to enable logging. Most Rust clients use pure-Rust gRPC | ||||||
| implementations like `tonic`. | ||||||
|
|
||||||
| ### Prerequisites | ||||||
|
|
||||||
| Ensure your crate includes the necessary features for the gRPC transport. You | ||||||
| can verify your dependencies in `Cargo.toml`. | ||||||
|
|
||||||
| ### Transport logging with gRPC | ||||||
|
|
||||||
| The primary method for debugging gRPC calls in Rust is using the `tracing` | ||||||
| subscriber filters. You can target specific gRPC crates to see underlying | ||||||
| transport details. | ||||||
|
|
||||||
| NOTE: The `tracing` crate requires that you first initialize a | ||||||
| [`tracing_subscriber`][tracing_subscriber]. | ||||||
|
|
||||||
| For example, setting the `RUST_LOG` environment variable to include | ||||||
| `tonic=debug` or `h2=debug` will dump a lot of information regarding the gRPC | ||||||
| and HTTP/2 layers. | ||||||
|
|
||||||
| ```sh | ||||||
| RUST_LOG=debug,tonic=debug,h2=debug cargo run --example your_program | ||||||
| ``` | ||||||
|
|
||||||
| If you are using a client that wraps the gRPC C-core, environment variables like | ||||||
| `GRPC_TRACE` and `GRPC_VERBOSITY` may also be relevant. | ||||||
|
Comment on lines
+39
to
+40
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is never a wrapped C-core for gRPC in Rust.
Suggested change
|
||||||
|
|
||||||
| ## How can I diagnose proxy issues? | ||||||
|
|
||||||
| See [Client Configuration: Configuring a Proxy][client-configuration]. | ||||||
|
|
||||||
| ## Reporting a problem | ||||||
|
|
||||||
| If your issue is still not resolved, ask for help. If you have a support | ||||||
| contract with Google, create an issue in the | ||||||
| [support console][support] instead of filing on GitHub. | ||||||
| This will ensure a timely response. | ||||||
|
|
||||||
| Otherwise, file an issue on GitHub. Although there are multiple GitHub | ||||||
| repositories associated with the Google Cloud Libraries, we recommend filing | ||||||
| an issue in | ||||||
| [https://github.com/googleapis/google-cloud-rust][google-cloud-rust] | ||||||
| unless you are certain that it belongs elsewhere. The maintainers may move it to | ||||||
| a different repository where appropriate, but you will be notified of this using | ||||||
| the email associated with your GitHub account. | ||||||
|
|
||||||
| When filing an issue, include as much of the following information as possible. | ||||||
| This will enable us to help you quickly. | ||||||
|
|
||||||
| [client-configuration]: /configure_client.md | ||||||
| [google-cloud-rust]: https://github.com/googleapis/google-cloud-rust | ||||||
| [support]: https://cloud.google.com/support/ | ||||||
| [tracing_subscriber]: https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/index.html | ||||||
| [enable-logging]: https://docs.cloud.google.com/rust/enable-logging | ||||||
Uh oh!
There was an error while loading. Please reload this page.