Skip to content
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@ This plugin offers a caching layer for optimal performance. It will be used if y

The default TTL for all cache objects is 5 minutes, but it can be [configured per query or request](../extending/query.md#cache_ttl-intnullcallable). Error responses are cached for 30 seconds to avoid overwhelming the remote data source under error conditions. Multiple requests for the same data within a single page load will be deduplicated even if the requests are not cacheable.

### Cache isolation and custom request headers

The response cache is shared across queries. When the site uses a persistent object cache, it is also shared across requests and users. Cache entries distinguish requests by their method, URI, body, and a configured list of request headers. `Authorization` and `Cache-Control` are included in that list by default, but arbitrary request headers are not included automatically.

**Security warning:** If an API uses a custom header for authentication, authorization, tenancy, or any other value that changes the response, that header must be added to the cache key. Otherwise, requests that differ only by that header can share a cache entry. With a persistent object cache, this can cause a response fetched with one credential or security context to be returned to a request using another, potentially exposing protected remote data.

Use each query's [`cache_key_request_headers`](../extending/query.md#cache_key_request_headers-array) configuration to add every custom header that can affect the authorized or returned data. Headers defined by a data source are not added to cache keys automatically. The built-in defaults cannot be removed.

## Technical concepts

If you want to understand the internals of Remote Data Blocks so that you can write code to extend its functionality, head over to the [extending guide](../extending/index.md).
Expand Down
2 changes: 2 additions & 0 deletions docs/extending/data-source.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ An associative array of headers that will be sent with each HTTP request. Querie

When providing authentication credentials, take care to avoid committing them to code repositories. We strongly recommend using environment variables or secure storage.

**Security warning:** Defining a custom authentication, authorization, tenancy, or response-varying header on a data source does not automatically include it in cache keys. Add the header name to the [`cache_key_request_headers`](query.md#cache_key_request_headers-array) configuration of every query that uses it. Otherwise, requests with different header values can share cached responses and potentially expose protected data across requests or users when a persistent object cache is enabled.

### Next steps

After defining a data source in code, you can use it in a [query](query.md) to define how data is retrieved.
18 changes: 18 additions & 0 deletions docs/extending/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,24 @@ The `request_headers` property defines the request headers for the query. It can
},
```

### cache_key_request_headers: array

A static list of additional request header names whose values will be included in the object cache key for this query. `Authorization` and `Cache-Control` are always included by default, and duplicate names are removed case-insensitively. A configured header that is absent from a request is ignored.

```php
'cache_key_request_headers' => [ 'X-Request-Scope' ],
```

**Security warning:** Add every header that can affect authentication, authorization, tenancy, or the returned data, including custom headers inherited from the query's data source. Data-source request headers are not added to cache keys automatically. Omitting such a header can allow requests with different security contexts to share a cached response, potentially exposing protected data across requests and users when a persistent object cache is enabled.

If you implement `HttpQueryInterface` directly, implement `get_cache_key_request_headers()`. Return an empty array to use only the built-in `Authorization` and `Cache-Control` defaults, or return additional header names for that query:

```php
public function get_cache_key_request_headers(): array {
return [ 'X-Api-Key' ];
}
```

### request_body: array|callable

The `request_body` property defines the request body for the query. It can be an associative array or a callable function that returns an associative array. The callable function accepts an associative array of input variables (`[ $var_name => $value ]`). If omitted, the query will not have a request body.
Expand Down
10 changes: 10 additions & 0 deletions inc/Config/Query/HttpQuery.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
use RemoteDataBlocks\Config\DataSource\HttpDataSource;
use RemoteDataBlocks\Config\DataSource\HttpDataSourceInterface;
use RemoteDataBlocks\Config\QueryRunner\QueryRunner;
use RemoteDataBlocks\HttpClient\CacheKeyRequestHeaders;
use RemoteDataBlocks\Validation\ConfigSchemas;
use WP_Error;

Expand Down Expand Up @@ -38,6 +39,15 @@ public function execute_batch( array $array_of_input_variables ): array|WP_Error
return $query_runner->execute_batch( $this, $array_of_input_variables );
}

/**
* Get the request header names whose values should be included in cache keys.
*
* @return array<string> Request header names included in cache keys.
*/
public function get_cache_key_request_headers(): array {
return $this->config['cache_key_request_headers'] ?? [];
}

/**
* Define the cache object TTL for the current query execution's responses:
* - Return a positive integer to set a custom TTL in seconds.
Expand Down
8 changes: 8 additions & 0 deletions inc/Config/Query/HttpQueryInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@
interface HttpQueryInterface extends QueryInterface {
public function get_data_source(): HttpDataSourceInterface;
public function get_cache_ttl( array $input_variables ): null|int;

/**
* Get the request header names whose values should be included in cache keys.
*
* @return array<string> Request header names included in cache keys.
*/
public function get_cache_key_request_headers(): array;

public function get_endpoint( array $input_variables ): string;
public function get_request_method(): string;
public function get_request_headers( array $input_variables ): array|WP_Error;
Expand Down
10 changes: 9 additions & 1 deletion inc/Config/QueryRunner/QueryRunner.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
use GuzzleHttp\RequestOptions;
use RemoteDataBlocks\Config\Query\HttpQueryInterface;
use RemoteDataBlocks\Editor\DataBinding\Pagination;
use RemoteDataBlocks\HttpClient\CacheKeyRequestHeaders;
use RemoteDataBlocks\HttpClient\HttpClient;
use RemoteDataBlocks\HttpClient\RdbCacheMiddleware;
use RemoteDataBlocks\HttpClient\RdbCacheStrategy;
use WP_Error;

Expand Down Expand Up @@ -57,6 +59,10 @@ protected function get_request_details( HttpQueryInterface $query, array $input_
$body = $query->get_request_body( $input_variables );
$endpoint = $query->get_endpoint( $input_variables );
$cache_ttl = $query->get_cache_ttl( $input_variables );
$cache_key_request_headers = CacheKeyRequestHeaders::merge(
CacheKeyRequestHeaders::DEFAULT_HEADERS,
$query->get_cache_key_request_headers()
);
$parsed_url = wp_parse_url( $endpoint );

if ( false === $parsed_url ) {
Expand Down Expand Up @@ -91,7 +97,9 @@ protected function get_request_details( HttpQueryInterface $query, array $input_
$pass = ( $user || $pass ) ? $pass . '@' : '';
$origin = sprintf( '%s://%s%s%s%s', $scheme, $user, $pass, $host, $port );

$cache_headers = [];
$cache_headers = [
RdbCacheMiddleware::CACHE_KEY_REQUEST_HEADERS_HEADER => $cache_key_request_headers,
];
if ( intval( $cache_ttl ) > 0 ) {
$cache_headers[ RdbCacheStrategy::CACHE_TTL_REQUEST_HEADER ] = $cache_ttl;
}
Expand Down
32 changes: 32 additions & 0 deletions inc/HttpClient/CacheKeyRequestHeaders.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<?php declare(strict_types = 1);

namespace RemoteDataBlocks\HttpClient;

final class CacheKeyRequestHeaders {
public const DEFAULT_HEADERS = [ 'Authorization', 'Cache-Control' ];
Comment thread
maxschmeling marked this conversation as resolved.
Outdated

/**
* Merge request header name lists without case-insensitive duplicates.
*
* @param array<string> ...$header_lists Request header name lists.
* @return array<string> Merged request header names.
*/
public static function merge( array ...$header_lists ): array {
$merged_headers = [];
$seen_headers = [];

foreach ( $header_lists as $header_list ) {
foreach ( $header_list as $header ) {
$normalized_header = strtolower( $header );
if ( isset( $seen_headers[ $normalized_header ] ) ) {
continue;
}

$seen_headers[ $normalized_header ] = true;
$merged_headers[] = $header;
}
}

return $merged_headers;
Comment thread
maxschmeling marked this conversation as resolved.
Outdated
}
}
12 changes: 12 additions & 0 deletions inc/HttpClient/RdbCacheMiddleware.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,19 @@

namespace RemoteDataBlocks\HttpClient;

use Psr\Http\Message\RequestInterface;

class RdbCacheMiddleware extends \Kevinrob\GuzzleCache\CacheMiddleware {
public const CACHE_KEY_REQUEST_HEADERS_HEADER = 'X-Remote-Data-Blocks-Cache-Key-Headers';
Comment thread
chriszarate marked this conversation as resolved.
Outdated

public function __invoke( callable $handler ): callable {
Comment thread
maxschmeling marked this conversation as resolved.
Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand what is accomplished by this __invoke override. I believe this header is already removed by the RdbCacheStrategy#getCacheObject

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left over duplicate. I've removed it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The __invoke is still present

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, I don't think this __invoke has any effect. Can you test this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't. I removed it but didn't get that in the commit I guess. Pushed in 78741db

$handler_without_cache_metadata = function ( RequestInterface $request, array $options ) use ( $handler ) {
return $handler( $request->withoutHeader( self::CACHE_KEY_REQUEST_HEADERS_HEADER ), $options );
};

return parent::__invoke( $handler_without_cache_metadata );
}

/**
* @var array<string, true>
*/
Expand Down
29 changes: 16 additions & 13 deletions inc/HttpClient/RdbCacheStrategy.php
Original file line number Diff line number Diff line change
Expand Up @@ -19,31 +19,30 @@ class RdbCacheStrategy extends GreedyCacheStrategy {
public const CACHE_TTL_REQUEST_HEADER = GreedyCacheStrategy::HEADER_TTL;
public const WP_OBJECT_CACHE_GROUP = 'remote-data-blocks';

private const CACHE_INVALIDATING_REQUEST_HEADERS = [ 'Authorization', 'Cache-Control' ];
private const ERROR_CACHE_TTL_IN_SECONDS = 30; // 30 seconds for error responses
private const FALLBACK_CACHE_TTL_IN_SECONDS = 300; // 5 minutes for success responses

public function __construct( ?CacheStorageInterface $storage = null ) {
// Filter this if customization is needed.
$vary_headers = new KeyValueHttpHeader( self::CACHE_INVALIDATING_REQUEST_HEADERS );

parent::__construct(
$storage ?? new WordPressObjectCacheStorage( self::WP_OBJECT_CACHE_GROUP ),
self::FALLBACK_CACHE_TTL_IN_SECONDS,
$vary_headers
Comment thread
chriszarate marked this conversation as resolved.
self::FALLBACK_CACHE_TTL_IN_SECONDS
);
}

public static function get_object_cache_key_from_request( RequestInterface $request ): string {
$request_body = (string) $request->getBody();
$request_headers = $request->getHeaders();
$request_method = $request->getMethod();
$request_uri = (string) $request->getUri();

$cache_key_request_headers = CacheKeyRequestHeaders::merge(
CacheKeyRequestHeaders::DEFAULT_HEADERS,
$request->getHeader( RdbCacheMiddleware::CACHE_KEY_REQUEST_HEADERS_HEADER )
);

$cache_headers = [];
foreach ( self::CACHE_INVALIDATING_REQUEST_HEADERS as $header ) {
if ( isset( $request_headers[ $header ] ) ) {
$cache_headers[ $header ] = $request_headers[ $header ];
foreach ( $cache_key_request_headers as $header ) {
if ( $request->hasHeader( $header ) ) {
$cache_headers[ $header ] = $request->getHeader( $header );
}
}

Expand Down Expand Up @@ -81,11 +80,15 @@ protected function getCacheObject( RequestInterface $request, ResponseInterface
$jitter = intval( ceil( min( $ttl * 0.1, 20 ) ) );
$ttl = intval( $ttl ) + wp_rand( 0, $jitter );

// NOTE: We skip the vary headers '*' check from the parent method
// since our defined vary headers cannot accept a '*' value.
// Cache-key request headers are resolved per request in getCacheKey(), so
// the parent's static vary-header check does not apply here.

$response = $response->withoutHeader( 'Etag' )->withoutHeader( 'Last-Modified' );

return new CacheEntry( $request->withoutHeader( static::HEADER_TTL ), $response, new DateTime( sprintf( '%+d seconds', $ttl ) ) );
$cache_request = $request
->withoutHeader( static::HEADER_TTL )
->withoutHeader( RdbCacheMiddleware::CACHE_KEY_REQUEST_HEADERS_HEADER );

return new CacheEntry( $cache_request, $response, new DateTime( sprintf( '%+d seconds', $ttl ) ) );
}
}
16 changes: 8 additions & 8 deletions inc/HttpClient/RdbLogMiddleware.php
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ public function __invoke( callable $handler ): callable {
return function ( RequestInterface $request, array &$options ) use ( $handler ): PromiseInterface {
return $handler( $request, $options )
->then(
$this->handle_success( $request, $options ),
$this->handle_failure( $request, $options )
$this->handle_success( $request ),
Comment thread
maxschmeling marked this conversation as resolved.
Outdated
$this->handle_failure( $request )
);
};
}
Expand Down Expand Up @@ -57,20 +57,20 @@ private function log( RequestInterface $request, ?ResponseInterface $response, ?
/**
* Returns a function which is handled when a request was rejected.
*/
private function handle_failure( RequestInterface $request, array $options ): callable {
return function ( \Exception $reason ) use ( $request, $options ) {
private function handle_failure( RequestInterface $request ): callable {
return function ( \Exception $reason ) use ( $request ) {
$response = ( $reason instanceof RequestException && $reason->hasResponse() === true ) ? $reason->getResponse() : null;
$this->log( $request, $response, $reason, $options );
$this->log( $request, $response, $reason );
return Create::rejectionFor( $reason );
};
}

/**
* Returns a function which is handled when a request was successful.
*/
private function handle_success( RequestInterface $request, array $options ): callable {
return function ( ResponseInterface $response ) use ( $request, $options ) {
$this->log( $request, $response, null, $options );
private function handle_success( RequestInterface $request ): callable {
return function ( ResponseInterface $response ) use ( $request ) {
$this->log( $request, $response, null );
return $response;
};
}
Expand Down
2 changes: 2 additions & 0 deletions inc/Integrations/Shopify/ShopifyIntegration.php
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ public static function register_blocks(): void {
public static function get_queries( ShopifyDataSource $data_source ): array {
return [
'shopify_get_product' => GraphqlQuery::from_array( [
'cache_key_request_headers' => [ 'X-Shopify-Storefront-Access-Token' ],
'display_name' => 'Get Shopify product by ID',
'data_source' => $data_source,
'input_schema' => [
Expand Down Expand Up @@ -88,6 +89,7 @@ public static function get_queries( ShopifyDataSource $data_source ): array {
'graphql_query' => file_get_contents( __DIR__ . '/Queries/GetProductById.graphql' ),
] ),
'shopify_search_products' => GraphqlQuery::from_array( [
'cache_key_request_headers' => [ 'X-Shopify-Storefront-Access-Token' ],
'display_name' => 'Search Shopify products',
'data_source' => $data_source,
'input_schema' => [
Expand Down
1 change: 1 addition & 0 deletions inc/Validation/ConfigSchemas.php
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ private static function generate_http_query_config_schema(): array {
return Types::object( [
'display_name' => Types::nullable( Types::string() ),
'cache_ttl' => Types::nullable( Types::one_of( Types::callable(), Types::integer(), Types::null() ) ),
'cache_key_request_headers' => Types::nullable( Types::list_of( Types::string() ) ),
'data_source' => Types::one_of(
Types::instance_of( HttpDataSource::class ),
Types::serialized_config_for( HttpDataSource::class ),
Expand Down
27 changes: 27 additions & 0 deletions tests/inc/Config/QueryRunnerTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@
namespace RemoteDataBlocks\Tests\Config;

use GuzzleHttp\Psr7\Response;
use GuzzleHttp\RequestOptions;
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;
use RemoteDataBlocks\Config\Query\HttpQueryInterface;
use RemoteDataBlocks\Config\QueryRunner\QueryRunner;
use RemoteDataBlocks\HttpClient\HttpClient;
use RemoteDataBlocks\HttpClient\RdbCacheMiddleware;
use RemoteDataBlocks\Tests\Mocks\MockDataSource;
use RemoteDataBlocks\Tests\Mocks\MockQuery;
use WP_Error;
Expand Down Expand Up @@ -93,6 +96,30 @@ public function testExecuteSuccessfulRequest( string $endpoint ): void {
$this->assertArrayHasKey( 'results', $result );
}

public function testRequestDetailsIncludeCacheKeyRequestHeadersHeader(): void {
$data_source = MockDataSource::create();
$this->assertInstanceOf( MockDataSource::class, $data_source );

$query = MockQuery::create( [
'cache_key_request_headers' => [ 'X-Api-Key' ],
'data_source' => $data_source,
] );
$this->assertInstanceOf( MockQuery::class, $query );

$query_runner = new class($this->http_client, []) extends QueryRunner {
public function get_request_details_for_test( HttpQueryInterface $query ): array|WP_Error {
return $this->get_request_details( $query, [] );
}
};

$request_details = $query_runner->get_request_details_for_test( $query );
$this->assertIsArray( $request_details );
$this->assertSame(
[ 'Authorization', 'Cache-Control', 'X-Api-Key' ],
$request_details['options'][ RequestOptions::HEADERS ][ RdbCacheMiddleware::CACHE_KEY_REQUEST_HEADERS_HEADER ] ?? null
);
}

public static function provideInvalidEndpoints(): array {
return [
[
Expand Down
22 changes: 22 additions & 0 deletions tests/inc/Config/QueryTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
namespace RemoteDataBlocks\Tests\Config;

use PHPUnit\Framework\TestCase;
use RemoteDataBlocks\Config\DataSource\HttpDataSource;
use RemoteDataBlocks\Config\Query\HttpQuery;
use RemoteDataBlocks\Tests\Mocks\MockDataSource;

Expand Down Expand Up @@ -37,6 +38,27 @@ public function testGetRequestHeaders(): void {
$this->assertSame( [ 'Content-Type' => 'application/json' ], $result );
}

public function testCacheKeyRequestHeadersMergeDefaultsAndQueryOnly(): void {
$data_source = HttpDataSource::from_array( [
'display_name' => 'Custom API',
'endpoint' => 'https://example.com/api',
'cache_key_request_headers' => [ 'X-Api-Key', 'x-tenant-id' ],
] );
$this->assertInstanceOf( HttpDataSource::class, $data_source );

$query = HttpQuery::from_array( [
'data_source' => $data_source,
'output_schema' => [ 'type' => 'null' ],
'cache_key_request_headers' => [ 'x-api-key', 'X-Request-Scope' ],
] );
$this->assertInstanceOf( HttpQuery::class, $query );

$this->assertSame(
[ 'Authorization', 'Cache-Control', 'x-api-key', 'X-Request-Scope' ],
$query->get_cache_key_request_headers()
);
}

public function testGetRequestBody(): void {
$this->assertNull( $this->query_context->get_request_body( [] ) );
}
Expand Down
Loading
Loading