Skip to content

[TEST]#10812

Draft
yuqi1129 wants to merge 12 commits intomainfrom
feat/cache-jcasbin-id-mapping
Draft

[TEST]#10812
yuqi1129 wants to merge 12 commits intomainfrom
feat/cache-jcasbin-id-mapping

Conversation

@yuqi1129
Copy link
Copy Markdown
Contributor

…er operations

  • Add updated_at BIGINT NOT NULL DEFAULT 0 to role_meta, user_meta, group_meta, owner_meta
  • Create entity_change_log table for HA cross-node targeted metadataIdCache invalidation
  • Add covering indexes for auth read-path version checks
  • Add new mapper methods: touchUpdatedAt/batchGetUpdatedAt (RoleMetaMapper),
    touchUpdatedAt/getUserInfo (UserMetaMapper), touchUpdatedAt/getGroupInfoByUserId
    (GroupMetaMapper), selectOwnerByMetadataObjectId/selectChangedOwners (OwnerMetaMapper)
  • Create EntityChangeLogMapper with selectChanges/insertChange/pruneOldEntries
  • Add result types UserAuthInfo, GroupAuthInfo, OwnerInfo, RoleUpdatedAt,
    ChangedOwnerInfo, EntityChangeRecord (plain Java classes, Java 11 compatible)
  • Add schema-1.3.0 and upgrade-1.2.0-to-1.3.0 scripts for MySQL, H2, PostgreSQL
  • Add unit tests TestAuthMappers covering all new mapper methods

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com<!--

  1. Title: [#] ():
    Examples:
  2. If the PR is unfinished, please mark this PR as draft.
    -->

What changes were proposed in this pull request?

(Please outline the changes and how this PR fixes the issue.)

Why are the changes needed?

(Please clarify why the changes are needed. For instance,

  1. If you propose a new API, clarify the use case for a new API.
  2. If you fix a bug, describe the bug.)

Fix: #(issue)

Does this PR introduce any user-facing change?

(Please list the user-facing changes introduced by your change, including

  1. Change in user-facing APIs.
  2. Addition or removal of property keys.)

How was this patch tested?

(Please test your changes, and provide instructions on how to test it:

  1. If you add a feature or fix a bug, add a test to cover your changes.
  2. If you fix a flaky test, repeat it for many times to prove it works.)

yuqi1129 and others added 12 commits April 16, 2026 09:40
…er operations

- Add updated_at BIGINT NOT NULL DEFAULT 0 to role_meta, user_meta, group_meta, owner_meta
- Create entity_change_log table for HA cross-node targeted metadataIdCache invalidation
- Add covering indexes for auth read-path version checks
- Add new mapper methods: touchUpdatedAt/batchGetUpdatedAt (RoleMetaMapper),
  touchUpdatedAt/getUserInfo (UserMetaMapper), touchUpdatedAt/getGroupInfoByUserId
  (GroupMetaMapper), selectOwnerByMetadataObjectId/selectChangedOwners (OwnerMetaMapper)
- Create EntityChangeLogMapper with selectChanges/insertChange/pruneOldEntries
- Add result types UserAuthInfo, GroupAuthInfo, OwnerInfo, RoleUpdatedAt,
  ChangedOwnerInfo, EntityChangeRecord (plain Java classes, Java 11 compatible)
- Add schema-1.3.0 and upgrade-1.2.0-to-1.3.0 scripts for MySQL, H2, PostgreSQL
- Add unit tests TestAuthMappers covering all new mapper methods

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ervice write paths

- RoleMetaService: touchUpdatedAt on privilege grant/revoke (same transaction)
- UserMetaService: touchUpdatedAt on role assign/revoke (same transaction)
- GroupMetaService: touchUpdatedAt on role assign/revoke (same transaction)
- OwnerRelPO: add updatedAt field; insertOwnerRel SQL includes updated_at column
- POConverters: set updatedAt=currentTimeMillis in initializeOwnerRelPOsWithVersion
- CatalogMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- SchemaMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- TableMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- FilesetMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- TopicMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- ViewMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- ModelMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- MetalakeMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- All entity_change_log INSERTs are in the same DB transaction as the data change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…lementations

- GravitinoCache<K,V>: general-purpose cache interface for auth subsystem
  - getIfPresent, put, invalidate, invalidateAll, invalidateByPrefix, size
  - invalidateByPrefix enables hierarchical cascade invalidation for metadataIdCache
- CaffeineGravitinoCache<K,V>: Caffeine-backed implementation with configurable TTL and max size
- NoOpsGravitinoCache<K,V>: no-op implementation for testing and disabled-cache environments
- TestGravitinoCache: comprehensive tests covering put/get, invalidate, invalidateAll,
  invalidateByPrefix (hierarchical cascade and leaf), overwrite, non-string keys, and NoOps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…izer

Introduce version-validated caches (userRoleCache, groupRoleCache,
loadedRoles with updated_at) and eventual-consistency caches
(metadataIdCache, ownerRelCache) backed by GravitinoCache. Implement
4-step auth flow with group role loading, batch role version check,
and a ScheduledExecutorService poller for owner and entity structural
changes. Add handleEntityStructuralChange to GravitinoAuthorizer
interface and new config entries for metadata-id cache size and
change poll interval.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hange_log pruning

PostgreSQL does not support DELETE ... LIMIT syntax. Override
pruneOldEntityChanges in EntityChangeLogPostgreSQLProvider to use
DELETE ... WHERE id IN (SELECT id ... LIMIT 1000) instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Code Coverage Report

Overall Project 65.23% +0.86% 🟢
Files changed 85.59% 🟢

Module Coverage
aliyun 1.73% 🔴
api 47.09% 🟢
authorization-common 85.96% 🟢
aws 1.1% 🔴
azure 2.6% 🔴
catalog-common 10.2% 🔴
catalog-fileset 80.02% 🟢
catalog-glue 75.36% 🟢
catalog-hive 81.83% 🟢
catalog-jdbc-clickhouse 79.06% 🟢
catalog-jdbc-common 42.89% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.05% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.07% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.16% 🟢
catalog-lakehouse-paimon 77.71% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.63% 🟢
common 48.97% 🟢
core 81.6% +0.99% 🟢
filesystem-hadoop3 76.97% 🟢
flink 40.55% 🟢
flink-runtime 0.0% 🔴
gcp 14.2% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 46.14% 🟢
iceberg-common 50.73% 🟢
iceberg-rest-server 65.93% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 23.88% 🔴
lance-rest-server 57.84% 🟢
lineage 53.02% 🟢
optimizer 82.87% 🟢
optimizer-api 21.95% 🔴
server 85.89% 🟢
server-common 68.11% -1.54% 🟢
spark 32.79% 🔴
spark-common 39.09% 🔴
trino-connector 33.83% 🔴
Files
Module File Coverage
common ConfigConstants.java 0.0% 🔴
core CaffeineGravitinoCache.java 100.0% 🟢
NoOpsGravitinoCache.java 100.0% 🟢
DefaultMapperPackageProvider.java 100.0% 🟢
GroupMetaBaseSQLProvider.java 100.0% 🟢
OwnerMetaBaseSQLProvider.java 100.0% 🟢
RoleMetaBaseSQLProvider.java 100.0% 🟢
UserMetaBaseSQLProvider.java 100.0% 🟢
OwnerRelPO.java 100.0% 🟢
ChangedOwnerInfo.java 100.0% 🟢
EntityChangeRecord.java 100.0% 🟢
OwnerInfo.java 100.0% 🟢
UserAuthInfo.java 100.0% 🟢
MetalakeMetaService.java 100.0% 🟢
CatalogMetaService.java 98.89% 🟢
Configs.java 98.88% 🟢
FilesetMetaService.java 96.83% 🟢
OwnerMetaSQLProviderFactory.java 96.3% 🟢
RoleMetaSQLProviderFactory.java 96.0% 🟢
GroupMetaSQLProviderFactory.java 95.65% 🟢
UserMetaSQLProviderFactory.java 95.65% 🟢
EntityChangeLogSQLProviderFactory.java 94.12% 🟢
GroupMetaService.java 94.03% 🟢
UserMetaService.java 94.03% 🟢
RoleMetaService.java 93.22% 🟢
POConverters.java 88.11% 🟢
RoleUpdatedAt.java 87.5% 🟢
GroupAuthInfo.java 83.33% 🟢
TableMetaService.java 78.52% 🟢
SchemaMetaService.java 78.1% 🟢
ViewMetaService.java 76.15% 🟢
ModelMetaService.java 75.89% 🟢
EntityChangeLogBaseSQLProvider.java 75.0% 🟢
TopicMetaService.java 72.97% 🟢
GravitinoAuthorizer.java 0.0% 🔴
GravitinoCache.java 0.0% 🔴
EntityChangeLogMapper.java 0.0% 🔴
GroupMetaMapper.java 0.0% 🔴
OwnerMetaMapper.java 0.0% 🔴
RoleMetaMapper.java 0.0% 🔴
UserMetaMapper.java 0.0% 🔴
server-common CachedGroupRoles.java 100.0% 🟢
CachedUserRoles.java 100.0% 🟢
JcasbinAuthorizer.java 60.1% 🟢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant