-
Notifications
You must be signed in to change notification settings - Fork 71
feat: add extension-to-functions-codebase migration skill #143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shettyvarun268
wants to merge
3
commits into
firebase:main
Choose a base branch
from
shettyvarun268:migrate-extension-v2
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,136 @@ | ||
| --- | ||
| name: extension-to-functions-codebase | ||
| description: Skill for converting an installed Firebase Extension (or extension source) to a standalone Cloud Functions for Firebase (CF3) codebase, including upgrading triggers from V1 to V2 | ||
| --- | ||
|
|
||
| # Extension to Functions Codebase Migration | ||
|
|
||
| ## Overview | ||
|
|
||
| This skill guides the agent in migrating a Firebase Extension repository or instance into a standalone, self-owned Cloud Functions for Firebase (CF3) codebase. | ||
|
|
||
| It leverages the GA capabilities of the CF3 Fabricator to handle permissions, dependencies, and lifecycle hooks natively in the cloud, and provides instructions for modernizing legacy V1 triggers to V2 using the Destructuring Compatibility Shim. | ||
|
|
||
| --- | ||
|
|
||
| ## Triggers | ||
| Activate this skill when a user asks to migrate or convert an installed Firebase Extension (or extension source code) into a standalone functions codebase. | ||
|
|
||
| --- | ||
|
|
||
| ## Getting Started & Git Safety | ||
|
|
||
| 1. **Git Status**: Verify the workspace has a clean git status before starting. | ||
| 2. **In-Place Copying**: If the user asks to copy the code to a new subdirectory within the same repository: | ||
| * Use `git cp` (or copy files and commit) to copy the extension's source directory to the target functions codebase directory. | ||
| * Commit immediately with the message: | ||
| `"Copying [extension-name] extension to [directory] in preparation for rewrite"` | ||
|
|
||
| --- | ||
|
|
||
| ## Rules and Constraints | ||
|
|
||
| ### 1. Zero-Local-Overhead (CF3 Integration) | ||
| Assume that CF3 Workload Identities and SDK Lifecycle Hooks are fully GA. | ||
| * **Do NOT** output instructions or scripts telling the user to run `gcloud` commands or create service accounts. | ||
| * **Do NOT** write code comments telling the user to manually enable Google APIs in the cloud console. | ||
| * Instead, use declarative `requiresAPI` and `requiresRole` imports from the SDK. | ||
|
|
||
| ### 2. Global Parameter Access Restriction | ||
| * **Never call `.value()` on any parameter at the global scope.** | ||
| * If a global variable or class instance is initialized using a parameter value, declare the variable globally and initialize it inside the `onInit()` callback: | ||
| ```typescript | ||
| const bqDataset = defineString("DATASET_ID"); | ||
| let bqClient: BigQuery; | ||
|
|
||
| onInit(() => { | ||
| bqClient = new BigQuery({ datasetId: bqDataset.value() }); | ||
| }); | ||
| ``` | ||
|
|
||
| ### 3. Concurrency & Cost Parity for V2 | ||
| When upgrading triggers to V2: | ||
| * By default, V2 functions enable concurrency (up to 80 requests per instance). | ||
| * If you want to maintain V1 fractional CPU pricing (and disable concurrency), set `cpu: "gcf_gen1"` in the function's options object. | ||
|
|
||
| --- | ||
|
|
||
| ## Step-by-Step Migration Execution | ||
|
|
||
| ### Step 1: Code Extraction & Package Merge | ||
| 1. Extract the extension's trigger source code (usually located under `functions/src/` or a dedicated source zip) into the targeted codebase directory. | ||
| 2. Merge all dependencies and peer dependencies from the extension's `package.json` into the root `package.json` of the functions project. | ||
| 3. Align Node engines and dependencies: | ||
| * Read the runtime engine specified in the extension's `extension.yaml` (e.g., `nodejs20` maps to `"node": "20"`). | ||
| * Update `package.json` `engines` and `tsconfig.json` compiler options to target this runtime version. | ||
|
|
||
| ### Step 2: Parameterization Mapping | ||
| 1. Read the list of all parameters declared in `extension.yaml`. | ||
| 2. Define a matching parameter in the codebase for each one: | ||
| * If type is `secret`, use `defineSecret('PARAM_NAME')`. | ||
| * Otherwise, use `defineString('PARAM_NAME')` or `defineInt('PARAM_NAME')`. | ||
| 3. Locate all `process.env.PARAM_NAME` calls in the code and replace them with `PARAM_NAME.value()`, ensuring the global scope constraint is respected. | ||
| 4. **Custom Events**: If the extension emits custom events, list them as a `multiSelect` parameter named `events` with the label `"Events to emit"`. The description should be `"Select the events that this function should emit from the following list:"`, listing the events as options in `*[type]*: [description]\n` format. | ||
| 5. Generate a local `.env` file in the functions folder containing the active parameter values for local execution and deployment. | ||
|
|
||
| ### Step 3: Upgrading V1 Functions to V2 (Modernization) | ||
| If the extracted functions are written using the legacy Firebase Functions V1 SDK, upgrade them to V2 to ensure compatibility with CF3 Workload Identities: | ||
|
|
||
| 1. **Imports**: Replace legacy `* as functions` imports with targeted V2 trigger imports from `firebase-functions/v2/...` (e.g. `onDocumentCreated`, `onMessagePublished`). | ||
| 2. **Signature Modernization (Destructuring Shim)**: | ||
| Use the Destructuring Compatibility Shim to preserve internal V1 business logic. Instead of accepting two parameters `(data, context)`, accept a single `CloudEvent` object and destructure `{ [shimmedKey], context }`. | ||
| * *Example (Pub/Sub)*: | ||
| ```typescript | ||
| // V1 Legacy | ||
| export const myFn = functions.pubsub.topic("orders").onPublish((message, context) => { | ||
| const orderId = message.json.id; | ||
| }); | ||
|
|
||
| // V2 + Shim | ||
| import { onMessagePublished } from "firebase-functions/v2/pubsub"; | ||
| export const myFn = onMessagePublished("orders", ({ message, context }) => { | ||
| const orderId = message.json.id; // Logic remains untouched! | ||
| }); | ||
| ``` | ||
| * Refer to [signature-mapping.md](references/signature-mapping.md) for the exact shimmed keys for each trigger type. | ||
| 3. **HTTP Callables**: Callables do not use the destructuring shim. Rewrite the signature to destructure `({ data, auth })` directly. The legacy `context` object is unavailable in V2 callables. | ||
| 4. **Options Migration**: Move trigger configurations (memory, timeouts, secrets) from `.runWith(...)` to the V2 options argument (passed as the first parameter). Refer to [configuration-migration.md](references/configuration-migration.md) for property mapping. | ||
|
|
||
| ### Step 4: Declarative Requirements Injection | ||
| Inject the required IAM roles and APIs at the very top of the main entry point file (e.g., `index.ts`): | ||
| 1. **APIs**: For each service listed in the `apis` field in `extension.yaml`, inject `requiresAPI("service-name.googleapis.com")`. | ||
| 2. **Roles**: For each role listed in the `iamRoles` field in `extension.yaml`, inject `requiresRole("roles/role-name")`. | ||
|
|
||
| *Example Headers*: | ||
| ```typescript | ||
| import { requiresAPI, requiresRole } from "firebase-functions/core"; | ||
|
|
||
| requiresAPI("bigquery.googleapis.com"); | ||
| requiresRole("roles/bigquery.dataEditor"); | ||
| ``` | ||
|
|
||
| ### Step 5: Lifecycle Hook Migration | ||
| If `extension.yaml` contains `lifecycleEvents` (such as `onInstall` / `onUpdate` triggers): | ||
| 1. Ensure the backing function is defined using `functions.tasks.taskQueue().onDispatch(...)` (or `onTaskDispatched` in V2). | ||
| 2. Inject the declarative lifecycle call at the bottom of the entry point: | ||
| ```typescript | ||
| import { afterInstall } from "firebase-functions/lifecycle"; | ||
|
|
||
| afterInstall({ | ||
| task: { function: "myLifecycleTaskFunction" } | ||
| }); | ||
| ``` | ||
|
|
||
| ### Step 6: Firebase Integration & Verification | ||
| 1. Add the newly created functions codebase configuration under the `functions` block in `firebase.json`. | ||
| 2. Run `npm run build` (or `npx tsc`) to verify no compilation or type errors exist. | ||
| 3. **Tests**: If unit tests exist in the codebase: | ||
| * Run `npm test` to verify compliance. | ||
| * *Note*: Upgraded triggers change signature from two arguments `(data, context)` to one destructured object `({ change, context })`. Update test mocks to pass a single object with the correct shimmed key. | ||
|
|
||
| --- | ||
|
|
||
| ## References | ||
| * **Destructuring Shim**: See [destructuring-shim.md](references/destructuring-shim.md) for details on event property translation. | ||
| * **Trigger Mapping**: See [signature-mapping.md](references/signature-mapping.md) for V1 vs V2 trigger definitions and shim keys. | ||
| * **Configuration & Parameters**: See [configuration-migration.md](references/configuration-migration.md) for runWith and parameter definition mappings. | ||
126 changes: 126 additions & 0 deletions
126
skills/extension-to-functions-codebase/references/configuration-migration.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,126 @@ | ||
| # Migrating Runtime Configurations (runWith) | ||
|
|
||
| In Firebase Functions V1, you configured runtime settings like memory, timeout, and service accounts using `.runWith()`. In V2, `.runWith()` is removed and replaced by a more flexible options system. | ||
|
|
||
| You can configure V2 functions in two ways: **Globally** (for all functions in a file) or **Per-Function**. | ||
|
|
||
| --- | ||
|
|
||
| ## 🌍 1. Global Configuration (`setGlobalOptions`) | ||
|
|
||
| Use `setGlobalOptions` at the top of your file to set defaults for all functions defined after it. | ||
|
|
||
| ### V1 Legacy | ||
|
|
||
| ```typescript | ||
| import * as functions from "firebase-functions"; | ||
|
|
||
| export const myFn = functions | ||
| .runWith({ | ||
| memory: "1GB", | ||
| timeoutSeconds: 120, | ||
| serviceAccount: "custom-sa@my-project.iam.gserviceaccount.com", | ||
| }) | ||
| .https.onRequest((req, res) => { ... }); | ||
| ``` | ||
|
|
||
| ### V2 Modern Equivalent | ||
|
|
||
| ```typescript | ||
| import { setGlobalOptions } from "firebase-functions/v2"; | ||
| import { onRequest } from "firebase-functions/v2/https"; | ||
|
|
||
| // Set global defaults for this file | ||
| setGlobalOptions({ | ||
| memory: "1GiB", // Note: GiB instead of GB is preferred in V2 types | ||
| timeoutSeconds: 120, | ||
| serviceAccount: "custom-sa@my-project.iam.gserviceaccount.com", | ||
| }); | ||
|
|
||
| export const myFn = onRequest((req, res) => { ... }); | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 🎯 2. Per-Function Configuration | ||
|
|
||
| Pass the configuration object as the **first argument** to the V2 trigger function. | ||
|
|
||
| ### V1 Legacy | ||
|
|
||
| ```typescript | ||
| export const processOrder = functions | ||
| .runWith({ memory: "2GB" }) | ||
| .pubsub.topic("orders") | ||
| .onPublish((message, context) => { ... }); | ||
| ``` | ||
|
|
||
| ### V2 Modern Equivalent | ||
|
|
||
| ```typescript | ||
| import { onMessagePublished } from "firebase-functions/v2/pubsub"; | ||
|
|
||
| export const processOrder = onMessagePublished( | ||
| { | ||
| topic: "orders", | ||
| memory: "2GiB", // Options passed as the first argument! | ||
| }, | ||
| ({ message, context }) => { ... } // Destructuring shim pattern | ||
| ); | ||
| ``` | ||
|
|
||
| > [!TIP] | ||
| > **Memory Unit Caveat**: V1 accepted `"1GB"`. V2 types strongly prefer IEC units like `"1GiB"`, `"2GiB"`, etc. | ||
|
|
||
| --- | ||
|
|
||
| ## ⚠️ Common Property Translations | ||
|
|
||
| | V1 Property | V2 Property | Notes | | ||
| | :--- | :--- | :--- | | ||
| | `memory` | `memory` | Use `"1GiB"` instead of `"1GB"`. | | ||
| | `timeoutSeconds` | `timeoutSeconds` | Same. | | ||
| | `ingressSettings` | `ingressSettings` | Same. | | ||
| | `vpcConnector` | `vpcConnector` | Same. | | ||
| | `vpcConnectorEgressSettings` | `vpcConnectorEgressSettings` | Same. | | ||
| | `serviceAccount` | `serviceAccount` | Same. | | ||
| | `secrets` | `secrets` | Same. | | ||
| | `failurePolicy` | `retry` | Renamed to boolean `retry: true/false` in V2 Eventarc triggers. | | ||
|
|
||
| --- | ||
|
|
||
| ## 🔐 3. Migrating Environment Configurations (`functions.config()`) | ||
|
|
||
| In V1, you used `functions.config()` to access environment configuration. In V2, this is replaced by **Parameterized Configuration**. | ||
|
|
||
| ### Deterministic Rules for Migration | ||
|
|
||
| Follow these rules to ensure a deterministic and safe migration: | ||
|
|
||
| #### Typing | ||
| * **Numbers**: If the value is used as a number, use `defineNumber`. | ||
| * **Secrets**: If the key contains "KEY", "SECRET", "TOKEN", or "PASSWORD", use `defineSecret()`. | ||
| * *Note*: Secrets MUST be explicitly bound to the function that uses them in the options object (e.g., `{ secrets: [MY_SECRET] }`). | ||
|
shettyvarun268 marked this conversation as resolved.
Outdated
|
||
| * **Lists**: Use `defineList` for comma-separated lists. | ||
| * **JSON**: Use `defineJSON` for JSON strings. | ||
| * **Buckets**: If the param is a storage bucket, set `input: 'BUCKET_PICKER'`. | ||
|
|
||
| #### Initialization & Scope | ||
| * **Global Initialization**: If a variable was initialized globally in V1 (e.g., `const client = new Client(functions.config().key)`), you must split it to have declaration at global scope and initialization inside `onInit`: | ||
| ```typescript | ||
| import { onInit } from "firebase-functions/v2"; | ||
|
|
||
| const myKey = defineSecret("MY_KEY"); | ||
| let client: Client; | ||
|
|
||
| onInit(() => { | ||
| client = new Client(myKey.value()); | ||
| }); | ||
| ``` | ||
|
|
||
| #### Advanced Interpolation & Logic | ||
| * **String Interpolation**: Use the `expr` tagged template literal from `firebase-functions/params` (e.g., `expr`every ${period} days``) instead of standard template literals when constructing dynamic strings with parameters. Do NOT call `.value()` inside `expr`. | ||
|
shettyvarun268 marked this conversation as resolved.
Outdated
|
||
| * **Logic Operators**: Use expressions like `projectID.equals('prod').thenElse(1, 0)` for logical operations instead of ternary operators on `.value()`. | ||
|
|
||
| #### Built-ins | ||
| * Prefer built-in variables like `databaseURL`, `projectID`, `gcloudProject`, `storageBucket` rather than defining new params for these values. | ||
119 changes: 119 additions & 0 deletions
119
skills/extension-to-functions-codebase/references/destructuring-shim.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| # Architectural Deep Dive: Destructuring Compatibility Shim | ||
|
|
||
| The Destructuring Compatibility Shim is a **Zero-Touch Logic Migration** pattern. It allows you to upgrade a function's infrastructure to V2 (and take advantage of GCF 2nd Gen runtimes) without rewriting any of your internal business logic. | ||
|
|
||
| --- | ||
|
|
||
| ## 🛠️ How it Works | ||
|
|
||
| When you migrate a V1 function to V2, the signature changes from two parameters `(data, context)` to a single `CloudEvent` object. | ||
|
|
||
| Instead of manually rewriting all usages of `context.params` or `message.json` inside the function, you use JavaScript's **Object Destructuring** in the signature. | ||
|
|
||
| ### Example Transformation | ||
|
|
||
| #### Step 1: Legacy V1 | ||
|
|
||
| ```typescript | ||
| export const processOrder = functions.pubsub.topic("orders").onPublish((message, context) => { | ||
| const orderId = message.json.id; | ||
| console.log(`Processing order ${orderId} at ${context.timestamp}`); | ||
| }); | ||
| ``` | ||
|
|
||
| #### Step 2: Modern V2 + Shim | ||
|
|
||
| We change the trigger to `onMessagePublished`, and instead of accepting `event`, we destructure `{ message, context }` directly: | ||
|
|
||
| ```typescript | ||
| export const processOrder = onMessagePublished("orders", ({ message, context }) => { | ||
| const orderId = message.json.id; // Legacy logic remains untouched! | ||
| console.log(`Processing order ${orderId} at ${context.timestamp}`); | ||
| }); | ||
| ``` | ||
|
|
||
| ### 🧠 Why This Works | ||
|
|
||
| The Firebase Functions SDK uses a utility called `addV1Compat` to attach these properties via **Lazy Getters** on the `CloudEvent` object for standard event triggers. When you attempt to destructure `{ message, context }` from the event, the SDK transparently maps the V2 event properties back into V1-compatible objects on the fly! This feature is available in modern V2 environments supported by the SDK. | ||
|
|
||
| --- | ||
|
|
||
| ## 📖 Provider Mapping Examples | ||
|
|
||
| Here are the exact destructuring patterns for every supported V2 provider: | ||
|
|
||
| ### 1. Cloud Firestore | ||
|
|
||
| * **Created / Deleted** triggers: | ||
| ```typescript | ||
| // V2: onDocumentCreated, onDocumentDeleted | ||
| export const processDoc = onDocumentCreated("users/{id}", ({ snapshot, context }) => { ... }); | ||
| ``` | ||
| * **Updated / Written** triggers: | ||
| ```typescript | ||
| // V2: onDocumentUpdated, onDocumentWritten | ||
| export const processDoc = onDocumentUpdated("users/{id}", ({ change, context }) => { ... }); | ||
| ``` | ||
|
|
||
| ### 2. Cloud Storage | ||
|
|
||
| * **All** triggers (`onObjectFinalized`, `onObjectDeleted`, `onObjectArchived`, `onObjectMetadataUpdated`): | ||
| ```typescript | ||
| export const processFile = onObjectFinalized(({ object, context }) => { ... }); | ||
| ``` | ||
|
|
||
| ### 3. Realtime Database | ||
|
|
||
| * **Created / Deleted** triggers: | ||
| ```typescript | ||
| export const processData = onValueCreated("/users/{id}", ({ snapshot, context }) => { ... }); | ||
| ``` | ||
| * **Updated / Written** triggers: | ||
| ```typescript | ||
| export const processData = onValueWritten("/users/{id}", ({ change, context }) => { ... }); | ||
| ``` | ||
|
|
||
| ### 4. Remote Config | ||
|
|
||
| * **Updated** triggers: | ||
| ```typescript | ||
| export const processConfig = onConfigUpdated(({ version, context }) => { ... }); | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 🛡️ Best Practices for AI Agents | ||
|
|
||
| 1. **Do NOT attempt to rewrite variable access inside the body.** It is error-prone and time-consuming. | ||
| 2. **Rely on the shim by default.** Only attempt a pure V2 rewrite if the user explicitly asks for "clean-room" modernization. | ||
| 3. **Always type-check (`tsc`) after the rewrite.** If the types are wrong, the shim might not be fully supported for that specific provider yet! | ||
| 4. **HTTPS Callables (Flattened Context)**: Unlike event triggers, Callables do **not** use `V1Compat` or a `context` object. Instead, all context properties are flattened onto the request object. | ||
| * **V1 Priority**: `(data, context) => { ... }` | ||
| * **V2 Equivalent**: `({ data, auth, app }) => { ... }` | ||
|
|
||
| --- | ||
|
|
||
| ## 🔗 Related Migrations | ||
|
|
||
| Migrating event signatures is only one part of moving from V1 to V2. Another critical area is configuration management. | ||
|
|
||
| ### Parameterized Configuration | ||
| If your functions use `functions.config()`, you should migrate to the new **Parameterized Configuration** system in V2. The destructuring shim handles event signatures, but it does not shim `functions.config()`. | ||
|
|
||
| #### How to Migrate: | ||
| 1. **Identify Usages**: Search for `functions.config().path.to.value`. | ||
| 2. **Define Parameters**: At the top of your file, define the parameter using the appropriate primitive from `firebase-functions/params` (available types include `defineString`, `defineSecret`, `defineInt`, `defineBoolean`, `defineList`, and `defineJSON`): | ||
| ```typescript | ||
| import { defineString, defineSecret } from "firebase-functions/params"; | ||
|
|
||
| const stripeKey = defineSecret("STRIPE_KEY"); | ||
| const apiDomain = defineString("API_DOMAIN"); | ||
| ``` | ||
| 3. **Access Values**: Replace the V1 call with the `.value()` method of the defined parameter: | ||
| * **V1**: `const key = functions.config().stripe.key;` | ||
| * **V2**: `const key = stripeKey.value();` | ||
|
|
||
| > [!NOTE] | ||
| > When you migrate a function to V2 (even with the destructuring shim), `functions.config()` will return `undefined` unless you have explicitly set up environment variables or are running in a specific emulation mode. Parameterized configuration is the standard and recommended way to handle this in V2. | ||
|
|
||
| For a complete guide and deterministic rules on how to migrate configurations, refer to [configuration-migration.md](configuration-migration.md). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.