diff --git a/AGENTS.md b/AGENTS.md index 3fa7f99..d27d000 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,6 +57,7 @@ src/core/voice/ STT/TTS orchestration, ffmpeg conversion, cache src/core/gateway/controller.js messenger-neutral gateway controller src/core/opencode/ OpenCode client and server manager src/adapters/telegram/ grammY adapter, auth, media, albums, voice +src/adapters/telegram/stickers* Telegram sticker download, cache, store, rendering helpers tests/ Vitest tests with mocked external services by default ``` @@ -78,6 +79,7 @@ Add modules only when they reduce real complexity. Prefer the smallest correct c - Inline callback data must use short bounded tokens, not raw long session IDs or permission IDs. - Permission prompts must remain text-only, even when voice replies are enabled. - Photo downloads must not expose bot tokens in persisted attachment URLs. +- Sticker cache and saved pack state must not persist bot tokens, raw download URLs, chat IDs, user IDs, or raw Telegram payloads. - Always clean up downloaded media files in `finally` or equivalent cleanup paths. - Keep Telegram UX in the adapter; do not move Telegram reactions, message IDs, chat actions, or grammY types into core. @@ -87,6 +89,7 @@ Add modules only when they reduce real complexity. Prefer the smallest correct c - `telegram.botToken` and `telegram.allowedUserId` are required and must stay private. - Project-local `.opencode-remote/` is ignored because `config.json` contains secrets. - App state is non-secret SQLite data in the platform app-data directory; see `DEVELOPMENT.md` for exact paths. +- Telegram sticker pack state is non-secret adapter state in `telegram-stickers.db`; reusable visuals are disposable cache under `cache/stickers`. - Project state uses OpenCode-style identity: Git remote hash, then cached repo ID, then root commit; non-Git folders use the shared `global` identity. - Do not add model or provider env vars until the related feature is actually implemented. diff --git a/CHANGELOG.md b/CHANGELOG.md index 5da55c3..7413b6e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,13 @@ This project follows Semantic Versioning. ## Unreleased +## [0.6.0] - 2026-05-28 + +### Added + +- Added Telegram sticker understanding with static sticker attachments, generated visual previews for video and animated stickers, reusable sticker visual caching, and saved sticker pack management. (#20) +- Added saved sticker replies for explicit sticker requests and eligible reaction markers, including a safe description catalog built from cached sticker visuals. (#20) + ### Removed - Removed Dependabot configuration to stop automated dependency update pull requests. diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index a66a245..741fb26 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -7,6 +7,7 @@ Development notes for `@crankshift/opencode-remote`. - Node.js 22.18.0 or newer. Node.js 24 LTS is recommended. - pnpm 11.3.0. - Optional voice mode development: local `ffmpeg` for conversion and a Groq API key for live transcription smoke tests. +- Optional sticker development: local `ffmpeg` for video sticker preview smoke tests and python-lottie's `lottie_convert.py` for animated `.tgs` sticker preview smoke tests. ## Install Dependencies @@ -54,6 +55,8 @@ Gateway state is app-managed and stored in a SQLite database named `opencode-rem The database stores non-secret project state such as the active OpenCode session and `/progress` preference. It keys Git projects similarly to OpenCode: Git remote identity first, then a cached repo ID, then root commit. Non-Git folders use a shared global project identity. Generated voice files are cache under the same app-data root at `cache/voice` and can be removed with `opencode-remote cache clear`. +Telegram sticker pack state is stored separately in `telegram-stickers.db` under the same app-data directory. Sticker visual cache files live under `cache/stickers`. Sticker state stores Telegram `file_unique_id`, current reusable `file_id`, pack name, emoji, dimensions, type, optional safe visual descriptions, and cache metadata. It must not store bot tokens, raw Telegram download URLs, chat IDs, user IDs, raw update payloads, or temp paths. + Use `opencode-remote run --state-suffix dev` to use `opencode-remote-dev.db` instead of the normal state database. The source `pnpm dev` script uses this to keep development state separate from regular gateway state. ## Runtime Internals diff --git a/FEATURES.md b/FEATURES.md index 14ff329..4264977 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1,6 +1,6 @@ # Features -OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, and opt-in voice support. +OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, sticker, and opt-in voice support. ## Available Now @@ -18,6 +18,7 @@ OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, a - Interactive JSON config setup with project-local and global config discovery, selected-scope current defaults, highlighted arrow-key lists, and `ffmpeg` install/retry handling for voice setup. - SQLite app-state persistence for selected OpenCode sessions and progress preferences, scoped by OpenCode project identity. - Optional Telegram voice mode using Groq Whisper transcription, Edge TTS speech generation, and `ffmpeg` OGG Opus conversion. +- Telegram sticker understanding with static WebP sticker attachments, generated or fallback visual context for non-static stickers, saved sticker packs, and sticker replies. - CLI config updates with `opencode-remote config set` and voice cache clearing with `opencode-remote cache clear`. ## Telegram Chat Behavior @@ -28,6 +29,7 @@ OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, a - `/stop` requests abort for the active OpenCode session. - `/progress` shows or sets prompt activity visibility: `off`, `new`, `all`, or `verbose`. - `/voice` shows and controls voice mode, lists voices by required short country/locale filter, sets the active Edge TTS voice, and sends a test voice note. +- `/stickers` saves, lists, and forgets sticker packs for future sticker replies. - `/help` shows the available bot commands. - The Telegram slash-command menu is refreshed on gateway startup. - Non-command text from the authorized user is sent to OpenCode as a prompt. @@ -37,6 +39,8 @@ OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, a - OpenCode permission requests are sent as text with `Allow once`, `Always allow`, and `Deny` buttons, even when voice replies are enabled. - Incoming text prompts get a temporary eye reaction while processing. - OpenCode can request one Telegram emoji reaction by returning a hidden `[telegram_reaction: ...]` marker, which is removed before the user sees the reply. +- When saved sticker packs are available, eligible hidden reaction markers may be answered with a saved sticker reply instead of an emoji reaction. +- When saved sticker packs are available, explicit user requests for a sticker can be answered with a saved sticker reply through a hidden gateway marker. - User emoji reactions to recent bot messages are sent back to OpenCode as feedback prompts. - Telegram voice messages are transcribed and sent to OpenCode when voice mode is enabled. - Voice replies replace text replies after voice prompts in `/voice on` mode and after text, photo, and voice prompts in `/voice all` mode, with text fallback if speech generation or sending fails. @@ -56,6 +60,10 @@ OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, a - Album captions become the prompt text when present. - Photos without captions use a default short reaction prompt. - Temporary downloaded photo files are cleaned up after handling. +- Telegram sticker messages are sent to OpenCode with visual attachment context and safe sticker metadata. +- Static stickers use direct WebP image attachments. Video stickers use sampled preview sheets. Animated `.tgs` stickers use `lottie_convert.py` when available, with source-file fallback. +- Sticker visuals are cached under app-data cache storage and validated with `file_unique_id`, kind, dimensions, file size, and converter version. +- Cached sticker visuals can be summarized into short safe descriptions for the saved-sticker catalog used by future sticker replies. ## Voice Mode @@ -71,6 +79,7 @@ OpenCode Remote is currently a Telegram gateway for OpenCode with text, image, a - The bot ignores Telegram users outside the configured allowlist. - Secrets are configured through private `.opencode-remote/config.json` files, not persisted settings. - The selected active session is persisted as non-secret JSON state. +- Saved sticker packs persist only non-secret sticker identifiers and metadata. - Telegram reaction API failures are best-effort warnings and do not block prompt delivery. - Default tests mock Telegram and OpenCode; no live services are required for normal verification. diff --git a/README.md b/README.md index 5cc9290..e160dc5 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ OpenCode Remote lets you use OpenCode from Telegram. It runs on your machine, connects to your local or remote OpenCode server, and forwards messages from one authorized Telegram user to OpenCode sessions. -This is a Telegram MVP with text prompts, photo prompts, OpenCode permission approvals, and opt-in voice input/replies. Model switching and multi-messenger support are not implemented yet. +This is a Telegram MVP with text prompts, photo prompts, sticker prompts/replies, OpenCode permission approvals, and opt-in voice input/replies. Model switching and multi-messenger support are not implemented yet. See [Features](https://github.com/crankshift/opencode-remote/blob/main/FEATURES.md) for the full current capability list, [Contributing](https://github.com/crankshift/opencode-remote/blob/main/CONTRIBUTING.md) for contribution guidance, [Changelog](https://github.com/crankshift/opencode-remote/blob/main/CHANGELOG.md) for release notes, and [TODO](https://github.com/crankshift/opencode-remote/blob/main/TODO.md) for planned work. @@ -166,6 +166,7 @@ The bot currently supports: /stop Request stop for the active OpenCode session /progress Show or set tool progress visibility: off, new, all, verbose /voice Show or set voice mode +/stickers Manage saved sticker packs /help Show available commands ``` @@ -179,6 +180,18 @@ When OpenCode requests permission during a prompt, the bot sends a text message Telegram photo albums are handled as one OpenCode prompt when Telegram provides a shared `media_group_id`. The album caption becomes the prompt text. Separate text messages sent after an album are treated as separate prompts. +Telegram stickers are downloaded as visual prompt context for OpenCode. Static stickers are sent as WebP image attachments. Video stickers use `ffmpeg` to generate sampled preview sheets. Animated `.tgs` stickers use `lottie_convert.py` from python-lottie when it is available, with a source-file fallback if conversion is not installed. The gateway caches reusable sticker visuals under app-data cache storage, keyed by Telegram `file_unique_id` and safe visual metadata. When possible, cached sticker visuals are summarized into short saved-sticker descriptions so future sticker requests can use a compact text catalog instead of exposing cache paths or Telegram file identifiers. + +Sticker pack commands: + +```text +/stickers save +/stickers list +/stickers forget +``` + +Use `/stickers save` as a reply to a sticker to save that sticker pack for future sticker replies. `/stickers list` shows saved packs. `/stickers forget ` removes a saved pack and its cached sticker previews. Incoming stickers from unsaved packs may also show a `Save pack` button. Once packs are saved, asking the bot to send a sticker lets OpenCode request one through the gateway without exposing Telegram file identifiers to the model. Saved sticker data is non-secret Telegram file metadata; bot tokens, user IDs, chat IDs, and raw download URLs are not persisted. + Voice commands: ```text diff --git a/docs/superpowers/plans/2026-05-28-telegram-sticker-support.md b/docs/superpowers/plans/2026-05-28-telegram-sticker-support.md new file mode 100644 index 0000000..1efbfc3 --- /dev/null +++ b/docs/superpowers/plans/2026-05-28-telegram-sticker-support.md @@ -0,0 +1,93 @@ +# Telegram Sticker Support Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add Telegram sticker visual understanding, sticker preview caching, saved sticker pack management, and optional sticker replies for OpenCode reaction markers. + +**Architecture:** Keep all Telegram sticker specifics inside `src/adapters/telegram`. OpenCode continues receiving messenger-neutral prompt objects with file attachments and text context. Static stickers are attached directly as WebP images, while video/animated stickers use cached preview images. + +**Tech Stack:** Node.js ESM, grammY, SQLite via `node:sqlite`, `ffmpeg` through injectable process wrappers, optional python-lottie `lottie_convert.py`, Vitest, Biome. + +--- + +## File Structure + +- Create `src/adapters/telegram/stickerCache.js` for app-data sticker cache paths, cache validation, and cached file cleanup. +- Create `src/adapters/telegram/stickerStore.js` for SQLite-backed saved packs, seen stickers, and cached preview index records. +- Create `src/adapters/telegram/stickerRenderer.js` for static/video/animated representation helpers with injectable conversion functions. +- Create `src/adapters/telegram/stickers.js` for Telegram sticker download, prompt text formatting, and attachment orchestration. +- Modify `src/adapters/telegram/bot.js` for `/stickers`, save callbacks, `message:sticker`, and sticker-vs-emoji reply behavior. +- Modify `src/core/commands/commands.js` so command registration, help text, docs, and tests share `/stickers` from the central command source. +- Modify `src/runtime/bootstrap.js` to open/close the sticker store and pass sticker dependencies into the Telegram bot. +- Modify docs: `README.md`, `FEATURES.md`, and `DEVELOPMENT.md`. +- Add tests under `tests/adapters`, `tests/core`, and `tests/runtime`. + +## Tasks + +### Task 1: Sticker Command Definition + +- [ ] Add a failing assertion in `tests/core/commands.test.js` that `/stickers` appears in command definitions and help text. +- [ ] Run `pnpm test tests/core/commands.test.js` and verify the new assertion fails because `/stickers` is missing. +- [ ] Add `/stickers` to `src/core/commands/commands.js` with description `Manage saved sticker packs`. +- [ ] Run `pnpm test tests/core/commands.test.js` and verify it passes. + +### Task 2: Sticker Cache Helpers + +- [ ] Add failing tests in `tests/adapters/telegramStickerCache.test.js` for default cache directory, cache record validation, missing-file invalidation, metadata mismatch invalidation, and cached file cleanup. +- [ ] Run `pnpm test tests/adapters/telegramStickerCache.test.js` and verify failures are for missing exports. +- [ ] Implement `src/adapters/telegram/stickerCache.js` with `STICKER_CONVERTER_VERSION`, `getStickerCacheDir`, `isStickerCacheRecordUsable`, `cachedStickerFilePath`, and `removeCachedStickerFiles`. +- [ ] Run `pnpm test tests/adapters/telegramStickerCache.test.js` and verify it passes. + +### Task 3: Sticker Store + +- [ ] Add failing tests in `tests/adapters/telegramStickerStore.test.js` for saved-pack upsert, list, forget, match-by-emoji selection, fallback selection, seen-sticker metadata, cache records, and no secret fields. +- [ ] Run `pnpm test tests/adapters/telegramStickerStore.test.js` and verify failures are for missing exports. +- [ ] Implement `src/adapters/telegram/stickerStore.js` with `openTelegramStickerStore` and `createMemoryStickerStore` for tests and dependency injection. +- [ ] Run `pnpm test tests/adapters/telegramStickerStore.test.js` and verify it passes. + +### Task 4: Sticker Renderer And Prompt Attachments + +- [ ] Add failing tests in `tests/adapters/telegramStickers.test.js` for static WebP direct attachment, cached preview reuse, cache mismatch regeneration, thumbnail fallback, metadata prompt text, and temp cleanup. +- [ ] Run `pnpm test tests/adapters/telegramStickers.test.js` and verify failures are for missing exports. +- [ ] Implement `src/adapters/telegram/stickerRenderer.js` with static direct representation and injectable video/animated preview generation. +- [ ] Implement `src/adapters/telegram/stickers.js` with `downloadTelegramSticker`, `createStickerPrompt`, `formatStickerPromptText`, and `saveStickerPackFromSet`. +- [ ] Run `pnpm test tests/adapters/telegramStickers.test.js` and verify it passes. + +### Task 5: Telegram Bot Sticker Input And Commands + +- [ ] Add failing tests in `tests/adapters/telegramBot.test.js` for handler registration, static sticker prompt delivery, inline save-pack button, `/stickers save`, `/stickers list`, and `/stickers forget `. +- [ ] Run `pnpm test tests/adapters/telegramBot.test.js` and verify failures are for missing sticker behavior. +- [ ] Modify `src/adapters/telegram/bot.js` to accept sticker dependencies, register `message:sticker`, register `/stickers`, tokenize save callbacks, and call sticker helper/store methods. +- [ ] Run `pnpm test tests/adapters/telegramBot.test.js` and verify it passes. + +### Task 6: Sticker Replies For Reaction Markers + +- [ ] Add failing tests in `tests/adapters/telegramBot.test.js` for randomized sticker-vs-emoji selection, emoji-matched sticker preference, fallback to emoji reaction on sticker send failure, and unchanged incoming eye reaction. +- [ ] Run `pnpm test tests/adapters/telegramBot.test.js` and verify failures are for missing reply selection behavior. +- [ ] Modify `src/adapters/telegram/bot.js` so parsed reaction markers use `maybeSendStickerReaction` after visible replies, with injectable randomness for deterministic tests. +- [ ] Run `pnpm test tests/adapters/telegramBot.test.js` and verify it passes. + +### Task 7: Runtime Wiring + +- [ ] Add failing tests in `tests/runtime/bootstrap.test.js` that runtime opens a sticker store, passes it to `createTelegramBot`, and closes it on shutdown. +- [ ] Run `pnpm test tests/runtime/bootstrap.test.js` and verify failures are for missing runtime wiring. +- [ ] Modify `src/runtime/bootstrap.js` to create the sticker store and close it during shutdown. +- [ ] Run `pnpm test tests/runtime/bootstrap.test.js` and verify it passes. + +### Task 8: Docs + +- [ ] Update `README.md` to document sticker messages, inline save, and `/stickers save|list|forget`. +- [ ] Update `FEATURES.md` to list shipped sticker support and app-data sticker cache behavior. +- [ ] Update `DEVELOPMENT.md` to document sticker cache/state paths and mocked tests. +- [ ] Run `pnpm run lint` to verify docs formatting. + +### Task 9: Final Verification + +- [ ] Run `pnpm test`. +- [ ] Run `pnpm run lint`. +- [ ] Run `pnpm run check`. +- [ ] Inspect `git status --short` and `git diff` for unintended files, secrets, raw Telegram payloads, bot tokens, user IDs, and local machine paths. + +## Self-Review + +This plan covers all issue 20 acceptance criteria: sticker visual attachments, static direct handling, animated/video preview caching, saved pack identifiers, unchanged eye reactions, randomized sticker replies, text-only permission prompts, Telegram adapter boundaries, and mocked default tests. It avoids speculative multi-user behavior and keeps sticker persistence non-secret. diff --git a/docs/superpowers/specs/2026-05-28-telegram-sticker-support-design.md b/docs/superpowers/specs/2026-05-28-telegram-sticker-support-design.md new file mode 100644 index 0000000..ccfcfed --- /dev/null +++ b/docs/superpowers/specs/2026-05-28-telegram-sticker-support-design.md @@ -0,0 +1,99 @@ +# Telegram Sticker Support Design + +## Goal + +Add Telegram sticker understanding and sticker replies so OpenCode receives meaningful visual sticker context, while the gateway can remember user-approved sticker packs for future replies. + +## Decisions + +- Static Telegram stickers are sent to OpenCode as downloaded `image/webp` attachments with no animation parsing. +- Video stickers use a generated visual preview contact sheet so OpenCode receives image context rather than only metadata. +- Animated `.tgs` stickers use `lottie_convert.py` from python-lottie when available, with source-file fallback when conversion is unavailable. +- Generated previews are cached under the opencode-remote app-data cache, not in OpenCode settings or project files. +- Cache identity is based on Telegram `file_unique_id`, sticker kind, dimensions, optional `file_size`, and a local converter version. +- `file_id` is stored only as the current Telegram handle for downloading or re-sending, never as the sticker identity. +- Saved sticker packs are controlled through Telegram UI: inline `Save pack`, `/stickers save`, `/stickers list`, and `/stickers forget `. +- Sticker pack state stores only non-secret Telegram file identifiers and metadata. It does not store bot tokens, download URLs, chat IDs, user IDs, raw Telegram payloads, or local temp paths. +- Existing incoming-message eye reactions remain unchanged. +- OpenCode permission prompts remain text-only. +- Default tests mock Telegram, OpenCode, file conversion, and network behavior. + +## Architecture + +Sticker behavior stays in the Telegram adapter. Core gateway and OpenCode modules continue to exchange messenger-neutral prompt objects with `text`, `author`, and `attachments`. + +```text +src/adapters/telegram/stickerCache.js app-data sticker cache paths, cache validation, and preview file cleanup +src/adapters/telegram/stickerStore.js SQLite-backed saved packs and seen sticker metadata +src/adapters/telegram/stickerRenderer.js static/video/animated representation helpers with injectable conversion +src/adapters/telegram/stickers.js Telegram sticker download, metadata formatting, and prompt attachment orchestration +src/adapters/telegram/bot.js commands, callbacks, message handlers, and sticker-vs-emoji reply selection +``` + +Runtime wiring opens a Telegram sticker store beside existing project state and passes sticker dependencies into `createTelegramBot`. + +## Sticker Understanding Flow + +```text +Telegram sticker message + -> adapter stores seen sticker metadata + -> adapter checks cache using file_unique_id and visual metadata + -> static sticker downloads WebP and attaches it directly + -> video or animated sticker reuses or creates a contact-sheet preview image + -> prompt text includes sticker emoji, pack name, type, dimensions, and representation details + -> OpenCode receives file attachment plus metadata text + -> temporary downloads are cleaned up +``` + +If video or animated preview generation fails, the adapter logs a warning and falls back to Telegram's sticker thumbnail when available. If no visual attachment can be produced, the bot sends a safe short Telegram reply instead of sending metadata-only sticker understanding as a successful prompt. + +## Cache Validation + +A cached preview is reusable only when all checks pass: + +```text +same file_unique_id +same sticker kind: static, video, or animated +same width and height +same file_size when Telegram provides it +same converter version +cached preview file exists +``` + +If a sticker pack author replaces a sticker, Telegram should send a different `file_unique_id`, so the gateway treats it as a new sticker and regenerates the cached visual. + +## Sticker Pack Commands + +```text +/stickers save +/stickers list +/stickers forget +``` + +- `/stickers save` must be used as a reply to a sticker. It saves the sticker's pack with `getStickerSet` when available, falling back to the replied sticker if Telegram cannot return the full set. +- `/stickers list` shows saved pack names, sticker counts, and a compact emoji summary. +- `/stickers forget ` removes the saved pack from reply eligibility and deletes cached preview files associated with that pack. +- Incoming stickers with an unsaved `set_name` include an inline `Save pack` button. Callback data uses short bounded tokens, not raw pack names. + +## Sticker Replies + +The existing hidden OpenCode marker remains the model contract: + +```text +[telegram_reaction: πŸ‘] +``` + +When a marker is present and saved stickers exist, the adapter randomly chooses between the existing emoji reaction behavior and a sticker reply. Sticker selection prefers a saved sticker whose `emoji` matches the requested reaction. If no match exists, any saved sticker may be used. If sending the sticker fails, the bot falls back to the emoji reaction. + +The temporary incoming `πŸ‘€` reaction for text prompts remains unchanged and is not replaced by sticker behavior. + +## Error Handling + +- Telegram reaction and sticker-send failures are best-effort warnings and must not block prompt delivery. +- Sticker download/render failures use safe Telegram replies and log details without leaking tokens or raw provider bodies. +- Permission prompts continue to use plain text plus inline buttons, even when sticker packs are saved. +- Cache cleanup failures are logged as warnings. + +## Self-Review Notes + +The design keeps Telegram details inside the adapter, uses direct static sticker images without unnecessary parsing, caches expensive generated previews, and makes pack persistence explicit through user commands or an inline button. diff --git a/package.json b/package.json index 82d1575..a45b6a2 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "@crankshift/opencode-remote", "description": "A messenger-based chat interface for OpenCode, starting with Telegram.", - "version": "0.5.7", + "version": "0.6.0", "license": "MIT", "repository": { "type": "git", diff --git a/src/adapters/telegram/bot.js b/src/adapters/telegram/bot.js index 80fbdf3..142f98b 100644 --- a/src/adapters/telegram/bot.js +++ b/src/adapters/telegram/bot.js @@ -1,3 +1,4 @@ +import { rm } from "node:fs/promises" import { Bot, InlineKeyboard } from "grammy" import { botCommands, renderHelpText } from "../../core/commands/commands.js" import { chunkText } from "../../core/formatting/chunkText.js" @@ -15,6 +16,10 @@ import { selectLargestPhoto, } from "./media.js" import { createMediaGroupBuffer } from "./mediaGroupBuffer.js" +import { + createStickerPrompt as defaultCreateStickerPrompt, + stickerToStoreMetadata, +} from "./stickers.js" import { downloadTelegramVoice as defaultDownloadVoice, sendTelegramVoice as defaultSendVoice, @@ -51,11 +56,16 @@ export function createTelegramBot({ sendVoice = defaultSendVoice, cleanupMediaAttachments = defaultCleanupMediaAttachments, voiceService = null, + stickerStore = null, + createStickerPrompt = defaultCreateStickerPrompt, + cleanupStickerFiles = defaultCleanupStickerFiles, + random = Math.random, }) { const bot = new botFactory(token) let fallbackProgressVerbosity = progressVerbosity const sessionSelectionTokens = new Map() const permissionResponseTokens = createBoundedTokenStore(200) + const stickerSaveTokens = createBoundedTokenStore(200) const botMessageMemory = createBotMessageMemory(200) const mediaGroupBuffer = createMediaGroupBuffer({ waitMs: mediaGroupWaitMs, @@ -157,6 +167,21 @@ export function createTelegramBot({ } }) + bot.callbackQuery(/^sticker_save:(.+)$/u, async (ctx) => { + const sticker = stickerSaveTokens.get(ctx.match[1]) + if (!sticker) { + await ctx.answerCallbackQuery({ text: "Sticker save request expired" }) + return + } + + stickerSaveTokens.delete(ctx.match[1]) + const result = await saveStickerPackFromSticker(ctx, sticker) + await ctx.answerCallbackQuery({ text: "Sticker pack saved" }) + if (ctx.reply) { + await replyAndRemember(ctx, formatStickerSaveResult(result), botMessageMemory) + } + }) + bot.command("stop", async (ctx) => { const result = await controller.stop() if (!result.stopped) { @@ -238,6 +263,10 @@ export function createTelegramBot({ await replyAndRemember(ctx, voiceUsageText(), botMessageMemory) }) + bot.command("stickers", async (ctx) => { + await handleStickersCommand(ctx) + }) + bot.on("message_reaction", async (ctx) => { const update = ctx.messageReaction const botMessage = botMessageMemory.get(update.chat.id, update.message_id) @@ -249,14 +278,12 @@ export function createTelegramBot({ for (const emoji of addedEmojis) { const progress = await createPromptProgressRenderer(ctx) const response = await sendPromptWithProgress( - formatPromptWithTelegramReactionInstruction( - formatReactionFeedbackPrompt(emoji, botMessage), - ), + await formatPromptForTelegramGateway(formatReactionFeedbackPrompt(emoji, botMessage)), progress, ctx, ) await progress.flush() - const { visibleText } = parseTelegramReactionMarker(response, progress) + const { visibleText } = parseTelegramGatewayMarkers(response, progress) for (const chunk of chunkText(visibleText)) { await replyAndRemember(ctx, chunk, botMessageMemory) } @@ -276,7 +303,7 @@ export function createTelegramBot({ try { await setEmojiReaction(ctx, chatId, messageId, "πŸ‘€", logger) const response = await sendPromptWithProgress( - formatPromptWithTelegramReactionInstruction({ + await formatPromptForTelegramGateway({ text: ctx.message.text, author: authorContextFromTelegramMessage(ctx.message), }), @@ -284,16 +311,21 @@ export function createTelegramBot({ ctx, ) await progress.flush() - const parsedResponse = parseTelegramReactionMarker(response, progress) + const parsedResponse = parseTelegramGatewayMarkers(response, progress) requestedReaction = parsedResponse.requestedReaction + const requestedSticker = parsedResponse.requestedSticker await replyWithPreferredMode(ctx, parsedResponse.visibleText, "text") + if (requestedSticker) { + await sendRequestedSticker(ctx, requestedSticker) + requestedReaction = null + } } finally { await progress.flush() await clearMessageReaction(ctx, chatId, messageId, logger) stopTyping() } if (requestedReaction) { - await setEmojiReaction(ctx, chatId, messageId, requestedReaction, logger) + await handleRequestedReaction(ctx, chatId, messageId, requestedReaction) } }) @@ -310,6 +342,10 @@ export function createTelegramBot({ await handleVoiceMessage(ctx) }) + bot.on("message:sticker", async (ctx) => { + await handleStickerMessage(ctx) + }) + return bot async function handlePhotoMessages(ctx, messages) { @@ -339,7 +375,7 @@ export function createTelegramBot({ const progress = await createPromptProgressRenderer(ctx) const response = await sendPromptWithProgress( - formatPromptWithTelegramReactionInstruction({ + await formatPromptForTelegramGateway({ text: captionFromMessages(messages), author: authorContextFromTelegramMessage(messages[0]), attachments, @@ -348,15 +384,16 @@ export function createTelegramBot({ ctx, ) await progress.flush() - const parsedResponse = parseTelegramReactionMarker(response, progress) + const parsedResponse = parseTelegramGatewayMarkers(response, progress) await replyWithPreferredMode(ctx, parsedResponse.visibleText, "photo") - if (parsedResponse.requestedReaction) { - await setEmojiReaction( + if (parsedResponse.requestedSticker) { + await sendRequestedSticker(ctx, parsedResponse.requestedSticker) + } else if (parsedResponse.requestedReaction) { + await handleRequestedReaction( ctx, messages[0]?.chat?.id, messages[0]?.message_id, parsedResponse.requestedReaction, - logger, ) } } finally { @@ -403,7 +440,7 @@ export function createTelegramBot({ const transcript = await voiceService.transcribe(attachment.filePath) const progress = await createPromptProgressRenderer(ctx) const response = await sendPromptWithProgress( - formatPromptWithTelegramReactionInstruction({ + await formatPromptForTelegramGateway({ text: transcript, author: authorContextFromTelegramMessage(ctx.message), }), @@ -411,15 +448,65 @@ export function createTelegramBot({ ctx, ) await progress.flush() - const parsedResponse = parseTelegramReactionMarker(response, progress) + const parsedResponse = parseTelegramGatewayMarkers(response, progress) await replyWithPreferredMode(ctx, parsedResponse.visibleText, "voice") + if (parsedResponse.requestedSticker) { + await sendRequestedSticker(ctx, parsedResponse.requestedSticker) + } } finally { stopTyping() await cleanupMediaAttachments(attachments, logger) } } + async function handleStickerMessage(ctx) { + let cleanupFiles = [] + const stopTyping = startTypingIndicator(ctx, logger) + try { + const result = await createStickerPrompt({ + api: ctx.api, + token, + sticker: ctx.message.sticker, + store: stickerStore, + logger, + describeStickerVisual, + }) + cleanupFiles = result.cleanupFiles ?? [] + + const progress = await createPromptProgressRenderer(ctx) + const response = await sendPromptWithProgress( + await formatPromptForTelegramGateway({ + ...result.prompt, + author: authorContextFromTelegramMessage(ctx.message), + }), + progress, + ctx, + ) + await progress.flush() + const parsedResponse = parseTelegramGatewayMarkers(response, progress) + await replyWithPreferredMode(ctx, parsedResponse.visibleText, "sticker") + if (parsedResponse.requestedSticker) { + await sendRequestedSticker(ctx, parsedResponse.requestedSticker) + } else if (parsedResponse.requestedReaction) { + await handleRequestedReaction( + ctx, + ctx.message?.chat?.id, + ctx.message?.message_id, + parsedResponse.requestedReaction, + ) + } + await offerSaveStickerPack(ctx, ctx.message.sticker, result.packName) + } finally { + stopTyping() + await cleanupStickerFiles(cleanupFiles, logger) + } + } + async function replyWithPreferredMode(ctx, text, source) { + if (!String(text ?? "").trim()) { + return + } + if (!voiceService?.shouldSpeak?.({ source })) { await sendTextReply(ctx, text) return @@ -442,6 +529,155 @@ export function createTelegramBot({ } } + async function handleRequestedReaction(ctx, chatId, messageId, emoji) { + if (await maybeSendStickerReaction(ctx, emoji)) { + return + } + await setEmojiReaction(ctx, chatId, messageId, emoji, logger) + } + + async function maybeSendStickerReaction(ctx, emoji) { + if (!stickerStore || random() >= 0.5) { + return false + } + const sticker = await stickerStore.findStickerForEmoji(emoji, { random }) + if (!sticker?.fileId) { + return false + } + + return sendStickerFileId(ctx, sticker.fileId, "Could not send Telegram sticker reaction") + } + + async function sendRequestedSticker(ctx, selector) { + if (!stickerStore) { + return false + } + const requestedEmoji = normalizeStickerSelector(selector) + const sticker = + typeof stickerStore.findStickerForSelector === "function" + ? await stickerStore.findStickerForSelector(requestedEmoji, { random }) + : await stickerStore.findStickerForEmoji(requestedEmoji, { random }) + if (!sticker?.fileId) { + return false + } + return sendStickerFileId(ctx, sticker.fileId, "Could not send Telegram sticker reply") + } + + async function sendStickerFileId(ctx, fileId, warningMessage) { + try { + if (typeof ctx.replyWithSticker === "function") { + await ctx.replyWithSticker(fileId) + } else { + const chatId = ctx.chat?.id ?? ctx.message?.chat?.id ?? ctx.messageReaction?.chat?.id + if (!chatId || typeof ctx.api?.sendSticker !== "function") { + return false + } + await ctx.api.sendSticker(chatId, fileId) + } + return true + } catch (error) { + logger.warn({ error }, warningMessage) + return false + } + } + + async function handleStickersCommand(ctx) { + if (!stickerStore) { + await replyAndRemember(ctx, "Sticker support is not configured.", botMessageMemory) + return + } + + const request = parseStickersCommand(ctx.message?.text) + if (request.action === "save") { + const sticker = ctx.message?.reply_to_message?.sticker + if (!sticker) { + await replyAndRemember(ctx, "Reply to a sticker with /stickers save.", botMessageMemory) + return + } + if (!sticker.set_name) { + await replyAndRemember( + ctx, + "That sticker does not belong to a saveable sticker pack.", + botMessageMemory, + ) + return + } + const result = await saveStickerPackFromSticker(ctx, sticker) + await replyAndRemember(ctx, formatStickerSaveResult(result), botMessageMemory) + return + } + + if (request.action === "list") { + await replyAndRemember( + ctx, + formatSavedStickerPacks(await stickerStore.listPacks()), + botMessageMemory, + ) + return + } + + if (request.action === "forget") { + if (!request.packName) { + await replyAndRemember(ctx, "Use /stickers forget .", botMessageMemory) + return + } + const result = await stickerStore.forgetPack(request.packName) + await cleanupStickerFiles( + result.cacheRecords.map((record) => record.filePath), + logger, + ) + await replyAndRemember( + ctx, + result.deleted + ? `Forgot sticker pack ${request.packName}.` + : `No saved sticker pack named ${request.packName}.`, + botMessageMemory, + ) + return + } + + await replyAndRemember(ctx, stickersUsageText(), botMessageMemory) + } + + async function saveStickerPackFromSticker(ctx, sticker) { + if (!sticker?.set_name) { + throw new Error("Sticker does not belong to a saveable sticker pack") + } + + let packName = sticker.set_name + let stickers = [sticker] + try { + const stickerSet = await ctx.api?.getStickerSet?.(sticker.set_name) + packName = stickerSet?.name ?? packName + if (Array.isArray(stickerSet?.stickers) && stickerSet.stickers.length > 0) { + stickers = stickerSet.stickers + } + } catch (error) { + logger.warn({ error, packName }, "Could not fetch Telegram sticker set") + } + + await stickerStore.savePack({ + name: packName, + stickers: stickers.map(stickerToStoreMetadata), + }) + return { packName, stickerCount: stickers.length } + } + + async function offerSaveStickerPack(ctx, sticker, packName) { + if (!stickerStore || !packName || (await stickerStore.hasSavedPack(packName))) { + return + } + + const token = stickerSaveTokens.add(sticker) + const keyboard = new InlineKeyboard().text("Save pack", `sticker_save:${token}`) + await replyAndRemember( + ctx, + `Sticker pack ${packName} is not saved. Save it for future sticker replies?`, + botMessageMemory, + { reply_markup: keyboard }, + ) + } + async function createPromptProgressRenderer(ctx) { return createTelegramProgressRenderer({ ctx, @@ -451,6 +687,17 @@ export function createTelegramBot({ }) } + async function formatPromptForTelegramGateway(prompt) { + return formatPromptWithTelegramGatewayInstructions(prompt, { stickerStore, logger }) + } + + async function describeStickerVisual({ sticker, attachment, visualDescription }) { + return controller.sendPrompt({ + text: formatStickerDescriptionRequest(sticker, visualDescription), + attachments: [attachment], + }) + } + async function sendPromptWithProgress(prompt, progress, ctx) { const promptOptions = createPromptOptions(progress, ctx) if (promptOptions === undefined) { @@ -560,6 +807,40 @@ function parseVoiceCommand(text) { return { action } } +function parseStickersCommand(text) { + const parts = String(text ?? "") + .trim() + .split(/\s+/u) + .filter(Boolean) + const action = parts[1] ?? "list" + if (action === "forget") { + return { action, packName: parts[2] } + } + return { action } +} + +function formatStickerSaveResult(result) { + const stickerWord = result.stickerCount === 1 ? "sticker" : "stickers" + return `Saved sticker pack ${result.packName} (${result.stickerCount} ${stickerWord}).` +} + +function formatSavedStickerPacks(packs) { + if (!packs.length) { + return "No sticker packs saved. Reply to a sticker with /stickers save." + } + return ["Saved sticker packs:", ...packs.map(formatSavedStickerPack)].join("\n") +} + +function formatSavedStickerPack(pack) { + const stickerWord = pack.stickerCount === 1 ? "sticker" : "stickers" + const emojiSummary = pack.emojis.length > 0 ? `, ${pack.emojis.join(" ")}` : "" + return `- ${pack.name} (${pack.stickerCount} ${stickerWord}${emojiSummary})` +} + +function stickersUsageText() { + return "Use /stickers save, /stickers list, or /stickers forget ." +} + function parseVoiceListFilters(parts) { if (parts.length < 1 || parts.length > 2) { return null @@ -795,6 +1076,7 @@ async function replyAndRemember(ctx, text, botMessageMemory, options) { } const TELEGRAM_REACTION_MARKER = /\[telegram_reaction:\s*([^\]\n]+?)\s*\]/giu +const TELEGRAM_STICKER_MARKER = /\[telegram_sticker:\s*([^\]\n]+?)\s*\]/giu const TELEGRAM_REACTION_INSTRUCTION = [ "Telegram gateway note:", @@ -804,29 +1086,129 @@ const TELEGRAM_REACTION_INSTRUCTION = [ "Use only one standard Telegram emoji, and omit the marker when no reaction is useful. The marker will be removed before the user sees the reply.", ].join("\n") -function formatPromptWithTelegramReactionInstruction(prompt) { +async function formatPromptWithTelegramGatewayInstructions(prompt, { stickerStore, logger } = {}) { + const instructions = [TELEGRAM_REACTION_INSTRUCTION] + const stickerInstruction = await createTelegramStickerReplyInstruction(stickerStore, logger) + if (stickerInstruction) { + instructions.push(stickerInstruction) + } + return appendPromptInstruction(prompt, instructions.join("\n\n")) +} + +function appendPromptInstruction(prompt, instruction) { if (typeof prompt !== "string") { return { ...prompt, - text: [String(prompt?.text ?? ""), "", TELEGRAM_REACTION_INSTRUCTION].join("\n"), + text: [String(prompt?.text ?? ""), "", instruction].join("\n"), } } - return [prompt, "", TELEGRAM_REACTION_INSTRUCTION].join("\n") + return [prompt, "", instruction].join("\n") +} + +async function createTelegramStickerReplyInstruction(stickerStore, logger) { + if (typeof stickerStore?.listPacks !== "function") { + return null + } + + let packs + try { + packs = await stickerStore.listPacks() + } catch (error) { + logger?.warn?.({ error }, "Could not list saved Telegram sticker packs") + return null + } + + if (!Array.isArray(packs) || packs.length === 0) { + return null + } + + const emojis = savedStickerEmojis(packs) + const catalog = await readStickerCatalog(stickerStore, logger) + const exampleEmoji = emojis[0] ?? "any" + return [ + "Telegram sticker reply capability:", + "If the user explicitly asks for a sticker, include exactly one hidden marker anywhere in your response:", + `[telegram_sticker: ${exampleEmoji}]`, + "Use an emoji or short sticker description from the available saved sticker catalog when it matches the requested mood, or use [telegram_sticker: any]. The marker will be removed before the user sees the reply.", + `Available saved sticker packs: ${formatStickerInstructionPacks(packs)}`, + `Available saved sticker emojis: ${emojis.length > 0 ? emojis.join(" ") : "any"}`, + ...(catalog.length > 0 + ? ["Available saved sticker catalog:", ...catalog.map(formatStickerCatalogItem)] + : []), + ].join("\n") +} + +async function readStickerCatalog(stickerStore, logger) { + if (typeof stickerStore?.listStickerCatalog !== "function") { + return [] + } + try { + return (await stickerStore.listStickerCatalog()).filter((sticker) => sticker.description) + } catch (error) { + logger?.warn?.({ error }, "Could not list saved Telegram sticker catalog") + return [] + } +} + +function formatStickerCatalogItem(sticker) { + const packName = sticker.packName ?? "saved sticker" + const emoji = sticker.emoji ? `${sticker.emoji} ` : "" + return `- ${emoji}${packName}: ${sticker.description}` +} + +function savedStickerEmojis(packs) { + return [...new Set(packs.flatMap((pack) => pack.emojis ?? []).filter(Boolean))] +} + +function formatStickerInstructionPacks(packs) { + return packs.map(formatStickerInstructionPack).join(", ") } -function parseTelegramReactionMarker(text, progress) { +function formatStickerInstructionPack(pack) { + const emojis = + Array.isArray(pack.emojis) && pack.emojis.length > 0 ? ` (${pack.emojis.join(" ")})` : "" + return `${pack.name}${emojis}` +} + +function parseTelegramGatewayMarkers(text, progress) { let requestedReaction = null - const visibleText = String(text).replace(TELEGRAM_REACTION_MARKER, (_match, emoji) => { - requestedReaction ??= emoji.trim() - return "" - }) + let requestedSticker = null + const visibleText = String(text) + .replace(TELEGRAM_REACTION_MARKER, (_match, emoji) => { + requestedReaction ??= emoji.trim() + return "" + }) + .replace(TELEGRAM_STICKER_MARKER, (_match, sticker) => { + requestedSticker ??= normalizeStickerSelector(sticker) + return "" + }) return { visibleText: stripToolingAnnouncements(visibleText, progress?.toolingTerms), requestedReaction, + requestedSticker, } } +function normalizeStickerSelector(selector) { + return String(selector ?? "").trim() || "any" +} + +function formatStickerDescriptionRequest(sticker, visualDescription) { + return [ + "Gateway internal task: describe this Telegram sticker for a saved sticker catalog.", + "Use the attached cached sticker visual or preview.", + "Return only a short lowercase noun phrase of 2 to 6 words.", + "Do not include IDs, file paths, markdown, quotes, or hidden gateway markers.", + "Focus on visible content and mood, for example: laughing orange cat, thumbs up duck, angry wizard.", + "", + "Sticker metadata:", + `- Sticker emoji: ${sticker?.emoji ?? "unknown"}`, + `- Sticker pack: ${sticker?.set_name ?? "none"}`, + `- Sticker visual: ${visualDescription}`, + ].join("\n") +} + function stripToolingAnnouncements(text, toolingTerms = new Set()) { const lines = String(text) .trim() @@ -889,6 +1271,19 @@ const TOOLING_ANNOUNCEMENT_CONTEXT_PATTERNS = [ /^Π²ΠΈΠΊΠΎΡ€ΠΈΡΡ‚ΠΎΠ²ΡƒΡŽ\s+.+(?:skill|Π½Π°Π²ΠΈΡ‡ΠΊ|інструмСнт|для\b).*$/iu, ] +async function defaultCleanupStickerFiles(filePaths = [], logger) { + for (const filePath of filePaths) { + if (!filePath) { + continue + } + try { + await rm(filePath, { force: true }) + } catch (error) { + logger?.warn?.({ error, filePath }, "Could not clean up Telegram sticker file") + } + } +} + async function setEmojiReaction(ctx, chatId, messageId, emoji, logger) { if (!chatId || !messageId || !ctx.api?.setMessageReaction) { return diff --git a/src/adapters/telegram/stickerCache.js b/src/adapters/telegram/stickerCache.js new file mode 100644 index 0000000..b7f6c57 --- /dev/null +++ b/src/adapters/telegram/stickerCache.js @@ -0,0 +1,109 @@ +import { access, mkdir, rm } from "node:fs/promises" +import { posix, win32 } from "node:path" +import { getAppDataDir } from "../../core/state/appDataPath.js" + +export const STICKER_CONVERTER_VERSION = "1" + +export function getStickerCacheDir(options = {}) { + const platform = options.platform ?? process.platform + const pathApi = platform === "win32" ? win32 : posix + return pathApi.join(getAppDataDir({ ...options, platform }), "cache", "stickers") +} + +export function cachedStickerFilePath(sticker, options = {}) { + const directory = options.directory ?? getStickerCacheDir(options) + const platform = options.platform ?? process.platform + const pathApi = platform === "win32" ? win32 : posix + const fileUniqueId = sanitizeFilePart(sticker?.fileUniqueId ?? sticker?.file_unique_id) + const kind = sanitizeFilePart(sticker?.kind ?? stickerKind(sticker)) + const extension = sanitizeExtension(sticker?.extension ?? "png") + return pathApi.join( + directory, + `${fileUniqueId}-${kind}-v${STICKER_CONVERTER_VERSION}.${extension}`, + ) +} + +export async function ensureStickerCacheDir(options = {}) { + const directory = options.directory ?? getStickerCacheDir(options) + await mkdir(directory, { recursive: true }) + return directory +} + +export async function isStickerCacheRecordUsable({ + sticker, + record, + converterVersion = STICKER_CONVERTER_VERSION, + accessFn = access, +} = {}) { + if (!sticker || !record?.filePath) { + return false + } + if (record.fileUniqueId !== sticker.file_unique_id) { + return false + } + if (record.kind !== stickerKind(sticker)) { + return false + } + if ( + Number(record.width) !== Number(sticker.width) || + Number(record.height) !== Number(sticker.height) + ) { + return false + } + if (sticker.file_size !== undefined && Number(record.fileSize) !== Number(sticker.file_size)) { + return false + } + if (record.converterVersion !== converterVersion) { + return false + } + + try { + await accessFn(record.filePath) + return true + } catch { + return false + } +} + +export async function removeCachedStickerFiles(records = [], { logger, rmFn = rm } = {}) { + for (const record of records) { + const filePath = record?.filePath + if (!filePath) { + continue + } + + try { + await rmFn(filePath, { force: true }) + } catch (error) { + logger?.warn?.({ error, filePath }, "Could not remove cached Telegram sticker file") + } + } +} + +export function stickerKind(sticker) { + if (sticker?.is_video) { + return "video" + } + if (sticker?.is_animated) { + return "animated" + } + return "static" +} + +function sanitizeFilePart(value) { + const safe = String(value ?? "sticker") + .trim() + .replace(/[^a-zA-Z0-9._-]+/g, "-") + .replace(/\.{2,}/g, "-") + .replace(/-+/g, "-") + .replace(/^-+|-+$/g, "") + return safe || "sticker" +} + +function sanitizeExtension(value) { + const safe = String(value ?? "png") + .replace(/^\.+/u, "") + .replace(/[^a-zA-Z0-9]+/g, "") + .toLowerCase() + return safe || "png" +} diff --git a/src/adapters/telegram/stickerRenderer.js b/src/adapters/telegram/stickerRenderer.js new file mode 100644 index 0000000..c1685d1 --- /dev/null +++ b/src/adapters/telegram/stickerRenderer.js @@ -0,0 +1,28 @@ +import { execa as defaultExeca } from "execa" + +export async function renderVideoStickerPreview({ + inputPath, + outputPath, + execa = defaultExeca, +} = {}) { + await execa("ffmpeg", [ + "-y", + "-i", + inputPath, + "-vf", + "fps=2,scale=256:-1,tile=3x2", + "-frames:v", + "1", + outputPath, + ]) + return { mime: "image/png", filePath: outputPath } +} + +export async function renderAnimatedStickerPreview({ + inputPath, + outputPath, + execa = defaultExeca, +} = {}) { + await execa("lottie_convert.py", [inputPath, outputPath]) + return { mime: "image/png", filePath: outputPath } +} diff --git a/src/adapters/telegram/stickerStore.js b/src/adapters/telegram/stickerStore.js new file mode 100644 index 0000000..da186ba --- /dev/null +++ b/src/adapters/telegram/stickerStore.js @@ -0,0 +1,586 @@ +import { mkdirSync } from "node:fs" +import { dirname, posix, win32 } from "node:path" +import { DatabaseSync } from "node:sqlite" +import { getAppDataDir } from "../../core/state/appDataPath.js" + +const STICKER_DB_FILE_NAME = "telegram-stickers.db" + +export function getDefaultTelegramStickerDbPath(options = {}) { + const platform = options.platform ?? process.platform + const pathApi = platform === "win32" ? win32 : posix + return pathApi.join(getAppDataDir({ ...options, platform }), STICKER_DB_FILE_NAME) +} + +export function openTelegramStickerStore( + dbPath, + { Database = DatabaseSync, pathOptions = {} } = {}, +) { + const resolvedDbPath = dbPath ?? getDefaultTelegramStickerDbPath(pathOptions) + mkdirSync(dirname(resolvedDbPath), { recursive: true }) + const database = new Database(resolvedDbPath) + initialize(database) + + return { + path: resolvedDbPath, + async savePack(pack) { + savePack(database, pack) + }, + async listPacks() { + return listPacks(database) + }, + async listStickerCatalog() { + return listStickerCatalog(listSavedStickers(database)) + }, + async hasSavedPack(name) { + return Boolean(getSavedPack(database, name)) + }, + async forgetPack(name) { + return forgetPack(database, name) + }, + async findStickerForEmoji(emoji, options) { + return findStickerForEmoji(listSavedStickers(database), emoji, options) + }, + async findStickerForSelector(selector, options) { + return findStickerForSelector(listSavedStickers(database), selector, options) + }, + async upsertSeenSticker(sticker) { + upsertSeenSticker(database, sticker) + }, + async getSeenSticker(fileUniqueId) { + return rowToSticker(getSeenSticker(database, fileUniqueId)) + }, + async updateStickerDescription(fileUniqueId, description) { + updateStickerDescription(database, fileUniqueId, description) + }, + async writeCacheRecord(record) { + writeCacheRecord(database, record) + }, + async readCacheRecord(fileUniqueId, kind) { + return rowToCacheRecord(getCacheRecord(database, fileUniqueId, kind)) + }, + close() { + database.close() + }, + } +} + +export function createMemoryStickerStore() { + const packs = new Map() + const stickers = new Map() + const seen = new Map() + const cacheRecords = new Map() + + return { + async savePack(pack) { + const name = normalizePackName(pack?.name) + const normalizedStickers = normalizeStickerList(pack?.stickers, name) + packs.set(name, { name }) + for (const sticker of normalizedStickers) { + const stickerWithDescription = preserveDescription( + sticker, + stickers.get(sticker.fileUniqueId) ?? seen.get(sticker.fileUniqueId), + ) + stickers.set(sticker.fileUniqueId, stickerWithDescription) + seen.set(sticker.fileUniqueId, stickerWithDescription) + } + }, + async listPacks() { + return listPackSummaries([...packs.keys()], [...stickers.values()]) + }, + async listStickerCatalog() { + return listStickerCatalog([...stickers.values()]) + }, + async hasSavedPack(name) { + return packs.has(name) + }, + async forgetPack(name) { + if (!packs.has(name)) { + return { deleted: false, cacheRecords: [] } + } + const forgottenIds = [...stickers.values()] + .filter((sticker) => sticker.packName === name) + .map((sticker) => sticker.fileUniqueId) + const records = [...cacheRecords.values()].filter( + (record) => record.packName === name || forgottenIds.includes(record.fileUniqueId), + ) + packs.delete(name) + for (const fileUniqueId of forgottenIds) { + stickers.delete(fileUniqueId) + } + for (const record of records) { + cacheRecords.delete(cacheKey(record.fileUniqueId, record.kind)) + } + return { deleted: true, cacheRecords: records } + }, + async findStickerForEmoji(emoji, options) { + return findStickerForEmoji([...stickers.values()], emoji, options) + }, + async findStickerForSelector(selector, options) { + return findStickerForSelector([...stickers.values()], selector, options) + }, + async upsertSeenSticker(sticker) { + const normalized = normalizeSticker(sticker) + const previous = seen.get(normalized.fileUniqueId) + seen.set(normalized.fileUniqueId, preserveDescription(normalized, previous)) + }, + async getSeenSticker(fileUniqueId) { + return seen.get(fileUniqueId) ?? null + }, + async updateStickerDescription(fileUniqueId, description) { + const normalizedDescription = normalizeDescription(description) + for (const collection of [stickers, seen]) { + const sticker = collection.get(fileUniqueId) + if (sticker) { + collection.set(fileUniqueId, { ...sticker, description: normalizedDescription }) + } + } + }, + async writeCacheRecord(record) { + const normalized = normalizeCacheRecord(record) + cacheRecords.set(cacheKey(normalized.fileUniqueId, normalized.kind), normalized) + }, + async readCacheRecord(fileUniqueId, kind) { + return cacheRecords.get(cacheKey(fileUniqueId, kind)) ?? null + }, + close() {}, + } +} + +function initialize(database) { + database.exec(` + PRAGMA foreign_keys = ON; + PRAGMA journal_mode = WAL; + PRAGMA busy_timeout = 5000; + + CREATE TABLE IF NOT EXISTS saved_pack ( + name TEXT PRIMARY KEY, + time_created INTEGER NOT NULL, + time_updated INTEGER NOT NULL + ) STRICT; + + CREATE TABLE IF NOT EXISTS saved_sticker ( + file_unique_id TEXT PRIMARY KEY, + pack_name TEXT NOT NULL REFERENCES saved_pack(name) ON DELETE CASCADE, + file_id TEXT NOT NULL, + emoji TEXT, + description TEXT, + kind TEXT NOT NULL, + width INTEGER, + height INTEGER, + file_size INTEGER, + time_created INTEGER NOT NULL, + time_updated INTEGER NOT NULL + ) STRICT; + + CREATE TABLE IF NOT EXISTS seen_sticker ( + file_unique_id TEXT PRIMARY KEY, + pack_name TEXT, + file_id TEXT NOT NULL, + emoji TEXT, + description TEXT, + kind TEXT NOT NULL, + width INTEGER, + height INTEGER, + file_size INTEGER, + time_created INTEGER NOT NULL, + time_updated INTEGER NOT NULL + ) STRICT; + + CREATE TABLE IF NOT EXISTS sticker_cache ( + file_unique_id TEXT NOT NULL, + kind TEXT NOT NULL, + pack_name TEXT, + width INTEGER, + height INTEGER, + file_size INTEGER, + converter_version TEXT NOT NULL, + file_path TEXT NOT NULL, + time_created INTEGER NOT NULL, + time_updated INTEGER NOT NULL, + PRIMARY KEY (file_unique_id, kind) + ) STRICT; + `) + ensureColumn(database, "saved_sticker", "description", "TEXT") + ensureColumn(database, "seen_sticker", "description", "TEXT") +} + +function ensureColumn(database, table, column, definition) { + const columns = database.prepare(`PRAGMA table_info(${table})`).all() + if (columns.some((row) => row.name === column)) { + return + } + database.exec(`ALTER TABLE ${table} ADD COLUMN ${column} ${definition}`) +} + +function savePack(database, pack) { + const name = normalizePackName(pack?.name) + const stickers = normalizeStickerList(pack?.stickers, name) + const now = Date.now() + database + .prepare( + `INSERT INTO saved_pack (name, time_created, time_updated) + VALUES (?, ?, ?) + ON CONFLICT(name) DO UPDATE SET time_updated = excluded.time_updated`, + ) + .run(name, now, now) + + for (const sticker of stickers) { + const previousSeenSticker = rowToSticker(getSeenSticker(database, sticker.fileUniqueId)) + const stickerWithDescription = preserveDescription(sticker, previousSeenSticker) + upsertSeenSticker(database, stickerWithDescription) + database + .prepare( + `INSERT INTO saved_sticker + (file_unique_id, pack_name, file_id, emoji, description, kind, width, height, file_size, time_created, time_updated) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(file_unique_id) DO UPDATE SET + pack_name = excluded.pack_name, + file_id = excluded.file_id, + emoji = excluded.emoji, + description = COALESCE(excluded.description, saved_sticker.description), + kind = excluded.kind, + width = excluded.width, + height = excluded.height, + file_size = excluded.file_size, + time_updated = excluded.time_updated`, + ) + .run( + stickerWithDescription.fileUniqueId, + stickerWithDescription.packName, + stickerWithDescription.fileId, + stickerWithDescription.emoji, + stickerWithDescription.description, + stickerWithDescription.kind, + stickerWithDescription.width, + stickerWithDescription.height, + stickerWithDescription.fileSize, + now, + now, + ) + } +} + +function listPacks(database) { + const packNames = database + .prepare("SELECT name FROM saved_pack ORDER BY name") + .all() + .map((row) => row.name) + return listPackSummaries(packNames, listSavedStickers(database)) +} + +function getSavedPack(database, name) { + return database.prepare("SELECT name FROM saved_pack WHERE name = ?").get(name) +} + +function forgetPack(database, name) { + if (!getSavedPack(database, name)) { + return { deleted: false, cacheRecords: [] } + } + const cacheRecords = database + .prepare( + `SELECT file_unique_id, kind, pack_name, width, height, file_size, converter_version, file_path + FROM sticker_cache + WHERE pack_name = ? OR file_unique_id IN ( + SELECT file_unique_id FROM saved_sticker WHERE pack_name = ? + )`, + ) + .all(name, name) + .map(rowToCacheRecord) + database.prepare("DELETE FROM saved_pack WHERE name = ?").run(name) + for (const record of cacheRecords) { + database + .prepare("DELETE FROM sticker_cache WHERE file_unique_id = ? AND kind = ?") + .run(record.fileUniqueId, record.kind) + } + return { deleted: true, cacheRecords } +} + +function listSavedStickers(database) { + return database + .prepare( + `SELECT file_unique_id, pack_name, file_id, emoji, description, kind, width, height, file_size + FROM saved_sticker + ORDER BY rowid`, + ) + .all() + .map(rowToSticker) +} + +function upsertSeenSticker(database, sticker) { + const normalized = normalizeSticker(sticker) + const now = Date.now() + database + .prepare( + `INSERT INTO seen_sticker + (file_unique_id, pack_name, file_id, emoji, description, kind, width, height, file_size, time_created, time_updated) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(file_unique_id) DO UPDATE SET + pack_name = excluded.pack_name, + file_id = excluded.file_id, + emoji = excluded.emoji, + description = COALESCE(excluded.description, seen_sticker.description), + kind = excluded.kind, + width = excluded.width, + height = excluded.height, + file_size = excluded.file_size, + time_updated = excluded.time_updated`, + ) + .run( + normalized.fileUniqueId, + normalized.packName, + normalized.fileId, + normalized.emoji, + normalized.description, + normalized.kind, + normalized.width, + normalized.height, + normalized.fileSize, + now, + now, + ) +} + +function getSeenSticker(database, fileUniqueId) { + return database + .prepare( + `SELECT file_unique_id, pack_name, file_id, emoji, description, kind, width, height, file_size + FROM seen_sticker WHERE file_unique_id = ?`, + ) + .get(fileUniqueId) +} + +function updateStickerDescription(database, fileUniqueId, description) { + const normalizedDescription = normalizeDescription(description) + database + .prepare("UPDATE seen_sticker SET description = ?, time_updated = ? WHERE file_unique_id = ?") + .run(normalizedDescription, Date.now(), fileUniqueId) + database + .prepare("UPDATE saved_sticker SET description = ?, time_updated = ? WHERE file_unique_id = ?") + .run(normalizedDescription, Date.now(), fileUniqueId) +} + +function writeCacheRecord(database, record) { + const normalized = normalizeCacheRecord(record) + const now = Date.now() + database + .prepare( + `INSERT INTO sticker_cache + (file_unique_id, kind, pack_name, width, height, file_size, converter_version, file_path, time_created, time_updated) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + ON CONFLICT(file_unique_id, kind) DO UPDATE SET + pack_name = excluded.pack_name, + width = excluded.width, + height = excluded.height, + file_size = excluded.file_size, + converter_version = excluded.converter_version, + file_path = excluded.file_path, + time_updated = excluded.time_updated`, + ) + .run( + normalized.fileUniqueId, + normalized.kind, + normalized.packName, + normalized.width, + normalized.height, + normalized.fileSize, + normalized.converterVersion, + normalized.filePath, + now, + now, + ) +} + +function getCacheRecord(database, fileUniqueId, kind) { + return database + .prepare( + `SELECT file_unique_id, kind, pack_name, width, height, file_size, converter_version, file_path + FROM sticker_cache WHERE file_unique_id = ? AND kind = ?`, + ) + .get(fileUniqueId, kind) +} + +function listPackSummaries(packNames, stickers) { + return packNames.sort().map((name) => { + const packStickers = stickers.filter((sticker) => sticker.packName === name) + return { + name, + stickerCount: packStickers.length, + emojis: [...new Set(packStickers.map((sticker) => sticker.emoji).filter(Boolean))], + } + }) +} + +function listStickerCatalog(stickers) { + return stickers.map((sticker) => ({ + packName: sticker.packName, + emoji: sticker.emoji, + description: sticker.description, + })) +} + +function findStickerForEmoji(stickers, emoji, { random = Math.random } = {}) { + const matching = stickers.filter((sticker) => sticker.emoji === emoji) + const candidates = matching.length > 0 ? matching : stickers + if (candidates.length === 0) { + return null + } + const index = Math.min(candidates.length - 1, Math.floor(random() * candidates.length)) + return candidates[index] +} + +function findStickerForSelector(stickers, selector, { random = Math.random } = {}) { + const normalizedSelector = normalizeSelector(selector) + if (!normalizedSelector || normalizedSelector === "any") { + return pickSticker(stickers, random) + } + + const matchingEmoji = stickers.filter((sticker) => sticker.emoji === selector) + if (matchingEmoji.length > 0) { + return pickSticker(matchingEmoji, random) + } + + const matchingDescription = stickers.filter((sticker) => + normalizeSelector(sticker.description).includes(normalizedSelector), + ) + if (matchingDescription.length > 0) { + return pickSticker(matchingDescription, random) + } + + return pickSticker(stickers, random) +} + +function pickSticker(stickers, random) { + if (stickers.length === 0) { + return null + } + const index = Math.min(stickers.length - 1, Math.floor(random() * stickers.length)) + return stickers[index] +} + +function normalizeStickerList(stickers, packName) { + return [...(stickers ?? [])].map((sticker) => normalizeSticker({ ...sticker, packName })) +} + +function normalizeSticker(sticker) { + const fileUniqueId = firstString(sticker?.fileUniqueId, sticker?.file_unique_id) + const fileId = firstString(sticker?.fileId, sticker?.file_id) + if (!fileUniqueId || !fileId) { + throw new Error("Sticker metadata requires fileUniqueId and fileId") + } + return { + fileUniqueId, + fileId, + packName: firstString(sticker?.packName, sticker?.set_name) ?? null, + emoji: firstString(sticker?.emoji) ?? null, + description: normalizeDescription(sticker?.description), + kind: firstString(sticker?.kind) ?? "static", + width: numberOrNull(sticker?.width), + height: numberOrNull(sticker?.height), + fileSize: numberOrNull(sticker?.fileSize, sticker?.file_size), + } +} + +function normalizeCacheRecord(record) { + const fileUniqueId = firstString(record?.fileUniqueId, record?.file_unique_id) + const kind = firstString(record?.kind) + const filePath = firstString(record?.filePath, record?.file_path) + if (!fileUniqueId || !kind || !filePath) { + throw new Error("Sticker cache records require fileUniqueId, kind, and filePath") + } + return { + fileUniqueId, + kind, + packName: firstString(record?.packName, record?.pack_name) ?? null, + width: numberOrNull(record?.width), + height: numberOrNull(record?.height), + fileSize: numberOrNull(record?.fileSize, record?.file_size), + converterVersion: firstString(record?.converterVersion, record?.converter_version) ?? "1", + filePath, + } +} + +function rowToSticker(row) { + if (!row) { + return null + } + return { + fileUniqueId: row.file_unique_id, + fileId: row.file_id, + packName: row.pack_name ?? null, + emoji: row.emoji ?? null, + description: row.description ?? null, + kind: row.kind, + width: row.width ?? null, + height: row.height ?? null, + fileSize: row.file_size ?? null, + } +} + +function rowToCacheRecord(row) { + if (!row) { + return null + } + return { + fileUniqueId: row.file_unique_id, + kind: row.kind, + packName: row.pack_name ?? null, + width: row.width ?? null, + height: row.height ?? null, + fileSize: row.file_size ?? null, + converterVersion: row.converter_version, + filePath: row.file_path, + } +} + +function preserveDescription(sticker, previousSticker) { + return { + ...sticker, + description: normalizeDescription(sticker.description) ?? previousSticker?.description ?? null, + } +} + +function normalizeDescription(description) { + const value = firstString(description) + if (!value) { + return null + } + return value.replace(/\s+/gu, " ").slice(0, 160) +} + +function normalizeSelector(selector) { + return String(selector ?? "") + .trim() + .replace(/\s+/gu, " ") + .toLocaleLowerCase("en-US") +} + +function normalizePackName(name) { + const value = firstString(name) + if (!value) { + throw new Error("Sticker pack name is required") + } + return value +} + +function firstString(...values) { + for (const value of values) { + if (typeof value === "string" && value.trim()) { + return value.trim() + } + } + return null +} + +function numberOrNull(...values) { + for (const value of values) { + const number = Number(value) + if (Number.isFinite(number)) { + return number + } + } + return null +} + +function cacheKey(fileUniqueId, kind) { + return `${fileUniqueId}:${kind}` +} diff --git a/src/adapters/telegram/stickers.js b/src/adapters/telegram/stickers.js new file mode 100644 index 0000000..81e2faf --- /dev/null +++ b/src/adapters/telegram/stickers.js @@ -0,0 +1,311 @@ +import { randomUUID } from "node:crypto" +import { mkdir, writeFile } from "node:fs/promises" +import { tmpdir } from "node:os" +import { dirname, extname, join } from "node:path" +import { pathToFileURL } from "node:url" +import { + cachedStickerFilePath, + isStickerCacheRecordUsable, + STICKER_CONVERTER_VERSION, + stickerKind, +} from "./stickerCache.js" +import { + renderAnimatedStickerPreview as defaultRenderAnimatedStickerPreview, + renderVideoStickerPreview as defaultRenderVideoStickerPreview, +} from "./stickerRenderer.js" + +export async function downloadTelegramSticker({ + api, + token, + sticker, + directory, + destinationPath, + fetchFn = fetch, +} = {}) { + const file = await api.getFile(sticker.file_id) + if (!file?.file_path) { + throw new Error("Telegram did not return a file path for the sticker") + } + + const downloadUrl = `https://api.telegram.org/file/bot${token}/${file.file_path}` + const response = await fetchFn(downloadUrl) + if (!response.ok) { + throw new Error(`Could not download Telegram sticker (${response.status})`) + } + + const mime = stickerMime(sticker, file.file_path) + const filePath = + destinationPath ?? join(directory, `telegram-sticker-${randomUUID()}${extensionForMime(mime)}`) + const buffer = Buffer.from(await response.arrayBuffer()) + + await mkdir(dirname(filePath), { recursive: true }) + await writeFile(filePath, buffer) + + return { + mime, + url: pathToFileURL(filePath).href, + filePath, + } +} + +export async function createStickerPrompt({ + api, + token, + sticker, + store, + cacheDirectory, + mediaDirectory = tmpdir(), + fetchFn = fetch, + logger, + renderVideoStickerPreview = defaultRenderVideoStickerPreview, + renderAnimatedStickerPreview = defaultRenderAnimatedStickerPreview, + describeStickerVisual = null, +} = {}) { + const kind = stickerKind(sticker) + const stickerMeta = stickerToStoreMetadata(sticker) + await store?.upsertSeenSticker?.(stickerMeta) + + const cacheRecord = await store?.readCacheRecord?.(sticker.file_unique_id, kind) + if (await isStickerCacheRecordUsable({ sticker, record: cacheRecord })) { + const mime = kind === "static" ? "image/webp" : "image/png" + return finalizeStickerPromptResult({ + sticker, + attachment: attachmentFromFile(cacheRecord.filePath, mime), + visualDescription: kind === "static" ? "static WebP image" : "cached preview", + cleanupFiles: [], + store, + describeStickerVisual, + logger, + }) + } + + const extension = kind === "static" ? "webp" : "png" + const outputPath = cachedStickerFilePath( + { fileUniqueId: sticker.file_unique_id, kind, extension }, + { directory: cacheDirectory }, + ) + + if (kind === "static") { + const attachment = await downloadTelegramSticker({ + api, + token, + sticker, + destinationPath: outputPath, + fetchFn, + }) + await writeStickerCacheRecord(store, sticker, outputPath, kind) + return finalizeStickerPromptResult({ + sticker, + attachment, + visualDescription: "static WebP image", + cleanupFiles: [], + store, + describeStickerVisual, + logger, + }) + } + + const downloaded = await downloadTelegramSticker({ + api, + token, + sticker, + directory: mediaDirectory, + fetchFn, + }) + const cleanupFiles = [downloaded.filePath] + + try { + await mkdir(dirname(outputPath), { recursive: true }) + const rendered = + kind === "video" + ? await renderVideoStickerPreview({ inputPath: downloaded.filePath, outputPath, sticker }) + : await renderAnimatedStickerPreview({ + inputPath: downloaded.filePath, + outputPath, + sticker, + }) + const attachment = attachmentFromFile( + rendered.filePath ?? outputPath, + rendered.mime ?? "image/png", + ) + await writeStickerCacheRecord(store, sticker, attachment.filePath, kind) + return finalizeStickerPromptResult({ + sticker, + attachment, + visualDescription: `${kind} sticker sampled preview`, + cleanupFiles, + store, + describeStickerVisual, + logger, + }) + } catch (error) { + logger?.warn?.({ error, kind }, "Could not render Telegram sticker preview") + return finalizeStickerPromptResult({ + sticker, + attachment: downloaded, + visualDescription: `${kind} sticker source file`, + cleanupFiles, + store, + describeStickerVisual, + logger, + }) + } +} + +export function formatStickerPromptText(sticker, visualDescription) { + return [ + "React to this Telegram sticker as the Telegram bot persona.", + "Use the attached visual sticker content and the metadata below.", + "Do not describe the sticker in detail unless the user asks what is in it.", + "Keep the reply short, funny, and chatty.", + "", + "Sticker metadata:", + `- Sticker emoji: ${sticker.emoji ?? "unknown"}`, + `- Sticker pack: ${sticker.set_name ?? "none"}`, + `- Sticker type: ${stickerKind(sticker)}`, + `- Sticker dimensions: ${Number(sticker.width ?? 0)}x${Number(sticker.height ?? 0)}`, + `- Sticker visual: ${visualDescription}`, + ].join("\n") +} + +export function stickerToStoreMetadata(sticker) { + return { + fileUniqueId: sticker.file_unique_id, + fileId: sticker.file_id, + packName: sticker.set_name ?? null, + emoji: sticker.emoji ?? null, + kind: stickerKind(sticker), + width: sticker.width ?? null, + height: sticker.height ?? null, + fileSize: sticker.file_size ?? null, + } +} + +async function writeStickerCacheRecord(store, sticker, filePath, kind) { + await store?.writeCacheRecord?.({ + fileUniqueId: sticker.file_unique_id, + packName: sticker.set_name ?? null, + kind, + width: sticker.width ?? null, + height: sticker.height ?? null, + fileSize: sticker.file_size ?? null, + converterVersion: STICKER_CONVERTER_VERSION, + filePath, + }) +} + +function stickerPromptResult({ sticker, attachment, visualDescription, cleanupFiles }) { + return { + prompt: { + text: formatStickerPromptText(sticker, visualDescription), + attachments: [attachment], + }, + packName: sticker.set_name ?? null, + fileUniqueId: sticker.file_unique_id, + cleanupFiles, + } +} + +async function finalizeStickerPromptResult({ + sticker, + attachment, + visualDescription, + cleanupFiles, + store, + describeStickerVisual, + logger, +}) { + await maybeStoreStickerDescription({ + sticker, + attachment, + visualDescription, + store, + describeStickerVisual, + logger, + }) + return stickerPromptResult({ sticker, attachment, visualDescription, cleanupFiles }) +} + +async function maybeStoreStickerDescription({ + sticker, + attachment, + visualDescription, + store, + describeStickerVisual, + logger, +}) { + if ( + typeof describeStickerVisual !== "function" || + typeof store?.updateStickerDescription !== "function" + ) { + return + } + const existingSticker = await store?.getSeenSticker?.(sticker.file_unique_id) + if (existingSticker?.description) { + return + } + + try { + const description = sanitizeStickerDescription( + await describeStickerVisual({ sticker, attachment, visualDescription }), + ) + if (description) { + await store.updateStickerDescription(sticker.file_unique_id, description) + } + } catch (error) { + logger?.warn?.({ error }, "Could not describe Telegram sticker visual") + } +} + +function sanitizeStickerDescription(description) { + const line = String(description ?? "") + .split(/\r?\n/u) + .map((part) => part.trim()) + .find(Boolean) + if (!line) { + return null + } + const safe = line + .replace(/\[[^\]\n]*\]/gu, "") + .replace(/\s+/gu, " ") + .trim() + .slice(0, 160) + return safe || null +} + +function attachmentFromFile(filePath, mime) { + return { mime, url: pathToFileURL(filePath).href, filePath } +} + +function stickerMime(sticker, filePath) { + if (sticker?.is_video) { + return "video/webm" + } + if (sticker?.is_animated) { + return "application/gzip" + } + switch (extname(filePath).toLowerCase()) { + case ".png": + return "image/png" + case ".jpg": + case ".jpeg": + return "image/jpeg" + default: + return "image/webp" + } +} + +function extensionForMime(mime) { + switch (mime) { + case "video/webm": + return ".webm" + case "application/gzip": + return ".tgs" + case "image/png": + return ".png" + case "image/jpeg": + return ".jpg" + default: + return ".webp" + } +} diff --git a/src/bin/program.js b/src/bin/program.js index 6660c7b..f8ec027 100644 --- a/src/bin/program.js +++ b/src/bin/program.js @@ -36,7 +36,7 @@ export function createGatewayProgram({ const program = new Command() const afterCreate = createStartupAfterConfigHook({ enableGatewayStartup, output }) - program.name("opencode-remote").description("OpenCode messaging gateway").version("0.5.7") + program.name("opencode-remote").description("OpenCode messaging gateway").version("0.6.0") program .command("setup") diff --git a/src/core/commands/commands.js b/src/core/commands/commands.js index d43384b..31f6032 100644 --- a/src/core/commands/commands.js +++ b/src/core/commands/commands.js @@ -5,6 +5,7 @@ export const botCommands = [ { command: "stop", description: "Abort current OpenCode task" }, { command: "progress", description: "Set tool progress visibility" }, { command: "voice", description: "Show or set voice mode" }, + { command: "stickers", description: "Manage saved sticker packs" }, { command: "help", description: "Show available commands" }, ] diff --git a/src/runtime/bootstrap.js b/src/runtime/bootstrap.js index db1af31..1f72bae 100644 --- a/src/runtime/bootstrap.js +++ b/src/runtime/bootstrap.js @@ -2,6 +2,7 @@ import { createTelegramBot as defaultCreateTelegramBot, registerTelegramBotCommands as defaultRegisterTelegramBotCommands, } from "../adapters/telegram/bot.js" +import { openTelegramStickerStore as defaultOpenTelegramStickerStore } from "../adapters/telegram/stickerStore.js" import { loadConfig } from "../config/loadConfig.js" import { setConfigValuesAtPath as defaultSetConfigValuesAtPath } from "../config/writeConfig.js" import { createGatewayContext } from "../core/gateway/context.js" @@ -36,6 +37,8 @@ export async function runGateway({ const registerTelegramBotCommands = dependencies.registerTelegramBotCommands ?? defaultRegisterTelegramBotCommands const createVoiceService = dependencies.createVoiceService ?? defaultCreateVoiceService + const openTelegramStickerStore = + dependencies.openTelegramStickerStore ?? defaultOpenTelegramStickerStore const assertFfmpegAvailable = dependencies.assertFfmpegAvailable ?? defaultAssertFfmpegAvailable const setConfigValuesAtPath = dependencies.setConfigValuesAtPath ?? defaultSetConfigValuesAtPath @@ -69,6 +72,7 @@ export async function runGateway({ }) }, }) + const stickerStore = openTelegramStickerStore() const bot = createTelegramBot({ token: resolvedConfig.telegram.botToken, allowedUserId: resolvedConfig.telegram.allowedUserId, @@ -76,6 +80,7 @@ export async function runGateway({ logger: resolvedLogger, progressVerbosity: resolvedConfig.progressVerbosity, voiceService, + stickerStore, }) let stopping = false @@ -87,6 +92,7 @@ export async function runGateway({ resolvedLogger.info({ signal }, "Shutting down gateway") await bot.stop() await server.stop() + stickerStore.close?.() } processLike.once("SIGINT", shutdown) diff --git a/tests/adapters/telegramBot.test.js b/tests/adapters/telegramBot.test.js index fd6dd11..cc7183e 100644 --- a/tests/adapters/telegramBot.test.js +++ b/tests/adapters/telegramBot.test.js @@ -1,5 +1,6 @@ import { afterEach, describe, expect, test, vi } from "vitest" import { createTelegramBot } from "../../src/adapters/telegram/bot.js" +import { createMemoryStickerStore } from "../../src/adapters/telegram/stickerStore.js" class FakeBot { constructor(token) { @@ -56,10 +57,12 @@ describe("createTelegramBot", () => { "stop", "progress", "voice", + "stickers", ]) expect(bot.messageHandlers.has("message:text")).toBe(true) expect(bot.messageHandlers.has("message:photo")).toBe(true) expect(bot.messageHandlers.has("message:voice")).toBe(true) + expect(bot.messageHandlers.has("message:sticker")).toBe(true) expect(bot.messageHandlers.has("message_reaction")).toBe(true) expect(bot.errorHandler).toEqual(expect.any(Function)) expect(bot.api.setMyCommands).not.toHaveBeenCalled() @@ -1160,6 +1163,57 @@ describe("createTelegramBot", () => { ) }) + test("text prompts tell OpenCode how to request saved sticker replies", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "funny_cats", + stickers: [ + { + fileUniqueId: "cat-1", + fileId: "file-secret-cat", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + ], + }) + await stickerStore.updateStickerDescription("cat-1", "laughing orange cat") + const controller = { + sendPrompt: vi.fn(async () => "answer"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + + await bot.messageHandlers.get("message:text")({ + message: { + message_id: 10, + text: "send me a sticker", + chat: { id: 456 }, + from: { id: 123, is_bot: false, first_name: "Authorized", last_name: "User" }, + }, + api: { + sendChatAction: vi.fn(async () => undefined), + setMessageReaction: vi.fn(async () => true), + }, + reply: vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "answer" })), + }) + + const prompt = controller.sendPrompt.mock.calls[0][0] + expect(prompt.text).toContain("If the user explicitly asks for a sticker") + expect(prompt.text).toContain("[telegram_sticker: 😹]") + expect(prompt.text).toContain("[telegram_sticker: any]") + expect(prompt.text).toContain("funny_cats") + expect(prompt.text).toContain("😹") + expect(prompt.text).toContain("laughing orange cat") + expect(prompt.text).not.toContain("file-secret-cat") + }) + test("hidden telegram reaction markers are stripped and applied to the user message", async () => { const controller = { sendPrompt: vi.fn(async () => "Nice idea.\n[telegram_reaction: πŸ‘]"), @@ -1187,6 +1241,384 @@ describe("createTelegramBot", () => { expect(setMessageReaction).toHaveBeenNthCalledWith(3, 456, 10, [{ type: "emoji", emoji: "πŸ‘" }]) }) + test("hidden telegram sticker markers are stripped and send matching saved stickers", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "funny_cats", + stickers: [ + { + fileUniqueId: "cat-1", + fileId: "cat-file-id", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + { + fileUniqueId: "ok-1", + fileId: "ok-file-id", + packName: "funny_cats", + emoji: "πŸ‘", + kind: "static", + }, + ], + }) + const controller = { + sendPrompt: vi.fn(async () => "Here you go.\n[telegram_sticker: 😹]"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + random: vi.fn(() => 0), + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "Here you go." })) + const replyWithSticker = vi.fn(async () => ({ message_id: 12, chat: { id: 456 } })) + const setMessageReaction = vi.fn(async () => true) + + await bot.messageHandlers.get("message:text")({ + message: { message_id: 10, text: "send sticker", chat: { id: 456 } }, + api: { sendChatAction: vi.fn(async () => undefined), setMessageReaction }, + reply, + replyWithSticker, + }) + + expect(reply).toHaveBeenCalledWith("Here you go.") + expect(replyWithSticker).toHaveBeenCalledWith("cat-file-id") + expect(setMessageReaction).toHaveBeenNthCalledWith(1, 456, 10, [{ type: "emoji", emoji: "πŸ‘€" }]) + expect(setMessageReaction).toHaveBeenNthCalledWith(2, 456, 10, []) + expect(setMessageReaction).toHaveBeenCalledTimes(2) + }) + + test("any telegram sticker marker sends a saved sticker without empty text", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "funny_cats", + stickers: [ + { + fileUniqueId: "cat-1", + fileId: "cat-file-id", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + ], + }) + const controller = { + sendPrompt: vi.fn(async () => "[telegram_sticker: any]"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + random: vi.fn(() => 0), + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "" })) + const replyWithSticker = vi.fn(async () => ({ message_id: 12, chat: { id: 456 } })) + + await bot.messageHandlers.get("message:text")({ + message: { message_id: 10, text: "send sticker", chat: { id: 456 } }, + api: { + sendChatAction: vi.fn(async () => undefined), + setMessageReaction: vi.fn(async () => true), + }, + reply, + replyWithSticker, + }) + + expect(reply).not.toHaveBeenCalled() + expect(replyWithSticker).toHaveBeenCalledWith("cat-file-id") + }) + + test("telegram sticker markers can select saved stickers by description", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "funny_cats", + stickers: [ + { + fileUniqueId: "cat-1", + fileId: "cat-file-id", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + { + fileUniqueId: "duck-1", + fileId: "duck-file-id", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + ], + }) + await stickerStore.updateStickerDescription("cat-1", "laughing orange cat") + await stickerStore.updateStickerDescription("duck-1", "thumbs up duck") + const controller = { + sendPrompt: vi.fn(async () => "[telegram_sticker: thumbs up duck]"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + random: vi.fn(() => 0), + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const replyWithSticker = vi.fn(async () => ({ message_id: 12, chat: { id: 456 } })) + + await bot.messageHandlers.get("message:text")({ + message: { message_id: 10, text: "send duck sticker", chat: { id: 456 } }, + api: { + sendChatAction: vi.fn(async () => undefined), + setMessageReaction: vi.fn(async () => true), + }, + reply: vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "" })), + replyWithSticker, + }) + + expect(replyWithSticker).toHaveBeenCalledWith("duck-file-id") + }) + + test("sticker messages send visual sticker prompts and offer to save unsaved packs", async () => { + const attachment = { + mime: "image/webp", + url: "file:///cache/sticker.webp", + filePath: "/cache/sticker.webp", + } + const controller = { + sendPrompt: vi.fn(async () => "sticker answer"), + } + const stickerStore = createMemoryStickerStore() + const createStickerPrompt = vi.fn(async () => ({ + prompt: { text: "Sticker prompt", attachments: [attachment] }, + packName: "funny_cats", + cleanupFiles: ["/tmp/source.webp"], + })) + const cleanupStickerFiles = vi.fn(async () => undefined) + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + createStickerPrompt, + cleanupStickerFiles, + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async (text) => ({ message_id: 20, chat: { id: 456 }, text })) + + await bot.messageHandlers.get("message:sticker")({ + message: { + message_id: 10, + chat: { id: 456 }, + from: { id: 123, is_bot: false, first_name: "Authorized", last_name: "User" }, + sticker: telegramSticker({ set_name: "funny_cats" }), + }, + api: { sendChatAction: vi.fn(async () => undefined) }, + reply, + }) + + expect(createStickerPrompt).toHaveBeenCalledWith( + expect.objectContaining({ + token: "token", + sticker: expect.objectContaining({ file_id: "file-static" }), + }), + ) + expect(controller.sendPrompt).toHaveBeenCalledWith( + { + text: expect.stringContaining("Sticker prompt"), + attachments: [attachment], + author: { name: "Authorized User", source: "sender" }, + }, + expect.objectContaining({ onProgress: expect.any(Function) }), + ) + expect(reply).toHaveBeenCalledWith("sticker answer") + expect(reply).toHaveBeenCalledWith( + "Sticker pack funny_cats is not saved. Save it for future sticker replies?", + expect.objectContaining({ reply_markup: expect.any(Object) }), + ) + expect(cleanupStickerFiles).toHaveBeenCalledWith(["/tmp/source.webp"], expect.any(Object)) + }) + + test("stickers save command saves the replied sticker pack", async () => { + const stickerStore = createMemoryStickerStore() + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller: {}, + stickerStore, + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async (text) => ({ message_id: 11, chat: { id: 456 }, text })) + + await bot.commands.get("stickers")({ + message: { + text: "/stickers save", + reply_to_message: { sticker: telegramSticker({ set_name: "funny_cats" }) }, + }, + api: { + getStickerSet: vi.fn(async () => ({ + name: "funny_cats", + stickers: [telegramSticker({ file_unique_id: "one" })], + })), + }, + reply, + }) + + expect(await stickerStore.listPacks()).toEqual([ + { name: "funny_cats", stickerCount: 1, emojis: ["😹"] }, + ]) + expect(reply).toHaveBeenCalledWith("Saved sticker pack funny_cats (1 sticker).") + }) + + test("stickers save command rejects stickers without a pack name safely", async () => { + const stickerStore = createMemoryStickerStore() + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller: {}, + stickerStore, + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async (text) => ({ message_id: 11, chat: { id: 456 }, text })) + + await bot.commands.get("stickers")({ + message: { + text: "/stickers save", + reply_to_message: { sticker: telegramSticker({ set_name: undefined }) }, + }, + api: { getStickerSet: vi.fn() }, + reply, + }) + + expect(reply).toHaveBeenCalledWith("That sticker does not belong to a saveable sticker pack.") + expect(await stickerStore.listPacks()).toEqual([]) + }) + + test("stickers list and forget manage saved packs", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "funny_cats", + stickers: [ + { + fileUniqueId: "one", + fileId: "file-one", + packName: "funny_cats", + emoji: "😹", + kind: "static", + }, + ], + }) + const cleanupStickerFiles = vi.fn(async () => undefined) + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller: {}, + stickerStore, + cleanupStickerFiles, + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async (text) => ({ message_id: 11, chat: { id: 456 }, text })) + + await bot.commands.get("stickers")({ message: { text: "/stickers list" }, reply }) + await bot.commands.get("stickers")({ message: { text: "/stickers forget funny_cats" }, reply }) + + expect(reply).toHaveBeenCalledWith("Saved sticker packs:\n- funny_cats (1 sticker, 😹)") + expect(reply).toHaveBeenCalledWith("Forgot sticker pack funny_cats.") + expect(await stickerStore.listPacks()).toEqual([]) + }) + + test("saved stickers can replace requested emoji reactions", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "ok_pack", + stickers: [ + { + fileUniqueId: "ok-1", + fileId: "sticker-file-id", + packName: "ok_pack", + emoji: "πŸ‘", + kind: "static", + }, + ], + }) + const controller = { + sendPrompt: vi.fn(async () => "Nice.\n[telegram_reaction: πŸ‘]"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + random: vi.fn(() => 0), + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "Nice." })) + const replyWithSticker = vi.fn(async () => ({ message_id: 12, chat: { id: 456 } })) + const setMessageReaction = vi.fn(async () => true) + + await bot.messageHandlers.get("message:text")({ + message: { message_id: 10, text: "hello", chat: { id: 456 } }, + api: { sendChatAction: vi.fn(async () => undefined), setMessageReaction }, + reply, + replyWithSticker, + }) + + expect(reply).toHaveBeenCalledWith("Nice.") + expect(replyWithSticker).toHaveBeenCalledWith("sticker-file-id") + expect(setMessageReaction).toHaveBeenNthCalledWith(1, 456, 10, [{ type: "emoji", emoji: "πŸ‘€" }]) + expect(setMessageReaction).toHaveBeenNthCalledWith(2, 456, 10, []) + expect(setMessageReaction).toHaveBeenCalledTimes(2) + }) + + test("missing sticker send APIs fall back to emoji reactions", async () => { + const stickerStore = createMemoryStickerStore() + await stickerStore.savePack({ + name: "ok_pack", + stickers: [ + { + fileUniqueId: "ok-1", + fileId: "sticker-file-id", + packName: "ok_pack", + emoji: "πŸ‘", + kind: "static", + }, + ], + }) + const controller = { + sendPrompt: vi.fn(async () => "Nice.\n[telegram_reaction: πŸ‘]"), + } + const bot = createTelegramBot({ + token: "token", + allowedUserId: 123, + controller, + stickerStore, + random: vi.fn(() => 0), + logger: { warn: vi.fn(), error: vi.fn() }, + botFactory: FakeBot, + }) + const reply = vi.fn(async () => ({ message_id: 11, chat: { id: 456 }, text: "Nice." })) + const setMessageReaction = vi.fn(async () => true) + + await bot.messageHandlers.get("message:text")({ + message: { message_id: 10, text: "hello", chat: { id: 456 } }, + api: { sendChatAction: vi.fn(async () => undefined), setMessageReaction }, + reply, + }) + + expect(setMessageReaction).toHaveBeenNthCalledWith(3, 456, 10, [{ type: "emoji", emoji: "πŸ‘" }]) + }) + test("single photo messages send one image prompt and one response", async () => { const attachment = { mime: "image/jpeg", @@ -1596,3 +2028,18 @@ function photoContext({ messageId, fileId, caption = "", reply }) { reply, } } + +function telegramSticker(overrides = {}) { + return { + file_id: "file-static", + file_unique_id: "unique-static", + width: 512, + height: 512, + file_size: 100, + emoji: "😹", + set_name: "funny_cats", + is_animated: false, + is_video: false, + ...overrides, + } +} diff --git a/tests/adapters/telegramStickerCache.test.js b/tests/adapters/telegramStickerCache.test.js new file mode 100644 index 0000000..e747665 --- /dev/null +++ b/tests/adapters/telegramStickerCache.test.js @@ -0,0 +1,124 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { describe, expect, test, vi } from "vitest" +import { + cachedStickerFilePath, + getStickerCacheDir, + isStickerCacheRecordUsable, + removeCachedStickerFiles, + STICKER_CONVERTER_VERSION, +} from "../../src/adapters/telegram/stickerCache.js" + +describe("telegram sticker cache helpers", () => { + test("uses the app-data sticker cache directory", () => { + expect( + getStickerCacheDir({ + platform: "linux", + env: { XDG_DATA_HOME: "/data" }, + homeDir: "/home/user", + }), + ).toBe("/data/opencode-remote/cache/stickers") + }) + + test("builds safe cached sticker file paths", () => { + expect( + cachedStickerFilePath( + { fileUniqueId: "abc/../def", kind: "animated", extension: "png" }, + { directory: "/cache/stickers" }, + ), + ).toBe(`/cache/stickers/abc-def-animated-v${STICKER_CONVERTER_VERSION}.png`) + }) + + test("accepts usable cache records when sticker metadata and file match", async () => { + const directory = await mkdtemp(join(tmpdir(), "sticker-cache-valid-")) + const filePath = join(directory, "cached.png") + await writeFile(filePath, Buffer.from([1, 2, 3])) + + try { + await expect( + isStickerCacheRecordUsable({ + sticker: staticSticker({ file_unique_id: "unique-1", file_size: 123 }), + record: { + fileUniqueId: "unique-1", + kind: "static", + width: 512, + height: 512, + fileSize: 123, + converterVersion: STICKER_CONVERTER_VERSION, + filePath, + }, + }), + ).resolves.toBe(true) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("rejects cache records when metadata differs or the file is missing", async () => { + const sticker = staticSticker({ file_unique_id: "unique-1", file_size: 123 }) + + await expect( + isStickerCacheRecordUsable({ + sticker, + record: { + fileUniqueId: "unique-2", + kind: "static", + width: 512, + height: 512, + fileSize: 123, + converterVersion: STICKER_CONVERTER_VERSION, + filePath: "/missing.png", + }, + }), + ).resolves.toBe(false) + + await expect( + isStickerCacheRecordUsable({ + sticker, + record: { + fileUniqueId: "unique-1", + kind: "static", + width: 512, + height: 512, + fileSize: 123, + converterVersion: STICKER_CONVERTER_VERSION, + filePath: "/missing.png", + }, + }), + ).resolves.toBe(false) + }) + + test("removes cached sticker files and logs cleanup failures", async () => { + const rmFn = vi.fn(async (filePath) => { + if (filePath === "/cache/bad.png") { + throw new Error("locked") + } + }) + const logger = { warn: vi.fn() } + + await removeCachedStickerFiles( + [{ filePath: "/cache/ok.png" }, { filePath: "/cache/bad.png" }, { filePath: null }], + { logger, rmFn }, + ) + + expect(rmFn).toHaveBeenCalledWith("/cache/ok.png", { force: true }) + expect(rmFn).toHaveBeenCalledWith("/cache/bad.png", { force: true }) + expect(logger.warn).toHaveBeenCalledWith( + { error: expect.any(Error), filePath: "/cache/bad.png" }, + "Could not remove cached Telegram sticker file", + ) + }) +}) + +function staticSticker(overrides = {}) { + return { + file_id: "file-1", + file_unique_id: "unique-1", + width: 512, + height: 512, + is_animated: false, + is_video: false, + ...overrides, + } +} diff --git a/tests/adapters/telegramStickerRenderer.test.js b/tests/adapters/telegramStickerRenderer.test.js new file mode 100644 index 0000000..31e18f4 --- /dev/null +++ b/tests/adapters/telegramStickerRenderer.test.js @@ -0,0 +1,36 @@ +import { describe, expect, test, vi } from "vitest" +import { + renderAnimatedStickerPreview, + renderVideoStickerPreview, +} from "../../src/adapters/telegram/stickerRenderer.js" + +describe("telegram sticker renderer", () => { + test("renders video stickers with ffmpeg contact-sheet settings", async () => { + const execa = vi.fn(async () => undefined) + + await expect( + renderVideoStickerPreview({ inputPath: "/tmp/in.webm", outputPath: "/tmp/out.png", execa }), + ).resolves.toEqual({ mime: "image/png", filePath: "/tmp/out.png" }) + + expect(execa).toHaveBeenCalledWith("ffmpeg", [ + "-y", + "-i", + "/tmp/in.webm", + "-vf", + "fps=2,scale=256:-1,tile=3x2", + "-frames:v", + "1", + "/tmp/out.png", + ]) + }) + + test("renders animated TGS stickers through python-lottie when available", async () => { + const execa = vi.fn(async () => undefined) + + await expect( + renderAnimatedStickerPreview({ inputPath: "/tmp/in.tgs", outputPath: "/tmp/out.png", execa }), + ).resolves.toEqual({ mime: "image/png", filePath: "/tmp/out.png" }) + + expect(execa).toHaveBeenCalledWith("lottie_convert.py", ["/tmp/in.tgs", "/tmp/out.png"]) + }) +}) diff --git a/tests/adapters/telegramStickerStore.test.js b/tests/adapters/telegramStickerStore.test.js new file mode 100644 index 0000000..3c8654a --- /dev/null +++ b/tests/adapters/telegramStickerStore.test.js @@ -0,0 +1,200 @@ +import { mkdtemp, rm } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { afterEach, describe, expect, test } from "vitest" +import { + createMemoryStickerStore, + openTelegramStickerStore, +} from "../../src/adapters/telegram/stickerStore.js" + +describe("telegram sticker store", () => { + const stores = [] + + afterEach(() => { + for (const store of stores.splice(0)) { + store.close?.() + } + }) + + test("persists saved packs and summarizes emojis", async () => { + const { store, directory } = await openTempStore() + + try { + await store.savePack({ + name: "funny_cats", + stickers: [ + stickerMeta({ fileUniqueId: "cat-1", fileId: "file-cat-1", emoji: "😹" }), + stickerMeta({ fileUniqueId: "cat-2", fileId: "file-cat-2", emoji: "😹" }), + stickerMeta({ fileUniqueId: "cat-3", fileId: "file-cat-3", emoji: "πŸ‘" }), + ], + }) + + expect(await store.listPacks()).toEqual([ + { name: "funny_cats", stickerCount: 3, emojis: ["😹", "πŸ‘"] }, + ]) + expect(await store.hasSavedPack("funny_cats")).toBe(true) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("persists safe sticker descriptions for saved sticker catalogs", async () => { + const { store, directory } = await openTempStore() + + try { + await store.savePack({ + name: "funny_cats", + stickers: [stickerMeta({ fileUniqueId: "cat-1", fileId: "file-cat-1", emoji: "😹" })], + }) + await store.updateStickerDescription("cat-1", "laughing orange cat") + + await expect(store.listStickerCatalog()).resolves.toEqual([ + { + packName: "funny_cats", + emoji: "😹", + description: "laughing orange cat", + }, + ]) + await expect( + store.findStickerForSelector("laughing orange cat", { random: () => 0 }), + ).resolves.toEqual(expect.objectContaining({ fileUniqueId: "cat-1", fileId: "file-cat-1" })) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("preserves seen sticker descriptions when a pack is saved later", async () => { + const store = createMemoryStickerStore() + stores.push(store) + + await store.upsertSeenSticker( + stickerMeta({ fileUniqueId: "cat-1", fileId: "file-cat-1", packName: "funny_cats" }), + ) + await store.updateStickerDescription("cat-1", "wide-eyed cat") + await store.savePack({ + name: "funny_cats", + stickers: [stickerMeta({ fileUniqueId: "cat-1", fileId: "file-cat-1", emoji: "😹" })], + }) + + await expect(store.listStickerCatalog()).resolves.toEqual([ + { packName: "funny_cats", emoji: "😹", description: "wide-eyed cat" }, + ]) + }) + + test("selects a saved sticker by emoji with fallback to any saved sticker", async () => { + const store = createMemoryStickerStore() + stores.push(store) + await store.savePack({ + name: "mixed", + stickers: [ + stickerMeta({ fileUniqueId: "one", fileId: "file-one", emoji: "😹" }), + stickerMeta({ fileUniqueId: "two", fileId: "file-two", emoji: "πŸ‘" }), + ], + }) + + expect(await store.findStickerForEmoji("πŸ‘", { random: () => 0 })).toEqual( + expect.objectContaining({ fileUniqueId: "two", fileId: "file-two", emoji: "πŸ‘" }), + ) + expect(await store.findStickerForEmoji("πŸ”₯", { random: () => 0 })).toEqual( + expect.objectContaining({ fileUniqueId: "one", fileId: "file-one", emoji: "😹" }), + ) + }) + + test("forgets packs and returns associated cache records", async () => { + const store = createMemoryStickerStore() + stores.push(store) + await store.savePack({ + name: "mixed", + stickers: [stickerMeta({ fileUniqueId: "one", fileId: "file-one" })], + }) + await store.writeCacheRecord({ + fileUniqueId: "one", + packName: "mixed", + kind: "static", + width: 512, + height: 512, + fileSize: 100, + converterVersion: "1", + filePath: "/cache/one.webp", + }) + + await expect(store.forgetPack("mixed")).resolves.toEqual({ + deleted: true, + cacheRecords: [expect.objectContaining({ filePath: "/cache/one.webp" })], + }) + expect(await store.listPacks()).toEqual([]) + await expect(store.forgetPack("missing")).resolves.toEqual({ deleted: false, cacheRecords: [] }) + }) + + test("forgets SQLite cache rows tied by sticker ID even without a pack name", async () => { + const { store, directory } = await openTempStore() + + try { + await store.savePack({ + name: "mixed", + stickers: [stickerMeta({ fileUniqueId: "one", fileId: "file-one", packName: "mixed" })], + }) + await store.writeCacheRecord({ + fileUniqueId: "one", + kind: "static", + width: 512, + height: 512, + fileSize: 100, + converterVersion: "1", + filePath: "/cache/one.webp", + }) + + await expect(store.forgetPack("mixed")).resolves.toEqual({ + deleted: true, + cacheRecords: [expect.objectContaining({ filePath: "/cache/one.webp" })], + }) + await expect(store.readCacheRecord("one", "static")).resolves.toBeNull() + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("stores seen stickers and cache records without secret fields", async () => { + const store = createMemoryStickerStore() + stores.push(store) + await store.upsertSeenSticker(stickerMeta({ fileUniqueId: "seen-1", fileId: "file-seen" })) + await store.writeCacheRecord({ + fileUniqueId: "seen-1", + kind: "static", + width: 512, + height: 512, + fileSize: 100, + converterVersion: "1", + filePath: "/cache/seen.webp", + }) + + const seen = await store.getSeenSticker("seen-1") + const cache = await store.readCacheRecord("seen-1", "static") + + expect(seen).toEqual(expect.objectContaining({ fileUniqueId: "seen-1", fileId: "file-seen" })) + expect(cache).toEqual(expect.objectContaining({ filePath: "/cache/seen.webp" })) + expect(JSON.stringify({ seen, cache })).not.toContain("bot") + expect(JSON.stringify({ seen, cache })).not.toContain("chat") + expect(JSON.stringify({ seen, cache })).not.toContain("user") + }) +}) + +async function openTempStore() { + const directory = await mkdtemp(join(tmpdir(), "sticker-store-test-")) + const store = openTelegramStickerStore(join(directory, "stickers.db")) + return { store, directory } +} + +function stickerMeta(overrides = {}) { + return { + fileUniqueId: "unique-1", + fileId: "file-1", + packName: "pack", + emoji: "😹", + kind: "static", + width: 512, + height: 512, + fileSize: 100, + ...overrides, + } +} diff --git a/tests/adapters/telegramStickers.test.js b/tests/adapters/telegramStickers.test.js new file mode 100644 index 0000000..002bd32 --- /dev/null +++ b/tests/adapters/telegramStickers.test.js @@ -0,0 +1,359 @@ +import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { fileURLToPath } from "node:url" +import { describe, expect, test, vi } from "vitest" +import { createMemoryStickerStore } from "../../src/adapters/telegram/stickerStore.js" +import { + createStickerPrompt, + downloadTelegramSticker, + formatStickerPromptText, +} from "../../src/adapters/telegram/stickers.js" + +describe("telegram sticker helpers", () => { + test("downloads Telegram stickers without exposing the bot token in file URLs", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-sticker-download-")) + const api = { getFile: vi.fn(async () => ({ file_path: "stickers/static.webp" })) } + const fetchFn = vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([1, 2, 3]).buffer, + })) + + try { + const attachment = await downloadTelegramSticker({ + api, + token: "secret-token", + sticker: staticSticker(), + directory, + fetchFn, + }) + + expect(api.getFile).toHaveBeenCalledWith("file-static") + expect(fetchFn).toHaveBeenCalledWith( + "https://api.telegram.org/file/botsecret-token/stickers/static.webp", + ) + expect(attachment.mime).toBe("image/webp") + expect(attachment.url).toMatch(/^file:\/\//u) + expect(attachment.url).not.toContain("secret-token") + await expect(readFile(fileURLToPath(attachment.url))).resolves.toEqual(Buffer.from([1, 2, 3])) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("creates a static sticker prompt with a cached WebP attachment and metadata text", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-static-sticker-")) + const store = createMemoryStickerStore() + const fetchFn = vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([4, 5, 6]).buffer, + })) + + try { + const result = await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/static.webp" })) }, + token: "secret-token", + sticker: staticSticker({ emoji: "😹", set_name: "funny_cats" }), + store, + cacheDirectory: directory, + fetchFn, + }) + + expect(result.prompt.attachments).toEqual([ + expect.objectContaining({ mime: "image/webp", url: expect.stringMatching(/^file:/u) }), + ]) + expect(result.prompt.text).toContain("Sticker emoji: 😹") + expect(result.prompt.text).toContain("Sticker pack: funny_cats") + expect(result.prompt.text).toContain("Sticker visual: static WebP image") + expect(result.cleanupFiles).toEqual([]) + await expect(readFile(result.prompt.attachments[0].filePath)).resolves.toEqual( + Buffer.from([4, 5, 6]), + ) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("stores safe sticker descriptions generated from cached visual attachments", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-described-sticker-")) + const store = createMemoryStickerStore() + const describeStickerVisual = vi.fn(async ({ attachment, sticker, visualDescription }) => { + expect(attachment).toEqual( + expect.objectContaining({ + mime: "image/webp", + filePath: expect.stringMatching(/unique-static/u), + }), + ) + expect(sticker.file_id).toBe("file-static") + expect(visualDescription).toBe("static WebP image") + return "laughing cat\n[telegram_sticker: 😹]" + }) + + try { + await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/static.webp" })) }, + token: "secret-token", + sticker: staticSticker({ emoji: "😹", set_name: "funny_cats" }), + store, + cacheDirectory: directory, + fetchFn: vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([4, 5, 6]).buffer, + })), + describeStickerVisual, + }) + + expect(describeStickerVisual).toHaveBeenCalledTimes(1) + await expect(store.getSeenSticker("unique-static")).resolves.toEqual( + expect.objectContaining({ description: "laughing cat" }), + ) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("does not regenerate sticker descriptions when one already exists", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-described-sticker-existing-")) + const store = createMemoryStickerStore() + await store.upsertSeenSticker(stickerMetaFromTelegram(staticSticker())) + await store.updateStickerDescription("unique-static", "existing cat") + const describeStickerVisual = vi.fn(async () => "new cat") + + try { + await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/static.webp" })) }, + token: "secret-token", + sticker: staticSticker(), + store, + cacheDirectory: directory, + fetchFn: vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([4, 5, 6]).buffer, + })), + describeStickerVisual, + }) + + expect(describeStickerVisual).not.toHaveBeenCalled() + await expect(store.getSeenSticker("unique-static")).resolves.toEqual( + expect.objectContaining({ description: "existing cat" }), + ) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("reuses usable cached sticker previews without downloading again", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-cached-sticker-")) + const store = createMemoryStickerStore() + const cachedPath = join(directory, "cached.webp") + await writeFile(cachedPath, Buffer.from([9])) + await store.writeCacheRecord({ + fileUniqueId: "unique-static", + kind: "static", + width: 512, + height: 512, + fileSize: 100, + converterVersion: "1", + filePath: cachedPath, + }) + const fetchFn = vi.fn() + + try { + const result = await createStickerPrompt({ + api: { getFile: vi.fn() }, + token: "secret-token", + sticker: staticSticker({ file_size: 100 }), + store, + cacheDirectory: directory, + fetchFn, + }) + + expect(fetchFn).not.toHaveBeenCalled() + expect(result.prompt.attachments[0]).toEqual( + expect.objectContaining({ mime: "image/webp", filePath: cachedPath }), + ) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("regenerates video sticker previews when cache metadata no longer matches", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-video-sticker-")) + const store = createMemoryStickerStore() + const oldPath = join(directory, "old.png") + await writeFile(oldPath, Buffer.from([1])) + await store.writeCacheRecord({ + fileUniqueId: "unique-video", + kind: "video", + width: 128, + height: 128, + fileSize: 1, + converterVersion: "1", + filePath: oldPath, + }) + const renderVideoStickerPreview = vi.fn(async ({ outputPath }) => { + await writeFile(outputPath, Buffer.from([7, 8])) + return { mime: "image/png", filePath: outputPath } + }) + + try { + const result = await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/video.webm" })) }, + token: "secret-token", + sticker: videoSticker({ file_size: 200 }), + store, + cacheDirectory: directory, + mediaDirectory: directory, + fetchFn: vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([1, 2, 3]).buffer, + })), + renderVideoStickerPreview, + }) + + expect(renderVideoStickerPreview).toHaveBeenCalled() + expect(result.prompt.attachments[0]).toEqual( + expect.objectContaining({ + mime: "image/png", + filePath: expect.stringMatching(/unique-video-video-v1\.png$/u), + }), + ) + await expect(store.readCacheRecord("unique-video", "video")).resolves.toEqual( + expect.objectContaining({ fileSize: 200, filePath: result.prompt.attachments[0].filePath }), + ) + expect(result.cleanupFiles).toHaveLength(1) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("uses a safe temporary download directory when cache paths use defaults", async () => { + const store = createMemoryStickerStore() + const renderVideoStickerPreview = vi.fn(async ({ inputPath, outputPath }) => { + expect(inputPath).toMatch(/telegram-sticker-/u) + return { mime: "image/png", filePath: outputPath } + }) + let cleanupFiles = [] + + try { + const result = await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/video.webm" })) }, + token: "secret-token", + sticker: videoSticker(), + store, + fetchFn: vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([1, 2, 3]).buffer, + })), + renderVideoStickerPreview, + }) + cleanupFiles = result.cleanupFiles + + expect(result.prompt.attachments[0]).toEqual( + expect.objectContaining({ + mime: "image/png", + filePath: expect.stringMatching(/unique-video-video-v1\.png$/u), + }), + ) + } finally { + await Promise.all(cleanupFiles.map((filePath) => rm(filePath, { force: true }))) + } + }) + + test("creates preview cache directories before rendering non-static stickers", async () => { + const directory = await mkdtemp(join(tmpdir(), "telegram-missing-preview-dir-")) + const cacheDirectory = join(directory, "missing", "stickers") + const store = createMemoryStickerStore() + const renderVideoStickerPreview = vi.fn(async ({ outputPath }) => { + await writeFile(outputPath, Buffer.from([8, 9])) + return { mime: "image/png", filePath: outputPath } + }) + + try { + const result = await createStickerPrompt({ + api: { getFile: vi.fn(async () => ({ file_path: "stickers/video.webm" })) }, + token: "secret-token", + sticker: videoSticker(), + store, + cacheDirectory, + fetchFn: vi.fn(async () => ({ + ok: true, + arrayBuffer: async () => new Uint8Array([1, 2, 3]).buffer, + })), + renderVideoStickerPreview, + }) + + await expect(readFile(result.prompt.attachments[0].filePath)).resolves.toEqual( + Buffer.from([8, 9]), + ) + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + test("formats sticker prompt metadata without raw Telegram payloads", () => { + expect( + formatStickerPromptText( + staticSticker({ emoji: "πŸ‘", set_name: "ok_pack" }), + "cached preview", + ), + ).toBe( + [ + "React to this Telegram sticker as the Telegram bot persona.", + "Use the attached visual sticker content and the metadata below.", + "Do not describe the sticker in detail unless the user asks what is in it.", + "Keep the reply short, funny, and chatty.", + "", + "Sticker metadata:", + "- Sticker emoji: πŸ‘", + "- Sticker pack: ok_pack", + "- Sticker type: static", + "- Sticker dimensions: 512x512", + "- Sticker visual: cached preview", + ].join("\n"), + ) + }) +}) + +function staticSticker(overrides = {}) { + return { + file_id: "file-static", + file_unique_id: "unique-static", + width: 512, + height: 512, + file_size: 100, + emoji: "😹", + set_name: "funny_cats", + is_animated: false, + is_video: false, + ...overrides, + } +} + +function videoSticker(overrides = {}) { + return { + file_id: "file-video", + file_unique_id: "unique-video", + width: 512, + height: 512, + file_size: 200, + emoji: "🎬", + set_name: "video_pack", + is_animated: false, + is_video: true, + ...overrides, + } +} + +function stickerMetaFromTelegram(sticker) { + return { + fileUniqueId: sticker.file_unique_id, + fileId: sticker.file_id, + packName: sticker.set_name, + emoji: sticker.emoji, + kind: "static", + width: sticker.width, + height: sticker.height, + fileSize: sticker.file_size, + } +} diff --git a/tests/core/commands.test.js b/tests/core/commands.test.js index 6a28f72..555e9fd 100644 --- a/tests/core/commands.test.js +++ b/tests/core/commands.test.js @@ -10,6 +10,7 @@ describe("commands", () => { "stop", "progress", "voice", + "stickers", "help", ]) }) @@ -21,5 +22,6 @@ describe("commands", () => { expect(help).toContain("/sessions - List and switch OpenCode sessions") expect(help).toContain("/progress - Set tool progress visibility") expect(help).toContain("/voice - Show or set voice mode") + expect(help).toContain("/stickers - Manage saved sticker packs") }) }) diff --git a/tests/runtime/bootstrap.test.js b/tests/runtime/bootstrap.test.js index c1da37b..2cb8b7e 100644 --- a/tests/runtime/bootstrap.test.js +++ b/tests/runtime/bootstrap.test.js @@ -229,6 +229,40 @@ describe("runGateway", () => { expect(createTelegramBot).toHaveBeenCalledWith(expect.objectContaining({ voiceService })) }) + test("creates and passes the Telegram sticker store to the Telegram bot", async () => { + const server = { stop: vi.fn(async () => undefined) } + const bot = { + api: { setMyCommands: vi.fn(async () => undefined) }, + start: vi.fn(async () => undefined), + stop: vi.fn(async () => undefined), + } + const stickerStore = { close: vi.fn() } + const openTelegramStickerStore = vi.fn(() => stickerStore) + const createTelegramBot = vi.fn(() => bot) + + await runGateway({ + config: testConfig(), + logger: testLogger(), + dependencies: { + ensureOpenCodeServer: vi.fn(async () => server), + createOpenCodeClient: vi.fn(() => ({})), + resolveProjectIdentity: vi.fn(async () => ({ + id: "project-1", + worktree: "/project", + vcs: "git", + })), + createProjectStateStore: vi.fn(() => ({})), + createGatewayController: vi.fn(() => ({})), + openTelegramStickerStore, + createTelegramBot, + }, + processLike: { once: vi.fn() }, + }) + + expect(openTelegramStickerStore).toHaveBeenCalledWith() + expect(createTelegramBot).toHaveBeenCalledWith(expect.objectContaining({ stickerStore })) + }) + test("passes voice-aware gateway context to the controller", async () => { const logger = testLogger() const createGatewayController = vi.fn(() => ({})) @@ -333,6 +367,7 @@ describe("runGateway", () => { test("registered shutdown stops Telegram polling and owned server", async () => { const server = { stop: vi.fn(async () => undefined) } + const stickerStore = { close: vi.fn() } const bot = { api: { setMyCommands: vi.fn(async () => undefined) }, start: vi.fn(async () => undefined), @@ -354,6 +389,7 @@ describe("runGateway", () => { })), createProjectStateStore: vi.fn(() => ({})), createGatewayController: vi.fn(() => ({})), + openTelegramStickerStore: vi.fn(() => stickerStore), createTelegramBot: vi.fn(() => bot), }, processLike, @@ -363,6 +399,7 @@ describe("runGateway", () => { expect(bot.stop).toHaveBeenCalled() expect(server.stop).toHaveBeenCalled() + expect(stickerStore.close).toHaveBeenCalled() }) })