diff --git a/README.md b/README.md index f1b49778..7b83a81f 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ The easiest way to try out SEURON is to deploy it locally using docker compose. * *Optional* NVidia GPU support 1. NVidia kernel driver 450.80.02 or higher 2. [Install nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) -2. *Optional* [Setup a slack RTM bot account](https://api.slack.com/apps?new_classic_app=1) +2. *Optional* Setup a Slack bot (see [Slack Bot Setup](#slack-bot-setup) below) * Create a notification channel for SEURON runtime messages * Invite the bot to channels you want to interact with it * Recommended if you plan to use SEURON on GCP @@ -56,7 +56,7 @@ Deploying to Google Cloud is recommended when the dataset is large and/or you wa ### Requirement 1. Google Cloud SDK * [Install cloud SDK](https://cloud.google.com/sdk/docs/install) -2. **Recommended** [Setup slack RTM bot account](https://api.slack.com/apps?new_classic_app=1) +2. **Recommended** Setup a Slack bot (see [Slack Bot Setup](#slack-bot-setup) below) * Create a notification channel for SEURON runtime messages * Invite the bot to channels (not necessarily the notification channel) in which you plan to interact with it @@ -77,3 +77,53 @@ SEURON deployed to Google Cloud are created using Google Cloud Compute Engine de #### Add credentials If you need to write to cloud storages outside of your Google Cloud project, most likely you will need to provide a token/credential. SEURON stores them using [airflow variables](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/variables.html). Then you can mount these secrets using `MOUNT_SECRETS` key in the parameters for inference and segmentation. + +Slack Bot Setup +--------------- +SEURON supports two connection modes for its Slack bot: **Socket Mode** (recommended) and **RTM** (legacy). + +> **Important:** A Socket Mode Slack bot must be set up individually for every SEURON instance, whereas an RTM bot can be shared between multiple SEURON deployments. + +### Socket Mode (Recommended) +Socket Mode uses a WebSocket connection managed through an app-level token. It works with modern Slack apps and does not require a public URL or classic bot tokens. A pre-built [app manifest](slack_manifest.json) is provided to configure all required permissions and events automatically. + +#### 1. Create the Slack App from the manifest +1. Go to [https://api.slack.com/apps?new_app=1](https://api.slack.com/apps?new_app=1) and choose **From an app manifest** +2. Select your workspace and click **Next** +3. Choose the **JSON** tab and paste the contents of [`slack_manifest.json`](slack_manifest.json) +4. Click **Next**, review the summary, then click **Create** + +This configures Socket Mode, all required bot token scopes, and event subscriptions in one step. + +#### 2. Generate the App-Level Token +1. After creating the app, go to **Settings** > **Basic Information** +2. Scroll down to **App-Level Tokens** and click **Generate Token and Scopes** +3. Name the token (e.g. `seuron-socket`), add the `connections:write` scope, and click **Generate** +4. Copy the token (starts with `xapp-`) -- this is your `SLACK_APP_TOKEN` + +#### 3. Install and get the Bot Token +1. Go to **Settings** > **Install App** and click **Install to Workspace** +2. Authorize the requested permissions +3. Copy the **Bot User OAuth Token** (starts with `xoxb-`) -- this is your `SLACK_TOKEN` + +#### 4. Configure SEURON +For **local deployment**, the `start_seuronbot.local` script will prompt you for both tokens. Alternatively, set them directly in `.env.local`: +``` +SLACK_TOKEN=xoxb-your-bot-token +SLACK_APP_TOKEN=xapp-your-app-level-token +SLACK_NOTIFICATION_CHANNEL=seuron-alerts +``` + +For **Google Cloud deployment**, set the tokens in `cloud/google/cloud-deployment.yaml`: +```yaml +slack: + botToken: xoxb-your-bot-token + appToken: xapp-your-app-level-token + notificationChannel: seuron-alerts +``` + +#### 5. Invite the Bot +Invite the bot user to any Slack channels where you want to interact with it, including the notification channel. + +### RTM Mode (Legacy) +If you have an existing [classic Slack app](https://api.slack.com/apps?new_classic_app=1) with an RTM bot token, SEURON continues to support it. Simply provide the `SLACK_TOKEN` (the `xoxb-` bot token) and leave `SLACK_APP_TOKEN` empty. The bot will automatically use RTM mode when no app-level token is configured. diff --git a/cloud/google/cloud-deployment.yaml b/cloud/google/cloud-deployment.yaml index 300cadb0..8536f550 100644 --- a/cloud/google/cloud-deployment.yaml +++ b/cloud/google/cloud-deployment.yaml @@ -19,6 +19,7 @@ resources: composeLocation: https://raw.githubusercontent.com/seung-lab/seuron/main/deploy/docker-compose-CeleryExecutor.yml slack: botToken: # bot token for slack + appToken: # app-level token for Socket Mode (xapp-), leave empty for RTM mode notificationChannel: seuron-alerts enableJupyterInterface: False airflow: diff --git a/cloud/google/manager.py b/cloud/google/manager.py index a76d3a3d..c6b5090e 100644 --- a/cloud/google/manager.py +++ b/cloud/google/manager.py @@ -29,6 +29,7 @@ def GenerateEnvironVar(context, hostname_manager): env_variables = { 'VENDOR': 'Google', 'SLACK_TOKEN': context.properties['slack']['botToken'], + 'SLACK_APP_TOKEN': context.properties['slack'].get('appToken', ''), 'SLACK_NOTIFICATION_CHANNEL': context.properties['slack']['notificationChannel'], 'DEPLOYMENT': context.env['deployment'], 'ZONE': context.properties['zone'], diff --git a/deploy/docker-compose-CeleryExecutor.yml b/deploy/docker-compose-CeleryExecutor.yml index 4574436a..b4acd6ba 100644 --- a/deploy/docker-compose-CeleryExecutor.yml +++ b/deploy/docker-compose-CeleryExecutor.yml @@ -212,6 +212,7 @@ services: <<: *airflow-common-env ENABLE_JUPYTER_INTERFACE: SLACK_TOKEN: + SLACK_APP_TOKEN: SLACK_NOTIFICATION_CHANNEL: DEPLOYMENT: command: python slackbot/slack_bot.py diff --git a/slack_manifest.json b/slack_manifest.json new file mode 100644 index 00000000..a5af0268 --- /dev/null +++ b/slack_manifest.json @@ -0,0 +1,46 @@ +{ + "display_information": { + "name": "seuron", + "description": "SEUnglab neuRON pipeline task manager" + }, + "features": { + "bot_user": { + "display_name": "seuronbot", + "always_online": true + } + }, + "oauth_config": { + "scopes": { + "bot": [ + "chat:write", + "chat:write.customize", + "files:read", + "files:write", + "users:read", + "reactions:read", + "reactions:write", + "channels:history", + "groups:history", + "im:history", + "mpim:history", + "channels:read" + ] + } + }, + "settings": { + "event_subscriptions": { + "bot_events": [ + "message.channels", + "message.groups", + "message.im", + "message.mpim", + "reaction_added" + ] + }, + "interactivity": { + "is_enabled": true + }, + "org_deploy_enabled": false, + "socket_mode_enabled": true + } +} diff --git a/slackbot/bot_info.py b/slackbot/bot_info.py index a98f4dee..afba3be8 100644 --- a/slackbot/bot_info.py +++ b/slackbot/bot_info.py @@ -12,6 +12,7 @@ def get_botid(): slack_token = environ.get("SLACK_TOKEN", None) +slack_app_token = environ.get("SLACK_APP_TOKEN", None) slack_notification_channel = environ.get("SLACK_NOTIFICATION_CHANNEL", "seuron-alerts") botid = get_botid() workerid = "seuron-worker-"+environ["DEPLOYMENT"] diff --git a/slackbot/seuronbot.py b/slackbot/seuronbot.py index 73bf962d..63f2fb14 100644 --- a/slackbot/seuronbot.py +++ b/slackbot/seuronbot.py @@ -5,15 +5,23 @@ import itertools import difflib import concurrent.futures +import logging import time import tenacity +from threading import Event from slack_sdk.rtm_v2 import RTMClient +from slack_sdk.socket_mode import SocketModeClient +from slack_sdk.socket_mode.request import SocketModeRequest +from slack_sdk.socket_mode.response import SocketModeResponse +from slack_sdk.web import WebClient from bot_info import botid, workerid, broker_url from bot_utils import replyto, extract_command, update_slack_thread, create_run_token, send_message from airflow_api import check_running, set_variable, get_variable from kombu_helper import get_message, visible_messages from docker_helper import get_registry_data +logger = logging.getLogger(__name__) + _help_trigger = ["help"] @@ -71,21 +79,32 @@ class SeuronBot: hello_listeners = [] - def __init__(self, slack_token=None): + def __init__(self, slack_token=None, app_token=None): self.task_owner = "seuronbot" self.slack_token = slack_token + self.app_token = app_token + self.web_client = WebClient(token=slack_token) + + if self.app_token: + logger.info("Initializing bot in Socket Mode") + self.socket_client = SocketModeClient( + app_token=self.app_token, + web_client=self.web_client, + ) + self.socket_client.socket_mode_request_listeners.append( + self._handle_socket_mode_request + ) + else: + logger.info("Initializing bot in RTM mode") + self.rtmclient = RTMClient(token=slack_token) + self.rtmclient.on("message")(functools.partial(self.process_message.__func__, self)) + self.rtmclient.on("reaction_added")(functools.partial(self.process_reaction.__func__, self)) + self.rtmclient.on("hello")(functools.partial(self.process_hello.__func__, self)) - self.rtmclient = RTMClient(token=slack_token) - self.rtmclient.on("message")(functools.partial(self.process_message.__func__, self)) - self.rtmclient.on("reaction_added")(functools.partial(self.process_reaction.__func__, self)) - self.rtmclient.on("hello")(functools.partial(self.process_hello.__func__, self)) self.executor = concurrent.futures.ThreadPoolExecutor() def update_task_owner(self, msg): - sc = self.rtmclient.web_client - rc = sc.users_info( - user=msg['user'] - ) + rc = self.web_client.users_info(user=msg['user']) if rc["ok"]: self.task_owner = rc["user"]["profile"]["display_name"] @@ -137,7 +156,7 @@ def filter_msg(self, msg): if re.search(r"^{}[\s,:]".format(workerid), text, re.IGNORECASE): return True - def process_message(self, client: RTMClient, event: dict): + def process_message(self, client, event: dict): if self.filter_msg(event) or event.get("from_jupyter", False): handled = False check_image_updates(event) @@ -154,15 +173,32 @@ def process_message(self, client: RTMClient, event: dict): if not event.get("from_jupyter", False): self.update_task_owner(event) - def process_reaction(self, client: RTMClient, event: dict): + def process_reaction(self, client, event: dict): print("reaction added") print(json.dumps(event, indent=4)) - - def process_hello(self, client: RTMClient, event: dict): + def process_hello(self, client, event: dict): for listener in self.hello_listeners: listener() + def _handle_socket_mode_request(self, client: SocketModeClient, req: SocketModeRequest): + """Route Socket Mode envelope to the appropriate event handler.""" + response = SocketModeResponse(envelope_id=req.envelope_id) + client.send_socket_mode_response(response) + + if req.type == "events_api": + event = req.payload.get("event", {}) + event_type = event.get("type", "") + + if event_type == "message": + self.process_message(client, event) + elif event_type == "reaction_added": + self.process_reaction(client, event) + else: + logger.debug("Unhandled event type: %s", event_type) + elif req.type == "hello": + self.process_hello(client, {}) + @classmethod def on_hello(cls): def __call__(*args, **kwargs): @@ -219,7 +255,7 @@ def new_message_listener(context): wait=tenacity.wait_random_exponential(multiplier=1, max=60), ) def fetch_bot_messages(self, queue="bot-message-queue"): - client = self.rtmclient.web_client + client = self.web_client while True: msg_payload = get_message(broker_url, queue, timeout=30) if not msg_payload: @@ -255,7 +291,13 @@ def start(self): futures.append(self.executor.submit(self.fetch_bot_messages)) futures.append(self.executor.submit(self.fetch_jupyter_messages)) try: - self.rtmclient.start() + if self.app_token: + self.socket_client.connect() + logger.info("Socket Mode client connected") + self.process_hello(self.socket_client, {}) + Event().wait() + else: + self.rtmclient.start() except Exception: - pass + logger.exception("Error in bot main loop") concurrent.futures.wait(futures) diff --git a/slackbot/slack_bot.py b/slackbot/slack_bot.py index e811511e..1bb7db25 100644 --- a/slackbot/slack_bot.py +++ b/slackbot/slack_bot.py @@ -1,5 +1,5 @@ from airflow_api import set_variable -from bot_info import slack_token +from bot_info import slack_token, slack_app_token from seuronbot import SeuronBot import os @@ -66,5 +66,5 @@ def process_hello(): set_variable("webui_ip", "localhost") - seuronbot = SeuronBot(slack_token=slack_token) + seuronbot = SeuronBot(slack_token=slack_token, app_token=slack_app_token) seuronbot.start() diff --git a/start_seuronbot.local b/start_seuronbot.local index 389e98dc..fb8e2ed8 100755 --- a/start_seuronbot.local +++ b/start_seuronbot.local @@ -57,8 +57,16 @@ function generate_env() { worker_name=$(tr -dc 'a-z0-9' < /dev/urandom | head -c 4) - input_prompt "Token for slack RTM bot if you have one [xoxb-****]:" - read -r -s slack_token + input_prompt "App-level token for Socket Mode if you have one [xapp-****] (leave empty for RTM mode):" + read -r -s slack_app_token + + if [[ -n "${slack_app_token}" ]]; then + input_prompt "Bot token for Socket Mode [xoxb-****]:" + read -r -s slack_token + else + input_prompt "Token for slack RTM bot if you have one [xoxb-****]:" + read -r -s slack_token + fi if [[ -n "${slack_token}" ]]; then input_prompt "Slack channel to send generic bot messages [seuron-alerts]:" @@ -82,6 +90,10 @@ function generate_env() { # Add your slack bot token starts with xoxb- # https://api.slack.com/authentication/token-types#bot SLACK_TOKEN=${slack_token} +# App-level token for Socket Mode starts with xapp- +# Leave empty to use legacy RTM mode +# https://api.slack.com/apis/connections/socket +SLACK_APP_TOKEN=${slack_app_token} # Channel to send runtime notifications SLACK_NOTIFICATION_CHANNEL=${slack_notification_channel} # The bot will listen to commands starting with seuron-worker-example-deployment