diff --git a/CARS4U-Business Area and Action-AI function.ipynb b/CARS4U-Business Area and Action-AI function.ipynb index fc1311a..e19be2f 100644 --- a/CARS4U-Business Area and Action-AI function.ipynb +++ b/CARS4U-Business Area and Action-AI function.ipynb @@ -1,329 +1,299 @@ { - "nbformat_minor": 1, + "nbformat_minor": 1, "cells": [ { - "source": "\n \n \n \n \n
\"Icon\"Area and Action - AI function
", - "cell_type": "markdown", + "source": "\n \n \n \n \n
\"Icon\"Area and Action - AI function
", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\n\"Icon\" \n", - "cell_type": "markdown", + "source": "\n\"Icon\" \n", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "AI function with logic to sequentially call two models: BUSINESS_AREA and ACTION.", - "cell_type": "markdown", + "source": "AI function with logic to sequentially call two models: BUSINESS_AREA and ACTION.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Contents\n\n- [0. Setup](#setup)\n- [1. Itroduction](#introduction)\n- [2. Define AI function](#ai_function)\n- [3. Store the AI function in the repository](#persistence)\n- [4. Deploy AI function](#deploy)", - "cell_type": "markdown", + "source": "Contents\n\n- [0. Setup](#setup)\n- [1. Itroduction](#introduction)\n- [2. Define AI function](#ai_function)\n- [3. Store the AI function in the repository](#persistence)\n- [4. Deploy AI function](#deploy)", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\n## 0. Setup\n\nIn this section please use below cell to upgrade the `watson-machine-learning-client`.", - "cell_type": "markdown", + "source": "\n## 0. Setup\n\nIn this section please use below cell to upgrade the `watson-machine-learning-client`.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client\n!pip install --upgrade watson-machine-learning-client==1.0.260", - "cell_type": "code", - "metadata": {}, + "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client\n!pip install --upgrade watson-machine-learning-client==1.0.277", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Requirement already up-to-date: watson-machine-learning-client==1.0.260 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\nRequirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: ibm-cos-sdk in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client==1.0.260)\nRequirement not upgraded as not directly required: docutils>=0.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client==1.0.260)\n" } - ], + ], "execution_count": 1 - }, + }, { - "source": "\n## 1. Introduction\n\nModels to predict Business Area and Action based on client feedback will be embeded into AI function. An example of payload logging using AI function will be introduced.", - "cell_type": "markdown", + "source": "\n## 1. Introduction\n\nModels to predict Business Area and Action based on client feedback will be embeded into AI function. An example of payload logging using AI function will be introduced.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\n## 2. Define AI function", - "cell_type": "markdown", + "source": "\n## 2. Define AI function", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "In this section we define AI function and test it locally.", - "cell_type": "markdown", + "source": "In this section we define AI function and test it locally.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "### Template\n\nWe start with template version of AI function to present its structure in simplest case.", - "cell_type": "markdown", + "source": "### Template\n\nWe start with template version of AI function to present its structure in simplest case.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": " # Template version\n\n def score_template(payload):\n \"\"\"AI function example.\n\n # Here we present data schema to be used by function:\n Example:\n {\n \"fields\": [\"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\"],\n \"values\": [\n [\"Male\", \"M\", 2, 48, \"Inactive\"],\n [\"Female\", \"S\", 0, 23, \"Inactive\"]\n ]\n }\n \"\"\"\n fields = payload['fields'] + ['Prediction', 'Probability']\n values = [record + [int(record[0] == 'Male'), 0.9] for record in payload['values']]\n return { 'fields': fields, 'values': values }\n\n template_payload = {\"fields\": [],\n \"values\": [[\"Male\", \"M\", 2, 48, \"Inactive\"], [\"Female\", \"S\", 0, 23, \"Inactive\"]]}\n\n print(score_template(template_payload))", - "cell_type": "markdown", + "source": " # Template version\n\n def score_template(payload):\n \"\"\"AI function example.\n\n # Here we present data schema to be used by function:\n Example:\n {\n \"fields\": [\"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\"],\n \"values\": [\n [\"Male\", \"M\", 2, 48, \"Inactive\"],\n [\"Female\", \"S\", 0, 23, \"Inactive\"]\n ]\n }\n \"\"\"\n fields = payload['fields'] + ['Prediction', 'Probability']\n values = [record + [int(record[0] == 'Male'), 0.9] for record in payload['values']]\n return { 'fields': fields, 'values': values }\n\n template_payload = {\"fields\": [],\n \"values\": [[\"Male\", \"M\", 2, 48, \"Inactive\"], [\"Female\", \"S\", 0, 23, \"Inactive\"]]}\n\n print(score_template(template_payload))", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "### Deployed models metadata extraction\n\nWe have to identify model deployments to be used before we define AI function. We extract scoring url and information about model version.", - "cell_type": "markdown", + "source": "### Deployed models metadata extraction\n\nWe have to identify model deployments to be used before we define AI function. We extract scoring url and information about model version.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient", - "cell_type": "code", - "metadata": {}, + "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stderr", + "output_type": "stream", + "name": "stderr", "text": "2018-07-25 09:18:56,417 - watson_machine_learning_client.wml_client_error - WARNING - Failure during scoring. (POST https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/e2b5927f-9a0d-49dc-b731-39272f08b6bb/online)\nStatus code: 400, body: {\n \"trace\": \"1d2b26da3128fe8e81b5956956655b37\",\n \"errors\": [{\n \"code\": \"invalid_payload\",\n \"message\": \"Input Json parsing failed with error: invalid field: Business_Area\"\n }]\n}\n2018-07-25 09:19:58,125 - watson_machine_learning_client.wml_client_error - WARNING - Failure during scoring. (POST https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/e2b5927f-9a0d-49dc-b731-39272f08b6bb/online)\nStatus code: 400, body: {\n \"trace\": \"dd0c563bceae5b91bd3632472ad04a02\",\n \"errors\": [{\n \"code\": \"invalid_payload\",\n \"message\": \"Input Json parsing failed with error: struct already has field: Business_Area\"\n }]\n}\n2018-07-25 09:25:54,658 - watson_machine_learning_client.wml_client_error - WARNING - Failure during scoring. (POST https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/e2b5927f-9a0d-49dc-b731-39272f08b6bb/online)\nStatus code: 400, body: {\n \"trace\": \"ad6d81da5ada90ba743171226e32ed3f\",\n \"errors\": [{\n \"code\": \"invalid_payload\",\n \"message\": \"Input Json parsing failed with error: struct already has field: label\"\n }]\n}\n" } - ], + ], "execution_count": 2 - }, + }, { - "source": "# @hidden_cell\n# How to get associated service credentials\n\nwml_credentials = {\n \"instance_id\": \"000263d8-04e0-4060-ad69-fcfe40069018\",\n \"password\": \"7419325b-3de4-476c-94cb-4b158fa335b0\",\n \"url\": \"https://us-south.ml.cloud.ibm.com\",\n \"username\": \"cdc4b5da-8380-42f1-bd82-da044b283959\"\n}", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "# @hidden_cell\n# How to get associated service credentials\n\nwml_credentials = {\n \"instance_id\": \"000263d8-04e0-4060-ad69-fcfe40069018\",\n \"password\": \"7419325b-3de4-476c-94cb-4b158fa335b0\",\n \"url\": \"https://us-south.ml.cloud.ibm.com\",\n \"username\": \"cdc4b5da-8380-42f1-bd82-da044b283959\"\n}", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 3 - }, + }, { - "source": "client = WatsonMachineLearningAPIClient(wml_credentials)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "client = WatsonMachineLearningAPIClient(wml_credentials)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 4 - }, + }, { - "source": "We list all deployed models predicting `Action` and `Business Area` to select one to be used in the AI function.", - "cell_type": "markdown", + "source": "We list all deployed models predicting `Action` and `Business Area` to select one to be used in the AI function.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "client.deployments.list()", - "cell_type": "code", - "metadata": {}, + "source": "client.deployments.list()", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "------------------------------------ --------------------------------------------------------------------- ------ -------------- ------------------------ -------------- ----------\nGUID NAME TYPE STATE CREATED FRAMEWORK ASSET TYPE\ne2b5927f-9a0d-49dc-b731-39272f08b6bb CARS4U - Action Model Deployment online DEPLOY_SUCCESS 2018-07-25T08:49:12.081Z mllib-2.1 model\n9cfa6455-bba2-4cf3-bb71-e534c514dede CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-25T07:18:24.455Z n/a function\nd3b1a95b-1550-49ea-b6ee-4e1d75bca02e CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-25T06:52:23.826Z n/a function\n20b44d48-09e0-422c-9f86-ba61fd124836 CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-25T06:20:03.776Z n/a function\naeb667ed-5070-4c88-b12f-333e1325d1ad CARS4U - Sentiment Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T14:49:53.306Z n/a function\nb60d035e-fbd1-42e7-81f6-0c918d6bfa42 CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T13:48:37.871Z n/a function\ndfb792bd-c11f-4050-93d7-34cb0c91fe87 CARS4U - Sentiment Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T13:41:49.123Z n/a function\n4754aa03-bd0f-49b3-836a-835ed6d3bc67 CARS4U - Sentiment Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T13:38:37.692Z n/a function\nd8b4dcbd-70c2-4901-9618-3ba28cc7bdca CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T13:16:28.453Z n/a function\n56f494e5-d923-422c-b688-68f7324867f3 CARS4U - Business area and Action Prediction - AI Function Deployment online DEPLOY_FAILURE 2018-07-24T13:11:20.100Z n/a function\n505e603d-5574-4a68-bc46-bf95db85db82 CARS4U - Sentiment Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T12:05:09.265Z n/a function\nc379de8d-d78b-45d8-b9f8-e19a1e28827a Car Rental Deployment - Business Area online DEPLOY_SUCCESS 2018-07-24T12:04:34.695Z mllib-2.1 model\n51d847b8-b226-4598-9da3-8b409da81426 CARS4U - Sentiment Prediction - AI Function Deployment online DEPLOY_SUCCESS 2018-07-24T11:59:29.680Z n/a function\n726e333a-73a2-4720-b249-3be832903b4e CARS4U - Satisfaction Prediction Model Deployment online DEPLOY_SUCCESS 2018-07-24T09:49:11.282Z tensorflow-1.5 model\nb344d58c-590d-4a93-a007-23693947ce31 CARS4U - Business Area Prediction Model Deployment online DEPLOY_SUCCESS 2018-07-24T09:47:40.251Z mllib-2.1 model\n------------------------------------ --------------------------------------------------------------------- ------ -------------- ------------------------ -------------- ----------\n" } - ], + ], "execution_count": 5 - }, + }, { - "source": "We select deployed model to predict `Business Area` and model to recommend `Action`. We have to extract scoring endpoints for two models.", - "cell_type": "markdown", + "source": "We select deployed model to predict `Business Area` and model to recommend `Action`. We have to extract scoring endpoints for two models.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "deployments = client.deployments.get_details()['resources']\n\nfor x in deployments:\n if (x['entity']['name'] == 'CARS4U - Action Model Deployment'):\n action_scoring_url = client.deployments.get_scoring_url(x)\n if (x['entity']['name'] == 'CARS4U - Business Area Prediction Model Deployment'):\n business_area_scoring_url = client.deployments.get_scoring_url(x)\n \nprint(\"Action url: \", action_scoring_url)\nprint(\"Business area url: \", business_area_scoring_url)", - "cell_type": "code", - "metadata": {}, + "source": "deployments = client.deployments.get_details()['resources']\n\nfor x in deployments:\n if (x['entity']['name'] == 'CARS4U - Action Model Deployment'):\n action_scoring_url = client.deployments.get_scoring_url(x)\n if (x['entity']['name'] == 'CARS4U - Business Area Prediction Model Deployment'):\n business_area_scoring_url = client.deployments.get_scoring_url(x)\n \nprint(\"Action url: \", action_scoring_url)\nprint(\"Business area url: \", business_area_scoring_url)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Action url: https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/e2b5927f-9a0d-49dc-b731-39272f08b6bb/online\nBusiness area url: https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/b344d58c-590d-4a93-a007-23693947ce31/online\n" } - ], + ], "execution_count": 6 - }, + }, { - "source": "### AI function definition\n\nWe define AI function to be deployed in the IBM Cloud.", - "cell_type": "markdown", + "source": "### AI function definition\n\nWe define AI function to be deployed in the IBM Cloud.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Put all parameters required to define AI function in dictionary.", - "cell_type": "markdown", + "source": "Put all parameters required to define AI function in dictionary.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "ai_params = {\"wml_credentials\": wml_credentials, \n \"business_area_scoring_url\": business_area_scoring_url, \n \"action_scoring_url\": action_scoring_url}", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "ai_params = {\"wml_credentials\": wml_credentials, \n \"business_area_scoring_url\": business_area_scoring_url, \n \"action_scoring_url\": action_scoring_url}", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 7 - }, + }, { - "source": "def score_generator(params=ai_params):\n \n from watson_machine_learning_client import WatsonMachineLearningAPIClient\n\n wml_credentials = params[\"wml_credentials\"]\n business_area_scoring_url = params[\"business_area_scoring_url\"]\n action_scoring_url = params[\"action_scoring_url\"]\n\n client = WatsonMachineLearningAPIClient(wml_credentials)\n\n def score(payload):\n \"\"\"AI function with model version.\n\n Example:\n {\"fields\": [\"ID\", \"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\", \"Car_Owner\", \"Customer_Service\", \"Satisfaction\"],\n \"values\": [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1]]}\n \"\"\"\n \n scores_area = client.deployments.score(business_area_scoring_url, payload)\n idx_area = [i for i, x in enumerate(scores_area[\"fields\"]) if (x == 'probability') or (x == 'predictedLabel')]\n\n fields_from_action = payload[\"fields\"] + [\"predictedLabel\"]\n idx_from_action = set([i for i, x in enumerate(scores_area[\"fields\"]) if x in fields_from_action])\n values_action = [[v for i, v in enumerate(rec) if i in idx_from_action] for rec in scores_area[\"values\"]]\n \n payload_action = {\"fields\": payload[\"fields\"] + [\"Business_Area\"], \n \"values\": values_action}\n \n scores_action = client.deployments.score(action_scoring_url, payload_action)\n idx_action = [i for i, x in enumerate(scores_action[\"fields\"]) if (x == 'probability') or (x == 'predictedLabel')]\n\n values = [rec + [scores_area['values'][i][idx_area[0]], scores_area['values'][i][idx_area[1]], scores_action['values'][i][idx_action[0]], scores_action['values'][i][idx_action[1]]] for i, rec in enumerate(payload['values'])]\n\n fields = payload['fields'] + ['Probability_Area', 'Prediction_Area'] + ['Probability_Action', 'Prediction_Action']\n\n return {'fields': fields, 'values': values}\n\n return score", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "def score_generator(params=ai_params):\n \n from watson_machine_learning_client import WatsonMachineLearningAPIClient\n\n wml_credentials = params[\"wml_credentials\"]\n business_area_scoring_url = params[\"business_area_scoring_url\"]\n action_scoring_url = params[\"action_scoring_url\"]\n\n client = WatsonMachineLearningAPIClient(wml_credentials)\n\n def score(payload):\n \"\"\"AI function with model version.\n\n Example:\n {\"fields\": [\"ID\", \"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\", \"Car_Owner\", \"Customer_Service\", \"Satisfaction\"],\n \"values\": [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1]]}\n \"\"\"\n \n scores_area = client.deployments.score(business_area_scoring_url, payload)\n idx_area = [i for i, x in enumerate(scores_area[\"fields\"]) if (x == 'probability') or (x == 'predictedLabel')]\n\n fields_from_action = payload[\"fields\"] + [\"predictedLabel\"]\n idx_from_action = set([i for i, x in enumerate(scores_area[\"fields\"]) if x in fields_from_action])\n values_action = [[v for i, v in enumerate(rec) if i in idx_from_action] for rec in scores_area[\"values\"]]\n \n payload_action = {\"fields\": payload[\"fields\"] + [\"Business_Area\"], \n \"values\": values_action}\n \n scores_action = client.deployments.score(action_scoring_url, payload_action)\n idx_action = [i for i, x in enumerate(scores_action[\"fields\"]) if (x == 'probability') or (x == 'predictedLabel')]\n\n values = [rec + [scores_area['values'][i][idx_area[0]], scores_area['values'][i][idx_area[1]], scores_action['values'][i][idx_action[0]], scores_action['values'][i][idx_action[1]]] for i, rec in enumerate(payload['values'])]\n\n fields = payload['fields'] + ['Probability_Area', 'Prediction_Area'] + ['Probability_Action', 'Prediction_Action']\n\n return {'fields': fields, 'values': values}\n\n return score", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 20 - }, + }, { - "source": "### AI function local test\n\nLet's test defined AI function in notebook before we deploy it in the IBM Cloud. We start with preparing sample payload for `score` function.", - "cell_type": "markdown", + "source": "### AI function local test\n\nLet's test defined AI function in notebook before we deploy it in the IBM Cloud. We start with preparing sample payload for `score` function.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "sample_payload = {\"fields\": [\"ID\", \"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\", \"Car_Owner\", \"Customer_Service\", \"Satisfaction\"],\n \"values\": [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1]]} ", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "sample_payload = {\"fields\": [\"ID\", \"Gender\", \"Status\", \"Children\", \"Age\", \"Customer_Status\", \"Car_Owner\", \"Customer_Service\", \"Satisfaction\"],\n \"values\": [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1]]} ", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 21 - }, + }, { - "source": "Let's test AI function locally before we deploy it to the IBM Cloud.", - "cell_type": "markdown", + "source": "Let's test AI function locally before we deploy it to the IBM Cloud.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "score = score_generator()\nscores_ai = score(sample_payload)\nprint(scores_ai)", - "cell_type": "code", - "metadata": {}, + "source": "score = score_generator()\nscores_ai = score(sample_payload)\nprint(scores_ai)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'fields': ['ID', 'Gender', 'Status', 'Children', 'Age', 'Customer_Status', 'Car_Owner', 'Customer_Service', 'Satisfaction', 'Probability_Area', 'Prediction_Area', 'Probability_Action', 'Prediction_Action'], 'values': [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1, [0.3923076923076923, 0.3153846153846154, 0.046153846153846156, 0.06923076923076923, 0.05384615384615385, 0.06153846153846154, 0.038461538461538464, 0.023076923076923078], 'Service: Knowledge', [1.0, 0.0, 0.0, 0.0, 0.0], 'NA']]}\n" } - ], + ], "execution_count": 22 - }, + }, { - "source": "\n## 3. Store the AI function in the repository", - "cell_type": "markdown", + "source": "\n## 3. Store the AI function in the repository", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "In this section we store AI function to Watson machine Learning repository.", - "cell_type": "markdown", + "source": "In this section we store AI function to Watson machine Learning repository.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Required prerequisite to store AI function is runtime specification.", - "cell_type": "markdown", - "metadata": {} - }, - { - "source": "runtime_meta = {\n client.runtime_specs.ConfigurationMetaNames.NAME: \"Basic runtime specification\",\n client.runtime_specs.ConfigurationMetaNames.DESCRIPTION: \"Runtime for AI function\",\n client.runtime_specs.ConfigurationMetaNames.PLATFORM: {\n \"name\": \"python\",\n \"version\": \"3.5\"\n }}", - "cell_type": "code", - "metadata": {}, - "outputs": [], - "execution_count": 23 - }, - { - "source": "runtime_details = client.runtime_specs.create(runtime_meta)\nruntime_url = client.runtime_specs.get_url(runtime_details)\nprint(runtime_url)", - "cell_type": "code", - "metadata": {}, - "outputs": [ - { - "output_type": "stream", - "name": "stdout", - "text": "https://us-south.ml.cloud.ibm.com/v4/runtimes/c13820b5-ccc1-492b-ad8d-d49ca49184a2\n" - } - ], - "execution_count": 24 - }, - { - "source": "Now we are ready to save function to Watson Machine Learning repository.", - "cell_type": "markdown", - "metadata": {} - }, - { - "source": "meta_data = {\n client.repository.FunctionMetaNames.NAME: 'CARS4U - Business area and Action Prediction - AI Function',\n client.repository.FunctionMetaNames.RUNTIME_URL: runtime_url\n}\n\nfunction_details = client.repository.store_function(meta_props=meta_data, function=score_generator)", - "cell_type": "code", - "metadata": {}, + "source": "function_details = client.repository.store_function(score_generator, 'CARS4U - Business area and Action Prediction - AI Function')", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Recognized generator function.\n" } - ], + ], "execution_count": 25 - }, + }, { - "source": "ai_function_uid = client.repository.get_function_uid(function_details)\nprint(ai_function_uid)", - "cell_type": "code", - "metadata": {}, + "source": "ai_function_uid = client.repository.get_function_uid(function_details)\nprint(ai_function_uid)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "6df12de6-763f-46cb-8388-eca249fa2da7\n" } - ], + ], "execution_count": 26 - }, + }, { - "source": "\n## 4. Deploy AI function", - "cell_type": "markdown", + "source": "\n## 4. Deploy AI function", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "In this section we deploy AI function in the IBM Cloud and test created deployment using sample payload.", - "cell_type": "markdown", + "source": "In this section we deploy AI function in the IBM Cloud and test created deployment using sample payload.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "function_deployment_details = client.deployments.create(ai_function_uid, \"CARS4U - Business area and Action Prediction - AI Function Deployment\")", - "cell_type": "code", - "metadata": {}, + "source": "function_deployment_details = client.deployments.create(ai_function_uid, \"CARS4U - Business area and Action Prediction - AI Function Deployment\")", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '6df12de6-763f-46cb-8388-eca249fa2da7' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_IN_PROGRESS....\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='1d668efc-5c1a-44da-9e69-248b2f65924a'\n------------------------------------------------------------------------------------------------\n\n\n" } - ], + ], "execution_count": 27 - }, + }, { - "source": "### AI function online deployment test\n\nNow we are ready to test created deployment.", - "cell_type": "markdown", + "source": "### AI function online deployment test\n\nNow we are ready to test created deployment.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "scoring_url = client.deployments.get_scoring_url(function_deployment_details)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "scoring_url = client.deployments.get_scoring_url(function_deployment_details)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 28 - }, + }, { - "source": "scoring_results = client.deployments.score(scoring_url, sample_payload)\nprint(scoring_results)", - "cell_type": "code", - "metadata": {}, + "source": "scoring_results = client.deployments.score(scoring_url, sample_payload)\nprint(scoring_results)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'fields': ['ID', 'Gender', 'Status', 'Children', 'Age', 'Customer_Status', 'Car_Owner', 'Customer_Service', 'Satisfaction', 'Probability_Area', 'Prediction_Area', 'Probability_Action', 'Prediction_Action'], 'values': [[2624, 'Male', 'S', 0, 49.27, 'Active', 'No', \"Good experience with all the rental co.'s I contacted. I Just called with rental dates and received pricing and selected rental co.\", 1, [0.3923076923076923, 0.3153846153846154, 0.046153846153846156, 0.06923076923076923, 0.05384615384615385, 0.06153846153846154, 0.038461538461538464, 0.023076923076923078], 'Service: Knowledge', [1.0, 0.0, 0.0, 0.0, 0.0], 'NA']]}\n" } - ], + ], "execution_count": 29 - }, + }, { - "source": "---", - "cell_type": "markdown", + "source": "---", + "cell_type": "markdown", "metadata": {} } - ], + ], "metadata": { "kernelspec": { - "display_name": "Python 3.5", - "name": "python3", + "display_name": "Python 3.5", + "name": "python3", "language": "python" - }, + }, "language_info": { - "mimetype": "text/x-python", - "nbconvert_exporter": "python", - "version": "3.5.5", - "name": "python", - "pygments_lexer": "ipython3", - "file_extension": ".py", + "mimetype": "text/x-python", + "nbconvert_exporter": "python", + "version": "3.5.5", + "name": "python", + "pygments_lexer": "ipython3", + "file_extension": ".py", "codemirror_mode": { - "version": 3, + "version": 3, "name": "ipython" } } - }, + }, "nbformat": 4 -} \ No newline at end of file +} diff --git a/CARS4U-Satisfaction Prediction-Keras-AI function.ipynb b/CARS4U-Satisfaction Prediction-Keras-AI function.ipynb index 102d1f5..59304e3 100644 --- a/CARS4U-Satisfaction Prediction-Keras-AI function.ipynb +++ b/CARS4U-Satisfaction Prediction-Keras-AI function.ipynb @@ -1,692 +1,672 @@ { - "nbformat_minor": 1, + "nbformat_minor": 1, "cells": [ { - "source": "\n \n \n \n \n
\"Icon\"Customer Satisfaction Prediction
", - "cell_type": "markdown", + "source": "\n \n \n \n \n
\"Icon\"Customer Satisfaction Prediction
", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\"Icon\"", - "cell_type": "markdown", + "source": "\"Icon\"", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Keras model and AI function to determine if comment is a complain.", - "cell_type": "markdown", + "source": "Keras model and AI function to determine if comment is a complain.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Contents\n\n- [0. Setup](#setup)\n- [1. Introduction](#introduction)\n- [2. Load and explore data](#load)\n- [3. Create Keras model using TensorFlow backend](#model)\n- [4. Store the model in the repository](#persistence)\n- [5. Deploy the model](#deployment)\n- [6. AI function](#ai_function)", - "cell_type": "markdown", + "source": "Contents\n\n- [0. Setup](#setup)\n- [1. Introduction](#introduction)\n- [2. Load and explore data](#load)\n- [3. Create Keras model using TensorFlow backend](#model)\n- [4. Store the model in the repository](#persistence)\n- [5. Deploy the model](#deployment)\n- [6. AI function](#ai_function)", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\n## 0. Setup", - "cell_type": "markdown", + "source": "\n## 0. Setup", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Install TensorFlow version 1.5 and newest version of watson-machine-learning-client.", - "cell_type": "markdown", + "source": "Install TensorFlow version 1.5 and newest version of watson-machine-learning-client.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "!pip install --upgrade tensorflow==1.5", - "cell_type": "code", - "metadata": {}, + "source": "!pip install --upgrade tensorflow==1.5", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Requirement already up-to-date: tensorflow==1.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\nRequirement not upgraded as not directly required: tensorflow-tensorboard<1.6.0,>=1.5.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: wheel>=0.26 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: protobuf>=3.4.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: numpy>=1.12.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: absl-py>=0.1.6 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow==1.5)\nRequirement not upgraded as not directly required: markdown>=2.6.8 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)\nRequirement not upgraded as not directly required: werkzeug>=0.11.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)\nRequirement not upgraded as not directly required: html5lib==0.9999999 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)\nRequirement not upgraded as not directly required: bleach==1.5.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from tensorflow-tensorboard<1.6.0,>=1.5.0->tensorflow==1.5)\nRequirement not upgraded as not directly required: setuptools in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from protobuf>=3.4.0->tensorflow==1.5)\n" } - ], + ], "execution_count": 30 - }, + }, { - "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client\n!pip install --upgrade watson-machine-learning-client==1.0.260", - "cell_type": "code", - "metadata": {}, + "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client\n!pip install --upgrade watson-machine-learning-client==1.0.277", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Requirement already up-to-date: watson-machine-learning-client in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\nRequirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: ibm-cos-sdk in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client)\nRequirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: ibm-cos-sdk-s3transfer==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client)\nRequirement not upgraded as not directly required: ibm-cos-sdk-core==2.*,>=2.0.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk->watson-machine-learning-client)\nRequirement not upgraded as not directly required: jmespath<1.0.0,>=0.7.1 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client)\nRequirement not upgraded as not directly required: docutils>=0.10 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from ibm-cos-sdk-core==2.*,>=2.0.0->ibm-cos-sdk->watson-machine-learning-client)\n" } - ], + ], "execution_count": 31 - }, + }, { - "source": "\n## 1. Introduction", - "cell_type": "markdown", + "source": "\n## 1. Introduction", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "This notebook trains a **Keras** (TensorFlow) model to predict customer satisfaction based on provided feedback. Notebook also shows usage of **AI Function** for deep learning model data preprocessing required before model scoring.", - "cell_type": "markdown", + "source": "This notebook trains a **Keras** (TensorFlow) model to predict customer satisfaction based on provided feedback. Notebook also shows usage of **AI Function** for deep learning model data preprocessing required before model scoring.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\n## 2. Load and explore data", - "cell_type": "markdown", + "source": "\n## 2. Load and explore data", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "In this section the data is loaded as pandas dataframe.", - "cell_type": "markdown", + "source": "In this section the data is loaded as pandas dataframe.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "\nfrom ibmdbpy import IdaDataBase, IdaDataFrame\n\n# @hidden_cell\n# This connection object is used to access your data and contains your credentials.\n# You might want to remove those credentials before you share your notebook.\nidadb_c166344e776040b39f477655199897f8 = IdaDataBase(dsn='DASHDB;Database=BLUDB;Hostname=dashdb-entry-yp-dal10-01.services.dal.bluemix.net;Port=50000;PROTOCOL=TCPIP;UID=dash5120;PWD=G5_CehiL4_Ux')\n\ndata_df = IdaDataFrame(idadb_c166344e776040b39f477655199897f8, 'DASH5120.CAR_RENTAL_TRAINING').as_dataframe()\ndata_df.head()\n\n# You can close the database connection with the following code. Please keep the comment line with the @hidden_cell tag,\n# because the close function displays parts of the credentials.\n# @hidden_cell\n# idadb_c166344e776040b39f477655199897f8.close()\n# To learn more about the ibmdby package, please read the documentation: http://pythonhosted.org/ibmdbpy/\n", - "cell_type": "code", - "metadata": {}, + "source": "\nfrom ibmdbpy import IdaDataBase, IdaDataFrame\n\n# @hidden_cell\n# This connection object is used to access your data and contains your credentials.\n# You might want to remove those credentials before you share your notebook.\nidadb_c166344e776040b39f477655199897f8 = IdaDataBase(dsn='DASHDB;Database=BLUDB;Hostname=dashdb-entry-yp-dal10-01.services.dal.bluemix.net;Port=50000;PROTOCOL=TCPIP;UID=dash5120;PWD=G5_CehiL4_Ux')\n\ndata_df = IdaDataFrame(idadb_c166344e776040b39f477655199897f8, 'DASH5120.CAR_RENTAL_TRAINING').as_dataframe()\ndata_df.head()\n\n# You can close the database connection with the following code. Please keep the comment line with the @hidden_cell tag,\n# because the close function displays parts of the credentials.\n# @hidden_cell\n# idadb_c166344e776040b39f477655199897f8.close()\n# To learn more about the ibmdby package, please read the documentation: http://pythonhosted.org/ibmdbpy/\n", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "execute_result", + "output_type": "execute_result", "data": { - "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
IDGenderStatusChildrenAgeCustomer_StatusCar_OwnerCustomer_ServiceSatisfactionBusiness_AreaAction
074MaleM126.26ActiveNono wait for pick up and drop off was great, he...1Product: InformationNA
183FemaleM248.85InactiveYesI thought the representative handled the initi...0Product: Availability/Variety/SizeFree Upgrade
2140FemaleS036.92InactiveNoEveryone was very cooperative. The auto was r...1Product: FunctioningNA
3191MaleM045.51InactiveYeswhat customer service? It was a nightmare0Service: KnowledgeVoucher
4239MaleM146.00InactiveYesThey did not have the auto I wanted. upgraded...0Product: Availability/Variety/SizeFree Upgrade
\n
", + "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
IDGenderStatusChildrenAgeCustomer_StatusCar_OwnerCustomer_ServiceSatisfactionBusiness_AreaAction
074MaleM126.26ActiveNono wait for pick up and drop off was great, he...1Product: InformationNA
183FemaleM248.85InactiveYesI thought the representative handled the initi...0Product: Availability/Variety/SizeFree Upgrade
2140FemaleS036.92InactiveNoEveryone was very cooperative. The auto was r...1Product: FunctioningNA
3191MaleM045.51InactiveYeswhat customer service? It was a nightmare0Service: KnowledgeVoucher
4239MaleM146.00InactiveYesThey did not have the auto I wanted. upgraded...0Product: Availability/Variety/SizeFree Upgrade
\n
", "text/plain": " ID Gender Status Children Age Customer_Status Car_Owner \\\n0 74 Male M 1 26.26 Active No \n1 83 Female M 2 48.85 Inactive Yes \n2 140 Female S 0 36.92 Inactive No \n3 191 Male M 0 45.51 Inactive Yes \n4 239 Male M 1 46.00 Inactive Yes \n\n Customer_Service Satisfaction \\\n0 no wait for pick up and drop off was great, he... 1 \n1 I thought the representative handled the initi... 0 \n2 Everyone was very cooperative. The auto was r... 1 \n3 what customer service? It was a nightmare 0 \n4 They did not have the auto I wanted. upgraded... 0 \n\n Business_Area Action \n0 Product: Information NA \n1 Product: Availability/Variety/Size Free Upgrade \n2 Product: Functioning NA \n3 Service: Knowledge Voucher \n4 Product: Availability/Variety/Size Free Upgrade " - }, - "execution_count": 69, + }, + "execution_count": 69, "metadata": {} } - ], + ], "execution_count": 69 - }, + }, { - "source": "**Note:** 0 - not satisfied, 1 - satisfied", - "cell_type": "markdown", + "source": "**Note:** 0 - not satisfied, 1 - satisfied", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Extract needed columns and count number of records.", - "cell_type": "markdown", + "source": "Extract needed columns and count number of records.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "complain_data = data_df[['Customer_Service', 'Satisfaction']]", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "complain_data = data_df[['Customer_Service', 'Satisfaction']]", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 70 - }, + }, { - "source": "print(complain_data.count())", - "cell_type": "code", - "metadata": {}, + "source": "print(complain_data.count())", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Customer_Service 482\nSatisfaction 482\ndtype: int64\n" } - ], + ], "execution_count": 71 - }, + }, { - "source": "\n## 3. Create Keras model using TensorFlow backend\n", - "cell_type": "markdown", + "source": "\n## 3. Create Keras model using TensorFlow backend\n", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "from keras.preprocessing.text import Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Embedding, LSTM, SpatialDropout1D\nfrom sklearn.model_selection import train_test_split\nimport os\nimport numpy\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import Conv1D, MaxPooling1D\nfrom keras.layers.embeddings import Embedding\nfrom keras.preprocessing import sequence", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "from keras.preprocessing.text import Tokenizer\nfrom keras.preprocessing.sequence import pad_sequences\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Embedding, LSTM, SpatialDropout1D\nfrom sklearn.model_selection import train_test_split\nimport os\nimport numpy\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import Conv1D, MaxPooling1D\nfrom keras.layers.embeddings import Embedding\nfrom keras.preprocessing import sequence", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 223 - }, + }, { - "source": "### 3.1 Prepare data", - "cell_type": "markdown", + "source": "### 3.1 Prepare data", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "max_fatures = 500\n\nfor idx,row in complain_data.iterrows():\n row[0] = row[0].replace('rt',' ')\n\ntokenizer = Tokenizer(num_words=max_fatures, split=' ')\ntokenizer.fit_on_texts(complain_data['Customer_Service'].values)\nX = tokenizer.texts_to_sequences(complain_data['Customer_Service'].values)\n\nmaxlen = 50\n\nX = pad_sequences(X, maxlen=maxlen)\nprint(X.shape)", - "cell_type": "code", - "metadata": {}, + "source": "max_fatures = 500\n\nfor idx,row in complain_data.iterrows():\n row[0] = row[0].replace('rt',' ')\n\ntokenizer = Tokenizer(num_words=max_fatures, split=' ')\ntokenizer.fit_on_texts(complain_data['Customer_Service'].values)\nX = tokenizer.texts_to_sequences(complain_data['Customer_Service'].values)\n\nmaxlen = 50\n\nX = pad_sequences(X, maxlen=maxlen)\nprint(X.shape)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "(482, 50)\n" } - ], + ], "execution_count": 190 - }, + }, { - "source": "Split into train and test datasets.", - "cell_type": "markdown", + "source": "Split into train and test datasets.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Y = complain_data['Satisfaction'].values\nX_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.33, random_state = 42)\n\nprint(X_train.shape,Y_train.shape)\nprint(X_test.shape,Y_test.shape)", - "cell_type": "code", - "metadata": {}, + "source": "Y = complain_data['Satisfaction'].values\nX_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.33, random_state = 42)\n\nprint(X_train.shape,Y_train.shape)\nprint(X_test.shape,Y_test.shape)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "(322, 50) (322,)\n(160, 50) (160,)\n" } - ], + ], "execution_count": 191 - }, + }, { - "source": "### 3.2 Design and train model", - "cell_type": "markdown", + "source": "### 3.2 Design and train model", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Create the network definition based on Gated Recurrent Unit (Cho et al. 2014).", - "cell_type": "markdown", + "source": "Create the network definition based on Gated Recurrent Unit (Cho et al. 2014).", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "embedding_vector_length = 32\n\nmodel = Sequential()\nmodel.add(Embedding(max_fatures, embedding_vector_length, input_length=maxlen))\nmodel.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu'))\nmodel.add(MaxPooling1D(pool_size=2))\nmodel.add(LSTM(100))\nmodel.add(Dense(1, activation='sigmoid'))\nmodel.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\nprint(model.summary())", - "cell_type": "code", - "metadata": {}, + "source": "embedding_vector_length = 32\n\nmodel = Sequential()\nmodel.add(Embedding(max_fatures, embedding_vector_length, input_length=maxlen))\nmodel.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu'))\nmodel.add(MaxPooling1D(pool_size=2))\nmodel.add(LSTM(100))\nmodel.add(Dense(1, activation='sigmoid'))\nmodel.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\nprint(model.summary())", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "_________________________________________________________________\nLayer (type) Output Shape Param # \n=================================================================\nembedding_3 (Embedding) (None, 50, 32) 16000 \n_________________________________________________________________\nconv1d_3 (Conv1D) (None, 50, 32) 3104 \n_________________________________________________________________\nmax_pooling1d_3 (MaxPooling1 (None, 25, 32) 0 \n_________________________________________________________________\nlstm_3 (LSTM) (None, 100) 53200 \n_________________________________________________________________\ndense_3 (Dense) (None, 1) 101 \n=================================================================\nTotal params: 72,405\nTrainable params: 72,405\nNon-trainable params: 0\n_________________________________________________________________\nNone\n" } - ], + ], "execution_count": 75 - }, + }, { - "source": "Train the model.", - "cell_type": "markdown", + "source": "Train the model.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=20, batch_size=64)", - "cell_type": "code", - "metadata": {}, + "source": "history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=20, batch_size=64)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Train on 322 samples, validate on 160 samples\nEpoch 1/20\n322/322 [==============================] - 2s 5ms/step - loss: 0.6895 - acc: 0.5807 - val_loss: 0.6856 - val_acc: 0.5500\nEpoch 2/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.6764 - acc: 0.5776 - val_loss: 0.6793 - val_acc: 0.5500\nEpoch 3/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.6632 - acc: 0.5776 - val_loss: 0.6708 - val_acc: 0.5500\nEpoch 4/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.6514 - acc: 0.5776 - val_loss: 0.6561 - val_acc: 0.5500\nEpoch 5/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.6335 - acc: 0.6708 - val_loss: 0.6326 - val_acc: 0.6875\nEpoch 6/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.5894 - acc: 0.7391 - val_loss: 0.5967 - val_acc: 0.6438\nEpoch 7/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.5129 - acc: 0.7919 - val_loss: 0.5280 - val_acc: 0.7688\nEpoch 8/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.4668 - acc: 0.8789 - val_loss: 0.4603 - val_acc: 0.8375\nEpoch 9/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.3542 - acc: 0.8727 - val_loss: 0.3710 - val_acc: 0.8187\nEpoch 10/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.2457 - acc: 0.9317 - val_loss: 0.3301 - val_acc: 0.8688\nEpoch 11/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1854 - acc: 0.9627 - val_loss: 0.2704 - val_acc: 0.9250\nEpoch 12/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1461 - acc: 0.9348 - val_loss: 0.2620 - val_acc: 0.9062\nEpoch 13/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1228 - acc: 0.9752 - val_loss: 0.2704 - val_acc: 0.9125\nEpoch 14/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1033 - acc: 0.9752 - val_loss: 0.2462 - val_acc: 0.9250\nEpoch 15/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.0815 - acc: 0.9783 - val_loss: 0.2297 - val_acc: 0.9250\nEpoch 16/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.0640 - acc: 0.9876 - val_loss: 0.3215 - val_acc: 0.9125\nEpoch 17/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1232 - acc: 0.9752 - val_loss: 0.6927 - val_acc: 0.8375\nEpoch 18/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.1275 - acc: 0.9658 - val_loss: 0.2533 - val_acc: 0.9313\nEpoch 19/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.0690 - acc: 0.9876 - val_loss: 0.2262 - val_acc: 0.9125\nEpoch 20/20\n322/322 [==============================] - 1s 3ms/step - loss: 0.0480 - acc: 0.9876 - val_loss: 0.2266 - val_acc: 0.9125\n" } - ], + ], "execution_count": 76 - }, + }, { - "source": "print(\"Best accuracy on test: %3.3f\" % numpy.array(history.history['val_acc']).max())", - "cell_type": "code", - "metadata": {}, + "source": "print(\"Best accuracy on test: %3.3f\" % numpy.array(history.history['val_acc']).max())", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Best accuracy on test: 0.931\n" } - ], + ], "execution_count": 78 - }, + }, { - "source": "**Note:** For purpose of this demo model tuning has been skipped.", - "cell_type": "markdown", + "source": "**Note:** For purpose of this demo model tuning has been skipped.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Store and archive the model on notebook filesystem.", - "cell_type": "markdown", + "source": "Store and archive the model on notebook filesystem.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "# evaluate the model\nscores = model.evaluate(X_test, Y_test, verbose=0)\nprint(\"Evaluation Accuracy: %.2f%%\" % (scores[1]*100))", - "cell_type": "code", - "metadata": {}, + "source": "# evaluate the model\nscores = model.evaluate(X_test, Y_test, verbose=0)\nprint(\"Evaluation Accuracy: %.2f%%\" % (scores[1]*100))", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Evaluation Accuracy: 91.25%\n" } - ], + ], "execution_count": 79 - }, + }, { - "source": "filename = 'complain_model.h5'\nmodel.save(filename)\n\n#compress keras model\ntar_filename = filename + \".tgz\"\ncmdstring = \"tar -zcvf \" + tar_filename + \" \" + filename\nos.system(cmdstring);", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "filename = 'complain_model.h5'\nmodel.save(filename)\n\n#compress keras model\ntar_filename = filename + \".tgz\"\ncmdstring = \"tar -zcvf \" + tar_filename + \" \" + filename\nos.system(cmdstring);", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 80 - }, + }, { - "source": "!ls -lat", - "cell_type": "code", - "metadata": {}, + "source": "!ls -lat", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "total 1692\r\n-rw-r----- 1 dsxuser dsxuser 818583 Jul 24 09:49 complain_model.h5.tgz\r\n-rw-r----- 1 dsxuser dsxuser 903992 Jul 24 09:49 complain_model.h5\r\ndrwxr-x--- 2 dsxuser dsxuser 4096 Jul 24 08:51 .\r\ndrwx------ 1 dsxuser dsxuser 4096 Jul 24 07:27 ..\r\n" } - ], + ], "execution_count": 81 - }, + }, { - "source": "\n## 4. Store the model in the repository", - "cell_type": "markdown", + "source": "\n## 4. Store the model in the repository", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 82 - }, + }, { - "source": "# @hidden_cell\n\nwml_credentials = {\n \"instance_id\": \"000263d8-04e0-4060-ad69-fcfe40069018\",\n \"password\": \"7419325b-3de4-476c-94cb-4b158fa335b0\",\n \"url\": \"https://us-south.ml.cloud.ibm.com\",\n \"username\": \"cdc4b5da-8380-42f1-bd82-da044b283959\"\n}\n", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "# @hidden_cell\n\nwml_credentials = {\n \"instance_id\": \"000263d8-04e0-4060-ad69-fcfe40069018\",\n \"password\": \"7419325b-3de4-476c-94cb-4b158fa335b0\",\n \"url\": \"https://us-south.ml.cloud.ibm.com\",\n \"username\": \"cdc4b5da-8380-42f1-bd82-da044b283959\"\n}\n", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 83 - }, + }, { - "source": "client = WatsonMachineLearningAPIClient(wml_credentials)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "client = WatsonMachineLearningAPIClient(wml_credentials)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 84 - }, + }, { - "source": "model_props = {\n client.repository.ModelMetaNames.NAME: \"CARS4U - Satisfaction Prediction Model\",\n client.repository.ModelMetaNames.FRAMEWORK_NAME: \"tensorflow\",\n client.repository.ModelMetaNames.FRAMEWORK_VERSION: \"1.5\",\n client.repository.ModelMetaNames.RUNTIME_NAME: \"python\",\n client.repository.ModelMetaNames.RUNTIME_VERSION: \"3.5\",\n client.repository.ModelMetaNames.FRAMEWORK_LIBRARIES: [{'name':'keras', 'version': '2.1.3'}]\n}\n\npublished_model_details = client.repository.store_model(model=tar_filename, meta_props=model_props) ", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "model_props = {\n client.repository.ModelMetaNames.NAME: \"CARS4U - Satisfaction Prediction Model\",\n client.repository.ModelMetaNames.FRAMEWORK_NAME: \"tensorflow\",\n client.repository.ModelMetaNames.FRAMEWORK_VERSION: \"1.5\",\n client.repository.ModelMetaNames.RUNTIME_NAME: \"python\",\n client.repository.ModelMetaNames.RUNTIME_VERSION: \"3.5\",\n client.repository.ModelMetaNames.FRAMEWORK_LIBRARIES: [{'name':'keras', 'version': '2.1.3'}]\n}\n\npublished_model_details = client.repository.store_model(model=tar_filename, meta_props=model_props) ", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 85 - }, + }, { - "source": "model_uid = client.repository.get_model_uid(published_model_details)\nprint(model_uid)", - "cell_type": "code", - "metadata": {}, + "source": "model_uid = client.repository.get_model_uid(published_model_details)\nprint(model_uid)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "0b282150-0db3-407d-ac32-84500b262341\n" } - ], + ], "execution_count": 86 - }, + }, { - "source": "\n## 5. Deploy the model", - "cell_type": "markdown", + "source": "\n## 5. Deploy the model", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "### 5.1 Create deployment", - "cell_type": "markdown", + "source": "### 5.1 Create deployment", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "deployment = client.deployments.create(model_uid, 'CARS4U - Satisfaction Prediction Model Deployment')", - "cell_type": "code", - "metadata": {}, + "source": "deployment = client.deployments.create(model_uid, 'CARS4U - Satisfaction Prediction Model Deployment')", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '0b282150-0db3-407d-ac32-84500b262341' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_IN_PROGRESS..\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='726e333a-73a2-4720-b249-3be832903b4e'\n------------------------------------------------------------------------------------------------\n\n\n" } - ], + ], "execution_count": 87 - }, + }, { - "source": "client.deployments.list()", - "cell_type": "code", - "metadata": {}, + "source": "client.deployments.list()", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "------------------------------------ -------------------------------------------------- ------ -------------- ------------------------ -------------- ----------\nGUID NAME TYPE STATE CREATED FRAMEWORK ASSET TYPE\n726e333a-73a2-4720-b249-3be832903b4e CARS4U - Satisfaction Prediction Model Deployment online DEPLOY_SUCCESS 2018-07-24T09:49:11.282Z tensorflow-1.5 model\nb344d58c-590d-4a93-a007-23693947ce31 CARS4U - Business Area Prediction Model Deployment online DEPLOY_SUCCESS 2018-07-24T09:47:40.251Z mllib-2.1 model\n584a3b49-7b3c-4663-8b6f-6a433c041b1d CARS4U - Satisfaction Prediction Model Deployment online DEPLOY_SUCCESS 2018-07-24T09:46:21.374Z tensorflow-1.5 model\nf9cabb53-a6fd-426a-a678-6091566d2b3a CARS4U - Action Model Deployment online DEPLOY_SUCCESS 2018-07-23T18:59:52.145Z mllib-2.1 model\n------------------------------------ -------------------------------------------------- ------ -------------- ------------------------ -------------- ----------\n" } - ], + ], "execution_count": 88 - }, + }, { - "source": "### 5.2 Score the model", - "cell_type": "markdown", + "source": "### 5.2 Score the model", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Let's see if our deployment works.", - "cell_type": "markdown", + "source": "Let's see if our deployment works.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "scoring_endpoint = client.deployments.get_scoring_url(deployment)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "scoring_endpoint = client.deployments.get_scoring_url(deployment)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 89 - }, + }, { - "source": "print(scoring_endpoint)", - "cell_type": "code", - "metadata": {}, + "source": "print(scoring_endpoint)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/726e333a-73a2-4720-b249-3be832903b4e/online\n" } - ], + ], "execution_count": 90 - }, + }, { - "source": "index = 5\n\nscoring_data = X[index].tolist()\nprint(X_test[index])\nprint(Y_test[index])", - "cell_type": "code", - "metadata": {}, + "source": "index = 5\n\nscoring_data = X[index].tolist()\nprint(X_test[index])\nprint(Y_test[index])", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "[ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 40 107 131 19 1 22 77]\n0\n" } - ], + ], "execution_count": 324 - }, + }, { - "source": "scoring_payload = {'values': [scoring_data]}\nscores = client.deployments.score(scoring_endpoint, scoring_payload)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "scoring_payload = {'values': [scoring_data]}\nscores = client.deployments.score(scoring_endpoint, scoring_payload)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 325 - }, + }, { - "source": "print(scoring_payload)", - "cell_type": "code", - "metadata": {}, + "source": "print(scoring_payload)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'values': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 266, 138, 139, 267, 207, 115, 12, 8]]}\n" } - ], + ], "execution_count": 327 - }, + }, { - "source": "len(scoring_payload['values'][0])", - "cell_type": "code", - "metadata": {}, + "source": "len(scoring_payload['values'][0])", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "execute_result", + "output_type": "execute_result", "data": { "text/plain": "50" - }, - "execution_count": 330, + }, + "execution_count": 330, "metadata": {} } - ], + ], "execution_count": 330 - }, + }, { - "source": "print({'values': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 266, 138, 139, 267, 207, 115, 12, 8], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 2, 78, 5, 7, 13, 122, 3, 109, 51, 0, 0, 58, 15, 808, 31, 7, 23]]})", - "cell_type": "code", - "metadata": {}, + "source": "print({'values': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 266, 138, 139, 267, 207, 115, 12, 8], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 2, 78, 5, 7, 13, 122, 3, 109, 51, 0, 0, 58, 15, 808, 31, 7, 23]]})", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'values': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 266, 138, 139, 267, 207, 115, 12, 8], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 2, 78, 5, 7, 13, 122, 3, 109, 51, 0, 0, 58, 15, 808, 31, 7, 23]]}\n" } - ], + ], "execution_count": 329 - }, + }, { - "source": "Let's print scoring results.", - "cell_type": "markdown", + "source": "Let's print scoring results.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "print(str(scores))", - "cell_type": "code", - "metadata": {}, + "source": "print(str(scores))", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'fields': ['prediction', 'prediction_classes', 'probability'], 'values': [[[0.008548633195459843], [0], [0.008548633195459843]]]}\n" } - ], + ], "execution_count": 326 - }, + }, { - "source": "\n## 6. AI function", - "cell_type": "markdown", + "source": "\n## 6. AI function", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Let's define AI function that does data preprocessing and model scoring for us. As noticed above model expects numerical input, so the text comment needs to be preprocessed.", - "cell_type": "markdown", + "source": "Let's define AI function that does data preprocessing and model scoring for us. As noticed above model expects numerical input, so the text comment needs to be preprocessed.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "### 6.1 Definition", - "cell_type": "markdown", + "source": "### 6.1 Definition", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "Define some generic parameters our function will use to score the model.", - "cell_type": "markdown", + "source": "Define some generic parameters our function will use to score the model.", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "#### Parameters", - "cell_type": "markdown", + "source": "#### Parameters", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "ai_params = {\n 'scoring_endpoint': scoring_endpoint,\n 'wml_credentials': wml_credentials,\n 'word_index': tokenizer.word_index\n}", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "ai_params = {\n 'scoring_endpoint': scoring_endpoint,\n 'wml_credentials': wml_credentials,\n 'word_index': tokenizer.word_index\n}", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 262 - }, + }, { - "source": "#### Function", - "cell_type": "markdown", + "source": "#### Function", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "def score_generator(params=ai_params):\n\n def score(payload):\n import re\n from watson_machine_learning_client import WatsonMachineLearningAPIClient\n client = WatsonMachineLearningAPIClient(params['wml_credentials'])\n \n max_fatures = 500\n maxlen = 50\n\n preprocessed_records = []\n complain_data = payload['values']\n word_index = params['word_index']\n\n for data in complain_data:\n comment = data[0]\n cleanString = re.sub(r\"[!\\\"#$%&()*+,-./:;<=>?@[\\]^_`{|}~]\", \"\", comment)\n splitted_comment = cleanString.split()[:maxlen]\n hashed_tokens = []\n\n for token in splitted_comment:\n index = word_index.get(token, 0)\n if index < 501 and index > 0:\n hashed_tokens.append(index)\n\n hashed_tokens_size = len(hashed_tokens)\n padded_tokens = [0]*(maxlen-hashed_tokens_size) + hashed_tokens\n preprocessed_records.append(padded_tokens)\n\n scoring_payload = {'values': preprocessed_records}\n print(str(scoring_payload))\n \n return client.deployments.score(params['scoring_endpoint'], scoring_payload)\n \n \n return score", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "def score_generator(params=ai_params):\n\n def score(payload):\n import re\n from watson_machine_learning_client import WatsonMachineLearningAPIClient\n client = WatsonMachineLearningAPIClient(params['wml_credentials'])\n \n max_fatures = 500\n maxlen = 50\n\n preprocessed_records = []\n complain_data = payload['values']\n word_index = params['word_index']\n\n for data in complain_data:\n comment = data[0]\n cleanString = re.sub(r\"[!\\\"#$%&()*+,-./:;<=>?@[\\]^_`{|}~]\", \"\", comment)\n splitted_comment = cleanString.split()[:maxlen]\n hashed_tokens = []\n\n for token in splitted_comment:\n index = word_index.get(token, 0)\n if index < 501 and index > 0:\n hashed_tokens.append(index)\n\n hashed_tokens_size = len(hashed_tokens)\n padded_tokens = [0]*(maxlen-hashed_tokens_size) + hashed_tokens\n preprocessed_records.append(padded_tokens)\n\n scoring_payload = {'values': preprocessed_records}\n print(str(scoring_payload))\n \n return client.deployments.score(params['scoring_endpoint'], scoring_payload)\n \n \n return score", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 345 - }, + }, { - "source": "#### Test locally", - "cell_type": "markdown", + "source": "#### Test locally", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "sample_data = {\n 'fields': ['feedback'],\n 'values': [\n ['delayed shuttle, almost missed flight, bad customer service'],\n ['The car was great and they were able to provide all features I wanted with limited time they had.']\n ]\n}", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "sample_data = {\n 'fields': ['feedback'],\n 'values': [\n ['delayed shuttle, almost missed flight, bad customer service'],\n ['The car was great and they were able to provide all features I wanted with limited time they had.']\n ]\n}", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 346 - }, + }, { - "source": "score = score_generator()\nscore(sample_data)", - "cell_type": "code", - "metadata": {}, + "source": "score = score_generator()\nscore(sample_data)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'values': [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 266, 138, 139, 267, 207, 115, 12, 8], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 2, 78, 5, 7, 13, 122, 3, 109, 51, 58, 15, 31, 7, 23]]}\n" - }, + }, { - "output_type": "execute_result", + "output_type": "execute_result", "data": { "text/plain": "{'fields': ['prediction', 'prediction_classes', 'probability'],\n 'values': [[[0.008548641577363014], [0], [0.008548641577363014]],\n [[0.9832062125205994], [1], [0.9832062125205994]]]}" - }, - "execution_count": 347, + }, + "execution_count": 347, "metadata": {} } - ], + ], "execution_count": 347 - }, + }, { - "source": "**Note:** 0 - not satisfied. 1 - satisfied", - "cell_type": "markdown", + "source": "**Note:** 0 - not satisfied. 1 - satisfied", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "### 6.2 AI function storing", - "cell_type": "markdown", + "source": "### 6.2 AI function storing", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "runtime_meta = {\n client.runtime_specs.ConfigurationMetaNames.NAME: \"Runtime specification\",\n client.runtime_specs.ConfigurationMetaNames.PLATFORM: {\n \"name\": \"python\",\n \"version\": \"3.5\"\n }\n}", - "cell_type": "code", - "metadata": {}, - "outputs": [], - "execution_count": 348 - }, - { - "source": "runtime_details = client.runtime_specs.create(meta_props=runtime_meta)\nruntime_url = client.runtime_specs.get_url(runtime_details)\nprint(runtime_url)", - "cell_type": "code", - "metadata": {}, - "outputs": [ - { - "output_type": "stream", - "name": "stdout", - "text": "https://us-south.ml.cloud.ibm.com/v4/runtimes/9f6ab609-eada-4b56-a96f-fa36feaa00c4\n" - } - ], - "execution_count": 349 - }, - { - "source": "client.repository.FunctionMetaNames.show()", - "cell_type": "code", - "metadata": {}, + "source": "client.repository.FunctionMetaNames.show()", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "------------------ ---- -------- -------------\nMETA_PROP NAME TYPE REQUIRED DEFAULT_VALUE\nNAME str Y\nDESCRIPTION str N\nTYPE str N python\nRUNTIME_URL str N\nINPUT_DATA_SCHEMA dict N\nOUTPUT_DATA_SCHEMA dict N\nTAGS list N\n------------------ ---- -------- -------------\n" } - ], + ], "execution_count": 350 - }, + }, { - "source": "meta_data = {\n client.repository.FunctionMetaNames.NAME: 'CARS4U - Sentiment Prediction - AI Function',\n client.repository.FunctionMetaNames.RUNTIME_URL: runtime_url\n}\n\nfunction_details = client.repository.store_function(meta_props=meta_data, function=score_generator)", - "cell_type": "code", - "metadata": {}, + "source": "function_details = client.repository.store_function(score_generator, 'CARS4U - Sentiment Prediction - AI Function')", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "Recognized generator function.\n" } - ], + ], "execution_count": 351 - }, + }, { - "source": "client.repository.list_functions()", - "cell_type": "code", - "metadata": {}, + "source": "client.repository.list_functions()", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "------------------------------------ ---------------------------------------------------------- ------------------------ ------\nGUID NAME CREATED TYPE\n34574059-255d-48fe-aa5c-74adc1309068 CARS4U - Sentiment Prediction - AI Function 2018-07-24T14:49:49.064Z python\n2c4fdde6-353e-4746-9058-ae3a377b868c CARS4U - Business area and Action Prediction - AI Function 2018-07-24T13:48:30.139Z python\n0a69ab8c-0a92-4625-8350-972d09d692b9 CARS4U - Sentiment Prediction - AI Function 2018-07-24T13:41:43.298Z python\n86a6d7d3-daac-479b-9580-3accf27a2ab6 CARS4U - Sentiment Prediction - AI Function 2018-07-24T13:38:35.215Z python\n8c1e531b-27ef-40ac-a316-54268230356c CARS4U - Business area and Action Prediction - AI Function 2018-07-24T13:16:18.129Z python\nf3d4a4d4-0166-4f21-b389-3cbeae467617 CARS4U - Business area and Action Prediction - AI Function 2018-07-24T13:10:29.331Z python\nca8858e1-a87c-4440-99ff-74525f863473 CARS4U - Sentiment Prediction - AI Function 2018-07-24T12:05:01.074Z python\n1c362b5c-0824-4989-9892-e22fd202cbde CARS4U - Sentiment Prediction - AI Function 2018-07-24T11:50:58.352Z python\na8136209-dd99-456c-a7b2-bcb9f77ef84c CARS4U - Sentiment Prediction - AI Function 2018-07-24T11:47:59.677Z python\n------------------------------------ ---------------------------------------------------------- ------------------------ ------\n" } - ], + ], "execution_count": 352 - }, + }, { - "source": "### 6.3 AI function deployment", - "cell_type": "markdown", + "source": "### 6.3 AI function deployment", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "function_uid = client.repository.get_function_uid(function_details)\n\nfunction_deployment_details = client.deployments.create(asset_uid=function_uid, name='CARS4U - Sentiment Prediction - AI Function Deployment')", - "cell_type": "code", - "metadata": {}, + "source": "function_uid = client.repository.get_function_uid(function_details)\n\nfunction_deployment_details = client.deployments.create(function_uid, name='CARS4U - Sentiment Prediction - AI Function Deployment')", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '34574059-255d-48fe-aa5c-74adc1309068' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_IN_PROGRESS....\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='aeb667ed-5070-4c88-b12f-333e1325d1ad'\n------------------------------------------------------------------------------------------------\n\n\n" } - ], + ], "execution_count": 353 - }, + }, { - "source": "### Score AI function", - "cell_type": "markdown", + "source": "### Score AI function", + "cell_type": "markdown", "metadata": {} - }, + }, { - "source": "ai_function_scoring_endpoint = client.deployments.get_scoring_url(function_deployment_details)\n\nprint(ai_function_scoring_endpoint)", - "cell_type": "code", - "metadata": {}, + "source": "ai_function_scoring_endpoint = client.deployments.get_scoring_url(function_deployment_details)\n\nprint(ai_function_scoring_endpoint)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "https://us-south.ml.cloud.ibm.com/v3/wml_instances/000263d8-04e0-4060-ad69-fcfe40069018/deployments/aeb667ed-5070-4c88-b12f-333e1325d1ad/online\n" } - ], + ], "execution_count": 354 - }, + }, { - "source": "response = client.deployments.score(ai_function_scoring_endpoint, sample_data)", - "cell_type": "code", - "metadata": {}, - "outputs": [], + "source": "response = client.deployments.score(ai_function_scoring_endpoint, sample_data)", + "cell_type": "code", + "metadata": {}, + "outputs": [], "execution_count": 355 - }, + }, { - "source": "print(response)", - "cell_type": "code", - "metadata": {}, + "source": "print(response)", + "cell_type": "code", + "metadata": {}, "outputs": [ { - "output_type": "stream", - "name": "stdout", + "output_type": "stream", + "name": "stdout", "text": "{'fields': ['prediction', 'prediction_classes', 'probability'], 'values': [[[0.008548641577363014], [0], [0.008548641577363014]], [[0.9832062125205994], [1], [0.9832062125205994]]]}\n" } - ], + ], "execution_count": 356 - }, + }, { - "source": "---\n", - "cell_type": "markdown", + "source": "---\n", + "cell_type": "markdown", "metadata": {} } - ], + ], "metadata": { "kernelspec": { - "display_name": "Python 3.5", - "name": "python3", + "display_name": "Python 3.5", + "name": "python3", "language": "python" - }, + }, "language_info": { - "mimetype": "text/x-python", - "nbconvert_exporter": "python", - "version": "3.5.5", - "name": "python", - "pygments_lexer": "ipython3", - "file_extension": ".py", + "mimetype": "text/x-python", + "nbconvert_exporter": "python", + "version": "3.5.5", + "name": "python", + "pygments_lexer": "ipython3", + "file_extension": ".py", "codemirror_mode": { - "version": 3, + "version": 3, "name": "ipython" } } - }, + }, "nbformat": 4 -} \ No newline at end of file +}