Hardware integration

.gitignore

@@ -142,4 +142,31 @@ dmypy.json
 .pytype/
 
 # Cython debug symbols
-cython_debug/
+cython_debug/
+
+.vscode/
+
+temp.py
+temp2.py
+to_save.yaml
+to_load.yaml
+out.json
+dashapp.py
+dashapp-tut.py
+data.csv
+e1.json
+e1.yaml
+e1mongo.yaml
+e2.yaml
+e3.yaml
+e4.yaml
+e4mongo.yaml
+project_const.py
+pw.py
+.DS_Store
+blank.yaml
+pw.txt
+ximea_linux_sp_beta.tgz
+package
+firmware/
+to_implement/

README.md

@@ -10,12 +10,16 @@ Lab Automation is a modular framework for automating multiple devices or instrum
 4. [Usage](#usage)
     - [Automation](#automation)
     - [Optimization](#autonomous-process-optimization)
-5. [Further Details](#further-details)
+5. [Dash UI](#dash-ui)
+    - [UI Setup Instructions](#ui-setup-instructions)
+    - [Adding Recipes for the Recipe Builder](#adding-recipes-for-the-recipe-builder)
+    - [Next Steps](#next-steps)
+6. [Further Details](#further-details)
     - [Creating Device Modules](#creating-device-modules)
     - [Creating Command Modules](#creating-command-modules)
         - [Composite Commands](#composite-commands)
     - [Program Scheme](#program-scheme)
-6. [License](#license)
+7. [License](#license)
 
 ## Description
 Lab Automation was developed for the purpose of automating laboratory experiments to enable high-throughput data collection and process optimization. It can be used to automate experimental procedures that involve instruments from different vendors, with different communication protocols, and also home-built equipment. The automated procedures can then be tied into sequential model-based optimization algorithms (e.g. Bayesian optimization) to enable self-driving, autonomous lab experiments.
@@ -111,6 +115,96 @@ Example 6 emulates optimization of a material processing experiment. It uses a f
 
 You will need some more packages to run example 6 (listed above). Run example 6 as stated above from root.
 
+## Dash UI
+The entire functionality of the app can also be accessed through a user-friendly Dash app interface. This includes creating recipes, running them, granulated manual control, and new features involved in automated process optimization, such as parameter set sampling, the recipe builder, and the solution mapper.
+
+### UI Setup Instructions
+
+1. Clone this repository.
+```
+git clone https://github.com/moleculemaker/Lab_Automation
+```
+2. Enter the Lab Automation folder.
+```
+cd Lab_Automation
+```
+3. Check out the dash-ui branch.
+```
+git checkout dash-ui
+```
+4. Create a virtual environment.
+```
+python3 -m venv venv
+```
+5. Activate the virtual environment.
+
+Mac:
+```
+source venv/bin/activate
+```
+Windows:
+```
+venv\Scripts\activate
+```
+6. Install the requirements.
+```
+pip install -r requirements.txt
+```
+7. Add a .env file to the aamp_app folder and aamp_app/db/validation containing your MongoDB credentials. The files should be in the following format:
+```
+MONGO_URI=mongodb+srv://<username>:<connection-string>
+MONGO_DB_NAME=diaogroup
+```
+8. Enter the aamp_app folder.
+```
+cd aamp_app
+```
+9. When running the app for the first time with an empty database, the correct collections with validation schemas need to be created. Uncomment the following lines in app.py:
+```
+# solutions.init_collection()
+# devices.init_collection()
+# films.init_collection()
+# recipes.init_collection()
+```
+They can be commented again once the collections are created in the database.
+
+10. Run the following:
+```
+pip install --upgrade dotenv
+plotly_get_chrome
+```
+
+11. Run the app
+```
+python -m app
+```
+
+### Adding Recipes for the Recipe Builder
+Right now the recipe builder is formatted to work with one recipe template, and that is recipe_sample.py. A keyword search for this in the app.py file will reveal the place where it is being read in and saved as a constant string literal. To add a new recipe:
+1. Include it as a Python file in the recipes directory.
+2. Replace all literals for template variables where appropriate (concentration, printing gap, arm position, etc.). You can refer to recipe_sample.py for an example implementation of this. Adding additional template variables requires modification of the generate_recipes callback in app.py.
+3. Change the filename being read into app.py from recipe_sample.py to your new recipe's filename.
+4. Ensure that the generate_recipes callback runs without error with the new recipe.
+5. By default, all recipes are ignored. If you want git to track the new recipe, add !\<your-recipe>.py to the .gitignore file in the recipes folder.
+
+### Next Steps
+
+There are a number of pending tasks required for the Dash UI and automated optimization workflow to be fully operational:
+- Send experimental results to the Bayesian optimizer
+- Receive parameters from the optimizer in the recipe builder, either through database triggers or directly in the app
+- Validate custom inputs in the sampler
+- Add different sampling methods to the sampler (currently uses simple random sampling)
+- Allow a campaign to have more than one polymer
+- Attach polymer information to each parameter set rather than to campaign metadata
+- Validate the solution map JSON
+- Add a way to update or include more recipe templates
+- Get the Ximea Camera module working on Mac and Linux devices
+- Test running recipes sequentially in the recipe builder
+- Visualize parameter space exploration over the course of a campaign
+- Visualize optimal parameter ranges over the course of a campaign
+- Track campaign progress in the UI, along with logging and error tracking
+- Merge the dash-ui branch to master once a full campaign can be run through the UI (and most of the above objectives are complete)
+
 ## Further Details
 ### Creating Device Modules
 Please see the devices folder for examples of implementing device modules

aamp_app/Image_Processing/Image_processing/__init__.py

@@ -0,0 +1,18 @@
+"""Custom image processing pipeline for coverage analysis."""
+
+from .config import UniformityConfig
+from .coverage import analyze_film_coverage
+from .pipeline import analyze_image
+from .scoring import compute_uniformity_score, _match_reference_for
+from .uvvis import analyze_uvvis_data, find_uvvis_file
+
+__all__ = [
+    "UniformityConfig",
+    "analyze_film_coverage",
+    "analyze_image",
+    "compute_uniformity_score",
+    "_match_reference_for",
+    "analyze_uvvis_data",
+    "find_uvvis_file",
+]
+

aamp_app/Image_Processing/Image_processing/config.py

@@ -0,0 +1,48 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Optional
+from bson import ObjectId
+
+
+@dataclass
+class UniformityConfig:
+    """Lightweight configuration object.
+
+    Maintains compatibility with the existing imageproc.uniformity.config interface
+    by keeping the same field names.
+    """
+
+    campaign_id: Optional[ObjectId] = None
+    use_reference: bool = True
+    reference_name: str = "background_normal_0deg"
+
+    roi_x: int = 0
+    roi_y: int = 0
+    roi_width: int = 100
+    roi_height: int = 100
+
+    uvvis_abs_threshold: float = 0.1
+    uvvis_wavelength_min: float = 300.0
+    uvvis_wavelength_max: float = 800.0
+    uvvis_required: bool = False
+
+    coverage_threshold: float = 0.1
+    coverage_smoothing: float = 1.5
+    # Sigma multiplier (K value) used in HSV S channel threshold calculation. Adjust default value here if needed.
+    coverage_k: float = 3.0
+    coverage_min_std: float = 10.0
+
+    # Uniformity calculation sensitivity parameters (for Reference-based Absolute Scoring)
+    uniformity_std_sensitivity: float = 15.0  # K1: Sensitivity to Std difference
+    uniformity_ent_sensitivity: float = 2.0   # K2: Sensitivity to Entropy difference
+
+    compute_learning_features: bool = False
+
+    extra: dict = field(default_factory=dict)
+
+    @property
+    def roi_bbox(self) -> tuple[int, int, int, int]:
+        return (self.roi_x, self.roi_y, self.roi_width, self.roi_height)
+

aamp_app/Image_Processing/Image_processing/coverage.py

@@ -0,0 +1,149 @@
+"""HSV-based film coverage analysis module.
+
+Instead of the traditional reference subtraction approach,
+this module calculates coverage by combining dynamic thresholds
+based on reference image S-channel statistics with sample image HSV information.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Tuple, Dict
+
+import cv2
+import numpy as np
+from skimage.measure import shannon_entropy
+
+
+@dataclass
+class ROIParams:
+    x: int
+    y: int
+    width: int
+    height: int
+
+
+def _clamp_roi(image: np.ndarray, roi: ROIParams) -> ROIParams:
+    """Clamp ROI to ensure it stays within image boundaries."""
+    h, w = image.shape[:2]
+    x = max(0, min(roi.x, w - 1))
+    y = max(0, min(roi.y, h - 1))
+    width = max(1, min(roi.width, w - x))
+    height = max(1, min(roi.height, h - y))
+    return ROIParams(x, y, width, height)
+
+
+def _extract_roi(image: np.ndarray, roi: ROIParams) -> np.ndarray:
+    """Extract the image region corresponding to the clamped ROI."""
+    roi = _clamp_roi(image, roi)
+    return image[roi.y:roi.y + roi.height, roi.x:roi.x + roi.width]
+
+
+def _ensure_same_shape(sample_roi: np.ndarray, reference_roi: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
+    """Resize reference ROI if sample and reference ROI sizes differ."""
+    if sample_roi.shape[:2] == reference_roi.shape[:2]:
+        return sample_roi, reference_roi
+    resized_ref = cv2.resize(reference_roi, (sample_roi.shape[1], sample_roi.shape[0]))
+    return sample_roi, resized_ref
+
+
+def _apply_mask_smoothing(mask: np.ndarray, sigma: float) -> np.ndarray:
+    """Apply Gaussian blur to reduce noise."""
+    if sigma <= 0:
+        return mask
+    blurred = cv2.GaussianBlur(mask, (0, 0), sigma)
+    return (blurred > 127).astype(np.uint8) * 255
+
+
+def analyze_film_coverage(
+    image: np.ndarray,
+    reference: np.ndarray,
+    roi_x: int,
+    roi_y: int,
+    roi_width: int,
+    roi_height: int,
+    smoothing: float = 1.5,
+    k: float = 3.0,
+    min_std: float = 10.0,
+) -> Tuple[float, np.ndarray, np.ndarray, np.ndarray, Dict[str, float]]:
+    """HSV-based film coverage analysis.
+
+    Args:
+        image: Sample BGR image
+        reference: Reference BGR image
+        roi_x, roi_y, roi_width, roi_height: ROI coordinates
+        smoothing: Gaussian sigma for mask post-processing
+        k: Standard deviation multiplier for dynamic threshold
+        min_std: Minimum value to use when standard deviation is 0
+
+    Returns:
+        (coverage_pct, sample_roi_bgr, reference_roi_bgr, final_mask, debug_info)
+    """
+    roi = ROIParams(roi_x, roi_y, roi_width, roi_height)
+    sample_roi = _extract_roi(image, roi)
+    reference_roi = _extract_roi(reference, roi)
+    sample_roi, reference_roi = _ensure_same_shape(sample_roi, reference_roi)
+
+    # Convert to HSV
+    sample_hsv = cv2.cvtColor(sample_roi, cv2.COLOR_BGR2HSV)
+    reference_hsv = cv2.cvtColor(reference_roi, cv2.COLOR_BGR2HSV)
+
+    # Reference S channel statistics
+    ref_s = reference_hsv[:, :, 1].astype(np.float32)
+    ref_s_mean = float(np.mean(ref_s))
+    ref_s_std = float(np.std(ref_s))
+    effective_std = ref_s_std if ref_s_std > 0 else float(min_std)
+    saturation_threshold = float(np.clip(ref_s_mean + k * effective_std, 0, 255))
+
+    # Condition A: S channel dynamic threshold
+    sample_s = sample_hsv[:, :, 1].astype(np.float32)
+    condition_a = sample_s > saturation_threshold
+
+    # Condition B: V channel Otsu (dark regions)
+    sample_v = sample_hsv[:, :, 2].astype(np.uint8)
+    otsu_threshold, otsu_mask = cv2.threshold(
+        sample_v, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU
+    )
+    condition_b = otsu_mask > 0
+
+    # Final mask
+    final_mask = np.logical_or(condition_a, condition_b).astype(np.uint8) * 255
+    final_mask = _apply_mask_smoothing(final_mask, smoothing)
+
+    total_pixels = final_mask.size
+    film_pixels = int(np.sum(final_mask > 0))
+    coverage_percentage = float(film_pixels) / float(total_pixels) if total_pixels else 0.0
+
+    # Reference statistics (for Uniformity calculation)
+    ref_s_uint8 = reference_hsv[:, :, 1].astype(np.uint8)
+    ref_entropy = float(shannon_entropy(ref_s_uint8))
+
+    # Sample statistics (for Uniformity calculation)
+    sample_s_uint8 = sample_hsv[:, :, 1].astype(np.uint8)
+    sample_std = float(np.std(sample_s_uint8.astype(np.float32)))
+    sample_entropy = float(shannon_entropy(sample_s_uint8))
+
+    debug_info: Dict[str, float] = {
+        "coverage_percentage": coverage_percentage,
+        "roi_width": float(sample_roi.shape[1]),
+        "roi_height": float(sample_roi.shape[0]),
+        "ref_s_mean": ref_s_mean,
+        "ref_s_std": ref_s_std,
+        "effective_std": effective_std,
+        "saturation_threshold": saturation_threshold,
+        "k_value": k,
+        "min_std_floor": min_std,
+        "otsu_threshold": float(otsu_threshold),
+        "condition_a_pixels": int(np.sum(condition_a)),
+        "condition_b_pixels": int(np.sum(condition_b)),
+        "film_pixels": film_pixels,
+        "total_pixels": int(total_pixels),
+        # Reference statistics (for Uniformity calculation)
+        "ref_std": ref_s_std,
+        "ref_entropy": ref_entropy,
+        "sample_std": sample_std,
+        "sample_entropy": sample_entropy,
+    }
+
+    return coverage_percentage, sample_roi, reference_roi, final_mask, debug_info
+