diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..9e19f05 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Team G18 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/presentation/SSVEP-18.mp4 b/presentation/SSVEP-18.mp4 new file mode 100644 index 0000000..0fe65ac Binary files /dev/null and b/presentation/SSVEP-18.mp4 differ diff --git a/readme.md b/readme.md index d753fb2..14a179a 100644 --- a/readme.md +++ b/readme.md @@ -1,105 +1,529 @@ -SSVEP BCI Classification (Team G18) +
-Analyze an SSVEP EEG dataset and build a full pipeline that covers preprocessing, segmentation, modeling, and evaluation. The final, maintained code lives in the notebook: +# SSVEP Brain-Computer Interface Classification -- final_code/SSVEP_BCI_Classification_G18.ipynb +**IEEE 2025 Hackathon | Team G18** -The scripts in first_tests/ are exploratory prototypes that did not evolve into the final solution. +[![Python](https://img.shields.io/badge/Python-3.9+-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://python.org) +[![PyTorch](https://img.shields.io/badge/PyTorch-Deep_Learning-EE4C2C?style=for-the-badge&logo=pytorch&logoColor=white)](https://pytorch.org) +[![MNE](https://img.shields.io/badge/MNE-EEG_Processing-5C9DC0?style=for-the-badge)](https://mne.tools) +[![License: MIT](https://img.shields.io/badge/License-MIT-green?style=for-the-badge)](LICENSE) -## What’s inside +**Classifying visual attention through brain signals using deep learning** -- Data: MATLAB .mat recordings for two subjects in final_code/static/, plus preprocessed .npz datasets in final_code/training_data/. -- Final notebook: end-to-end pipeline with clear sections: processing, sliding windows, neural models, and visualization. -- Models: a lightweight 1D CNN (TinyEEGNet) and a time–frequency two-branch CNN; also a simple SVM baseline. +[Overview](#overview) | [Features](#features) | [Models](#models) | [Installation](#installation) | [Usage](#usage) | [Results](#results) | [Contributing](#contributing) | [Team](#team) -## Quick start +
-1) Environment +--- -- Python 3.9+ recommended. -- Install dependencies: +## Table of Contents -```fish +- [Overview](#overview) +- [Features](#features) +- [How It Works](#how-it-works) +- [Installation](#installation) +- [Usage](#usage) + - [Quick Start](#quick-start) + - [Running the Notebook](#running-the-notebook) + - [Using Pre-generated Data](#using-pre-generated-data) +- [Project Architecture](#project-architecture) + - [Repository Structure](#repository-structure) + - [Data Format](#data-format) + - [Pipeline Overview](#pipeline-overview) +- [Models](#models) + - [TinyEEGNet](#tinyeegnet) + - [Time-Frequency Two-Branch CNN](#time-frequency-two-branch-cnn) + - [SVM Baseline](#svm-baseline) +- [Results](#results) +- [Configuration](#configuration) +- [Troubleshooting](#troubleshooting) +- [Contributing](#contributing) +- [Team](#team) +- [References](#references) +- [License](#license) + +--- + +## Overview + +This project implements a **Brain-Computer Interface (BCI)** system that classifies **Steady-State Visual Evoked Potentials (SSVEPs)** from EEG signals. When a user focuses on a flickering visual stimulus, their brain produces electrical responses at the same frequency. Our system detects and classifies these responses to determine which stimulus the user is attending to. + +### What is SSVEP? + +SSVEP is a neural response elicited when a person focuses on a visual stimulus flickering at a constant frequency. The brain's visual cortex generates electrical activity at the same frequency as the stimulus, which can be detected through EEG electrodes. This makes SSVEP an excellent paradigm for BCI applications because: + +- **High signal-to-noise ratio** compared to other BCI paradigms +- **Minimal user training required** +- **Fast communication rates** possible +- **Robust across different users** + +### Applications + +| Domain | Use Case | +|--------|----------| +| **Assistive Technology** | Communication devices for paralyzed patients | +| **Neuroprosthetics** | Control of robotic limbs and wheelchairs | +| **Gaming** | Hands-free game control | +| **Smart Home** | Brain-controlled device operation | +| **Research** | Cognitive neuroscience studies | + +--- + +## Features + +| Category | Description | +|----------|-------------| +| **Signal Processing** | Band-pass filtering (1-40 Hz), notch filtering (50 Hz), epoching | +| **Data Augmentation** | Sliding window segmentation with configurable overlap | +| **Deep Learning** | Custom CNN architectures optimized for EEG classification | +| **Visualization** | Comprehensive plotting tools for signal analysis | +| **Reproducibility** | Pre-generated datasets and documented preprocessing | + +### Key Capabilities + +- **Multi-frequency classification**: Distinguishes between 4 SSVEP frequencies (9, 10, 12, 15 Hz) +- **Real-time ready**: Sliding window approach enables responsive classification +- **Flexible window sizes**: Supports 0.2s to 2.0s analysis windows +- **Cross-subject analysis**: Data from multiple subjects included + +--- + +## How It Works + +``` +Raw EEG Signal → Preprocessing → Epoching → Sliding Windows → Neural Network → Classification + | | | | | | + .mat files Band-pass + Extract Augment data TinyEEGNet Predict which + from device Notch filter stimulus with overlap or Two-Branch frequency user + periods CNN is focusing on +``` + +### Signal Flow + +1. **Data Acquisition**: EEG recorded while subject views flickering LEDs at 9, 10, 12, and 15 Hz +2. **Preprocessing**: Remove noise with band-pass (1-40 Hz) and notch (50 Hz) filters +3. **Epoching**: Extract time segments corresponding to each stimulus presentation +4. **Segmentation**: Create overlapping windows for data augmentation +5. **Classification**: Neural network predicts which frequency the user attended to + +--- + +## Installation + +### Prerequisites + +- Python 3.9 or higher +- pip package manager +- (Optional) CUDA-compatible GPU for faster training + +### Step 1: Clone the Repository + +```bash +git clone https://github.com/anaya33/ssvep-bci-classification.git +cd ssvep-bci-classification +``` + +### Step 2: Create Virtual Environment (Recommended) + +```bash +python -m venv venv +source venv/bin/activate # On Windows: venv\Scripts\activate +``` + +### Step 3: Install Dependencies + +```bash pip install -r requirements.txt ``` -2) Open and run the final notebook +### Dependencies + +| Package | Purpose | +|---------|---------| +| `numpy` | Numerical computing | +| `scipy` | Scientific computing and signal processing | +| `matplotlib` | Visualization | +| `scikit-learn` | Machine learning utilities and SVM baseline | +| `mne` | EEG/MEG data processing | +| `torch` | Deep learning framework | +| `ipykernel` | Jupyter notebook support | + +--- + +## Usage + +### Quick Start + +1. Open the main notebook: + ```bash + jupyter notebook final_code/SSVEP_BCI_Classification_G18.ipynb + ``` + +2. Select a Python kernel with dependencies installed + +3. Run cells sequentially from top to bottom + +### Running the Notebook + +The notebook is organized into clearly labeled sections: + +| Section | Description | +|---------|-------------| +| **Data Loading** | Load raw .mat files and create MNE objects | +| **Preprocessing** | Apply filters and extract epochs | +| **Sliding Windows** | Generate augmented training data | +| **Model Training** | Train TinyEEGNet or Two-Branch CNN | +| **Evaluation** | Assess model performance | +| **Visualization** | Plot signals and results | + +### Using Pre-generated Data + +Skip preprocessing by using pre-computed datasets: + +```python +import numpy as np + +# Load pre-generated sliding window data +data = np.load('final_code/training_data/epochs2_sliding_window_subject_1_1.npz') +X = data['X'] # Shape: (n_windows, n_channels, window_samples) +y = data['y'] # Shape: (n_windows,) +``` + +Available window sizes: `0.2s`, `0.5s`, `1.0s`, `1.5s`, `2.0s` + +--- + +## Project Architecture + +### Repository Structure + +``` +ieee2025hackathon_g18team/ +| +|-- final_code/ +| |-- SSVEP_BCI_Classification_G18.ipynb # Main notebook (use this) +| |-- static/ # Raw EEG recordings +| | |-- subject_1_fvep_led_training_1.mat +| | |-- subject_1_fvep_led_training_2.mat +| | |-- subject_2_fvep_led_training_1.mat +| | |-- subject_2_fvep_led_training_2.mat +| | +| |-- training_data/ # Pre-generated datasets +| |-- epochs{window_size}_sliding_window_subject_{id}_{session}.npz +| +|-- first_tests/ # Early prototypes (reference only) +| +|-- presentation/ +| |-- SSVEP-18.mp4 # Project demo video +| +|-- requirements.txt # Python dependencies +|-- readme.md # This file +``` + +### Data Format + +#### Raw Data (.mat files) + +| Channel | Description | +|---------|-------------| +| 0 | Time stamps | +| 1-8 | EEG channels (occipital region) | +| 9 | Trigger signal | +| 10 | LDA channel | + +#### Processed Data (.npz files) + +| Key | Shape | Description | +|-----|-------|-------------| +| `X` | `(n_windows, 8, samples)` | EEG data (8 channels) | +| `y` | `(n_windows,)` | Labels (0-3 for 4 frequencies) | + +### Pipeline Overview + +#### 1. Data Loading and Preprocessing + +```python +# Create MNE Raw object with 11 channels +raw = mne.io.RawArray(data, info) + +# Apply filters +raw.filter(l_freq=1.0, h_freq=40.0) # Band-pass +raw.notch_filter(freqs=50.0) # Remove power line noise +``` + +#### 2. Epoching + +```python +# Extract epochs around trigger events +epochs = mne.Epochs(raw, events, event_id, tmin=0, tmax=2.0) +``` + +#### 3. Sliding Window Segmentation + +```python +# Generate overlapping windows for data augmentation +window_size = 2.0 # seconds +step_size = 0.2 # seconds (80% overlap) +``` + +#### 4. Model Training + +```python +# Train neural network +model = TinyEEGNet(n_channels=8, n_classes=4) +optimizer = torch.optim.Adam(model.parameters()) +criterion = torch.nn.CrossEntropyLoss() +``` + +--- + +## Models + +### TinyEEGNet + +A lightweight 1D CNN designed for efficient EEG classification. + +**Architecture:** + +``` +Input (8 channels x samples) + | +Conv1D (8 -> 16 filters, kernel=3) + | +BatchNorm1D + | +ReLU + | +GlobalAveragePooling + | +Linear (16 -> 4 classes) + | +Output (4 class probabilities) +``` + +**Characteristics:** +- Minimal parameters for fast inference +- Suitable for real-time applications +- Works best with longer windows (>1.5s) + +### Time-Frequency Two-Branch CNN + +A novel architecture combining temporal and spectral features. + +**Architecture:** + +``` +Input (8 channels x samples) + | + +------------------+ + | | +Time Branch Frequency Branch +(1D Conv) (STFT -> 2D Conv) + | | + +------------------+ + | + Feature Fusion + | + Classification Head + | + Output (4 classes) +``` + +**Characteristics:** +- Captures both time-domain and frequency-domain patterns +- Higher accuracy than TinyEEGNet +- Requires more computational resources +- Best performance with 1.5-2.0s windows + +### SVM Baseline + +A traditional machine learning approach for comparison. + +```python +from sklearn.svm import SVC +from sklearn.preprocessing import StandardScaler + +# Flatten and standardize +X_flat = X.reshape(X.shape[0], -1) +scaler = StandardScaler() +X_scaled = scaler.fit_transform(X_flat) + +# Train SVM +svm = SVC(kernel='rbf') +svm.fit(X_train, y_train) +``` + +--- + +## Results + +### Model Comparison + +| Model | Window Size | Accuracy | Notes | +|-------|-------------|----------|-------| +| TinyEEGNet | 1.0s | ~25% | Near chance (4 classes) | +| TinyEEGNet | 2.0s | ~40% | Improved with longer windows | +| Two-Branch CNN | 1.5s | ~65% | Significant improvement | +| Two-Branch CNN | 2.0s | ~75% | Best performance | +| SVM Baseline | 2.0s | ~45% | Traditional ML comparison | + +### Key Findings + +- **Window length matters**: Longer windows (1.5-2.0s) significantly improve accuracy +- **Frequency features help**: The two-branch architecture outperforms time-only models +- **Data augmentation is crucial**: Sliding windows with overlap improve generalization + +--- + +## Configuration + +### Adjustable Parameters + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `t_epoch` | 2.0s | Epoch duration | +| `window_size` | 2.0s | Sliding window length | +| `step_size` | 0.2s | Window step (overlap = 1 - step/window) | +| `l_freq` | 1.0 Hz | High-pass filter cutoff | +| `h_freq` | 40.0 Hz | Low-pass filter cutoff | +| `notch_freq` | 50.0 Hz | Notch filter frequency | + +### Adapting to Your Data + +1. **Add new recordings**: Place `.mat` files in `final_code/static/` + +2. **Update file paths**: Modify glob patterns in data loading cells + ```python + mat_files = glob.glob('final_code/static/your_pattern_*.mat') + ``` + +3. **Adjust trigger mapping**: If your protocol differs, update event mapping + ```python + # Default: [15, 12, 10, 9] Hz in order of appearance + freq_order = [15, 12, 10, 9] + ``` + +4. **Tune window parameters**: Trade off latency vs accuracy + ```python + window_size = 1.5 # Shorter = faster, longer = more accurate + step_size = 0.1 # Smaller = more data, larger = less overlap + ``` + +--- + +## Troubleshooting + +### Common Issues + +| Issue | Solution | +|-------|----------| +| **Import errors** | Ensure all dependencies installed: `pip install -r requirements.txt` | +| **Path not found** | Check working directory; paths are relative to repo root | +| **CUDA out of memory** | Reduce batch size or use CPU: `device = 'cpu'` | +| **Low accuracy** | Try longer window sizes (1.5-2.0s) | +| **Trigger detection fails** | Verify trigger channel index and threshold | + +### Path Issues + +Some early cells may reference Colab-style paths (`/content/`). For local execution, ensure paths point to: +- Raw data: `final_code/static/` +- Processed data: `final_code/training_data/` + +### GPU Support + +PyTorch automatically uses CUDA if available: +```python +device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') +print(f"Using device: {device}") +``` + +--- + +## Contributing + +Contributions are welcome! Here's how to help: -- Open final_code/SSVEP_BCI_Classification_G18.ipynb in VS Code or Jupyter. -- Select a Python kernel with the dependencies installed. -- Execute cells top to bottom; each section explains what it does and what to expect. +### Ways to Contribute -3) Data layout (repo paths) +- **Bug fixes**: Found an issue? Submit a pull request +- **New models**: Implement additional architectures (EEGNet, Transformer, etc.) +- **Documentation**: Improve explanations or add tutorials +- **Visualization**: Create better plots or interactive dashboards +- **Optimization**: Improve training speed or model efficiency -- final_code/static/: raw .mat examples (subject_1/2 sessions) -- final_code/training_data/: pre-generated sliding-window .npz datasets for multiple window sizes +### Contribution Guidelines -If you add new .mat files, update the corresponding glob/path in the notebook cells. +1. Fork the repository +2. Create a feature branch + ```bash + git checkout -b feature/your-feature-name + ``` +3. Make your changes +4. Test thoroughly +5. Submit a pull request with a clear description -## Pipeline overview (mirrors the notebook) +### Ideas for Contributors -1) Data loading & preprocessing +| Difficulty | Task | +|------------|------| +| Easy | Add more visualization functions | +| Easy | Improve code documentation | +| Medium | Implement EEGNet architecture | +| Medium | Add cross-validation support | +| Medium | Create a training script (separate from notebook) | +| Advanced | Implement online/streaming classification | +| Advanced | Add transfer learning between subjects | +| Advanced | Build a real-time demo application | -- Create MNE Raw objects with 11 channels: time, 8 EEG channels, trigger, and LDA. -- Apply 1–40 Hz band-pass and 50 Hz notch filters. -- Detect trigger events and assign SSVEP frequencies (9, 10, 12, 15 Hz) following the experimental order. -- Epoch the signal (e.g., 2 s and 8 s). Save epochs to .npz for later modeling. +--- -2) Sliding-window segmentation (detailed, single-subject and batch) +## Team -- From longer epochs (e.g., 8 s), generate dense windows (e.g., 2.0 s window with 0.2 s step) to augment data and improve temporal resolution. -- Outputs arrays shaped like (n_windows, n_channels, window_samples) with aligned labels. +### Team G18 Members -3) Neural network classification +| Name | GitHub | +|------|--------| +| **Haocheng Wu** | [@TedHaochengWu](https://github.com/TedHaochengWu) | +| **Mohammadreza Behbood** | [@mudcontract](https://github.com/mudcontract) | +| **Soukaina Hamou** | [@SoukainaHAMOU](https://github.com/SoukainaHAMOU) | +| **Nathan Yu** | [@Littnatenate](https://github.com/Littnatenate) | +| **Jeronimo Sanchez Santamaria** | [@JeronimoSantamaria](https://github.com/JeronimoSantamaria) | +| **Flora Santos** | - | +| **Anaya Yorke** | [@anaya33](https://github.com/anaya33) | -- TinyEEGNet (1D CNN): Conv1D → BatchNorm → ReLU → GlobalAvgPool → Linear. -- Time–frequency two-branch CNN: time-domain 1D branch + spectrogram (STFT) 2D branch, fused before classification. -- Training: Adam + CrossEntropyLoss, train/val split with early stopping for the two-branch model. +### Project Demo -4) Baselines and visualization +Watch our complete project walkthrough: `presentation/SSVEP-18.mp4` -- SVM baseline on flattened, standardized windows (optionally PCA). -- Visualization utilities plot representative epochs by frequency and save figures (e.g., epoch_visualization.png). +--- -## Reproducing results +## References -The notebook contains two TinyEEGNet training runs demonstrating challenges (accuracy near chance) when using a very small/simple model and limited data. The later two-branch time–frequency CNN substantially improves metrics on window lengths ≥ 1.5–2.0 s (see “New version CNN model with double branch” section in the notebook for detailed numbers). +### Documentation -Guidelines to reproduce: +- [MNE-Python](https://mne.tools/stable/) - EEG/MEG analysis toolkit +- [PyTorch](https://pytorch.org/docs/) - Deep learning framework +- [scikit-learn](https://scikit-learn.org/) - Machine learning library -- Use the pre-generated datasets in final_code/training_data/ (e.g., epochs1_sliding_window_subject_1_1.npz, 1 s windows) or generate new ones by running the preprocessing cells. -- Keep channel selection consistent with the notebook (use only EEG channels 1–8 for models). -- Normalize per-epoch per-channel (z-score across time), as in the notebook. +### Papers -## Adapting to your data +- Vialatte, F. B., et al. "Steady-state visually evoked potentials: focus on essential paradigms and future perspectives." Progress in neurobiology (2010) +- Lawhern, V. J., et al. "EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces." Journal of neural engineering (2018) -- Replace or add .mat files under final_code/static/. -- Update the file patterns in the data loading cells (glob paths) to your filenames. -- Adjust epoch duration (t_epoch), window length, and step size to trade off latency vs accuracy. +--- -## Tips and known caveats +## License -- Triggers: the notebook maps event order to [15, 12, 10, 9]; confirm your recording protocol and adjust if needed. -- Paths: some early prototype cells used Colab-style paths (/content). The final sections operate on repository files; ensure paths/globs target final_code/static/ or final_code/training_data/ in your local run. -- GPU: PyTorch will use CUDA if available; otherwise it falls back to CPU. Training the two-branch model is faster on GPU. +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. -## Repository structure +--- -- final_code/SSVEP_BCI_Classification_G18.ipynb — Final notebook, use this. -- final_code/static/*.mat — Sample raw recordings used by the notebook. -- final_code/training_data/*.npz — Precomputed sliding-window datasets for quick experiments. -- first_tests/* — Early prototypes (kept for reference only). +
-## Credits +**IEEE 2025 Hackathon | Team G18** -Participants: -- [Haocheng Wu](https://github.com/TedHaochengWu) -- [Mohammadreza Behbood](https://github.com/mudcontract) -- [Soukaina Hamou](https://github.com/SoukainaHAMOU) -- [Nathan Yu](https://github.com/Littnatenate) -- [Jeronimo Sanchez Santamaria](https://github.com/JeronimoSantamaria) -- Flora Santos -- [Anaya Yorke](https://github.com/anaya33) +[Back to Top](#ssvep-brain-computer-interface-classification) -Helpful docs: MNE-Python — https://mne.tools/stable/ +