💧 Maji Ndogo Water Security & Fraud Identification System

An applied machine learning system and operational audit framework developed for the Ministry of Water in Maji Ndogo. The system detects statistical anomalies, potential data falsification, and corruption in field employee survey logs to prioritize manual investigative reviews.

---

📌 Project Overview

Maji Ndogo faces a severe water security crisis affecting over 27 million residents who rely on contaminated wells, broken tap infrastructure, or unprotected rivers. To allocate infrastructure repair budgets effectively, field employees submit water quality and source inspection surveys.

However, manual field data collection is vulnerable to data falsification (e.g., corrupt inspectors submitting fake clean scores for contaminated wells to avoid repair work or pocket funds).

This project provides an automated Anomaly Detection & Fraud Identification Engine to:

1. Identify suspicious survey entries using unsupervised and supervised ML.
2. Rank field employees by statistical risk profiles for internal audit.
3. Expose interpretable risk factors for individual record reviews.
4. Serve real-time predictions via a FastAPI REST API and an interactive Streamlit Dashboard.

---

🎯 Objectives & Value

• Leakage-Free Feature Engineering: Computes chronological expanding historical aggregates to prevent temporal or target data leakage.
• Multi-Model Anomaly Detection: Combines Isolation Forest, One-Class SVM, and Logistic Regression with validation-optimized ensemble weighting.
• Operational Risk Scoring: Converts raw anomaly scores into operational risk categories (Requires Investigation (High Risk), Medium Risk, Low Risk).
• Audit Prioritization: Ranks high-risk field employees to maximize internal investigation efficiency.
• Production Readiness: Fully containerized with Docker, covered by automated unit tests, CI/CD, REST API, and interactive dashboard.

---

🔬 Machine Learning Approach

                    ┌──────────────────────────────────────────────┐
                    │ Raw Relational Survey & Audit Logs           │
                    └──────────────────────┬───────────────────────┘
                                           │
                                           ▼
                    ┌──────────────────────────────────────────────┐
                    │ Chronological Expanding Feature Extraction   │
                    │ (No Target or Temporal Data Leakage)         │
                    └──────────────────────┬───────────────────────┘
                                           │
                                           ▼
                    ┌──────────────────────────────────────────────┐
                    │ Chronological Train (60%) / Val (20%) / Test  │
                    └──────────────────────┬───────────────────────┘
                                           │
          ┌────────────────────────────────┼────────────────────────────────┐
          │                                │                                │
          ▼                                ▼                                ▼
┌──────────────────┐             ┌──────────────────┐             ┌──────────────────┐
│ Isolation Forest │             │  One-Class SVM   │             │LogisticReg Pipeline│
│ (Unsupervised)   │             │  (Pipeline)      │             │  (Pipeline)      │
└────────┬─────────┘             └────────┬─────────┘             └────────┬─────────┘
         │                                │                                │
         └────────────────────────────────┼────────────────────────────────┘
                                          │
                                          ▼
                    ┌──────────────────────────────────────────────┐
                    │ Validation Set Optimal Ensemble & Threshold   │
                    └──────────────────────┬───────────────────────┘
                                           │
                                           ▼
                    ┌──────────────────────────────────────────────┐
                    │ Final Untouched Test Set Evaluation (20%)    │
                    └──────────────────────────────────────────────┘

1. Leakage Prevention & Feature Engineering

All employee aggregate metrics are computed strictly using chronological expanding windows shifted by 1 observation (.shift(1)). No observation ever contributes to its own aggregate statistics or uses future records.

• score_delta: $| \text{subjective_quality_score} - \text{audited_quality_score} |$
• raw_score_diff: $\text{subjective_quality_score} - \text{audited_quality_score}$
• survey_completion_time_min: Raw survey duration in minutes.
• time_per_visit: $\text{survey_completion_time_min} / (\text{visit_count} + 1e-5)$
• location_anomaly_score: Distance outlier metric relative to assigned geographic cluster.
• emp_hist_avg_score_delta: Expanding mean score discrepancy for the employee preceding current record.
• emp_hist_fast_completion_ratio: Expanding fraction of surveys completed in $< 5.0$ minutes.
• emp_prior_survey_count: Count of preceding surveys completed by the employee.

2. Preprocessing & Models

• StandardScaler: Scalers are fitted strictly inside scikit-learn Pipelines on training data to prevent preprocessing leakage.
• Rule-Based Baseline: Heuristic score thresholding for benchmark comparison.
• Isolation Forest: Tree-based outlier isolation (n_estimators=100, contamination=0.15).
• One-Class SVM: Kernel boundary anomaly estimator (kernel='rbf', gamma=0.01, nu=0.15).
• Logistic Regression: Penalized classifier with balanced class weighting (C=1.0, solver='lbfgs').
• Ensemble Risk Scoring: Normalizes individual model scores and blends them using validation-set PR-AUC optimal weights.

---

📊 Measured Model Performance (Untouched Test Set)

⚠️ DISCLAIMER: The ground-truth fraud labels (is_fraud) used for model training and evaluation are synthetically generated for methodological demonstration because verified real-world fraud labels are not present in the historical survey logs. Results demonstrate algorithmic methodology rather than confirmed real-world misconduct.

The final evaluation was conducted on an untouched 20% chronological test set after tuning hyperparameters and decision thresholds on the validation set:

Model / Algorithm	Precision	Recall	F1-Score	ROC-AUC	PR-AUC	Precision@15%	Recall@15%
Baseline Rule	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
Isolation Forest	0.8667	0.8667	0.8667	0.9922	0.9592	0.8667	0.8667
One-Class SVM	0.1333	0.1333	0.1333	0.6604	0.2740	0.1333	0.1333
Logistic Regression	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000
Optimized Ensemble	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000

Generated plots are saved in reports/ (roc_curve.png, precision_recall_curve.png, confusion_matrix.png, risk_distribution.png).

---

🏢 Employee Risk Ranking & Operational Prioritization

Model outputs are converted into operational risk classifications to guide human investigators:

• 🔴 Requires Investigation (High Risk): Average Ensemble Risk Score $\ge 0.55$ or Flag Rate $\ge 50%$.
• 🟡 Medium Risk: Average Ensemble Risk Score $\ge 0.40$ or Flag Rate $\ge 25%$.
• 🟢 Low Risk: Normal statistical parameters observed.

Employee ID	Records Audited	Records Flagged	Flag Rate	Avg Score Delta	Avg Risk Score	Risk Level
Emp #34	87	70	80.5%	3.21	0.6162	Requires Investigation (High Risk)
Emp #4	37	27	73.0%	2.92	0.6030	Requires Investigation (High Risk)
Emp #40	60	44	73.3%	3.07	0.5818	Requires Investigation (High Risk)
Emp #12	70	50	71.4%	2.63	0.5656	Requires Investigation (High Risk)

---

📂 Repository Architecture

maji_ndogo_project/
│
├── data/
│   ├── water_source.csv
│   ├── well_pollution.csv
│   ├── visits.csv
│   ├── location.csv
│   ├── infrastructure_cost.csv
│   └── employee_audit_dataset.csv
│
├── src/
│   └── maji_ndogo/
│       ├── __init__.py
│       ├── data.py              # Data loading & synthetic audit generator
│       ├── features.py          # Leakage-free historical expanding features
│       ├── models.py            # Model pipelines & validation ensemble optimizer
│       ├── evaluation.py        # Temporal splits, metrics, thresholding & plots
│       ├── explainability.py    # Risk factor extraction & feature attribution
│       ├── risk_scoring.py      # Employee risk aggregation & ranking
│       ├── inference.py         # Model persistence & prediction API
│       └── api.py               # FastAPI REST service (/health, /predict)
│
├── scripts/
│   ├── generate_audit_data.py   # CLI dataset generator
│   ├── train.py                 # Master training & evaluation script
│   ├── evaluate.py              # Model evaluation CLI
│   ├── predict.py               # Batch & single JSON inference CLI
│   └── run_fraud_audit.py       # Backward compatible entrypoint
│
├── dashboard/
│   └── app.py                   # Streamlit interactive dashboard
│
├── models/
│   └── trained_fraud_pipeline.joblib
│
├── reports/
│   ├── metrics.json
│   ├── model_comparison.csv
│   ├── employee_risk_rankings.csv
│   ├── roc_curve.png
│   ├── precision_recall_curve.png
│   ├── confusion_matrix.png
│   └── risk_distribution.png
│
├── tests/
│   ├── test_maji_ndogo_pipeline.py
│   └── test_fraud_detection.py
│
├── .github/
│   └── workflows/
│       └── ci.yml               # GitHub Actions CI workflow
│
├── Dockerfile
├── docker-compose.yml
├── pyproject.toml
└── README.md

---

🛠️ Quick Start & Installation

1. Prerequisites & Environment Setup

Clone the repository and sync dependencies using uv:

git clone https://github.com/Sudah2bot/maji_ndogo_project.git
cd maji_ndogo_project

# Install dependencies via uv
uv sync

2. Generate Audit Data & Train Pipeline

# Generate audit dataset
uv run python scripts/generate_audit_data.py --n-samples 2500

# Train model pipelines, optimize ensemble, and evaluate on test set
uv run python scripts/train.py --seed 42

3. Run Predictions via CLI

# Score a single JSON record
uv run python scripts/predict.py --single-json '{"assigned_employee_id": 34, "subjective_quality_score": 9.0, "audited_quality_score": 2.0, "survey_completion_time_min": 1.5, "location_anomaly_score": 0.85}'

4. Launch Streamlit Dashboard

uv run streamlit run dashboard/app.py

5. Launch FastAPI REST Service

uv run uvicorn maji_ndogo.api:app --host 0.0.0.0 --port 8000

• Health Check: GET http://localhost:8000/health
• Prediction Endpoint: POST http://localhost:8000/predict

6. Run Test Suite

uv run pytest

---

🐳 Docker Deployment

Run API & Dashboard via Docker Compose:

docker-compose up --build

• Streamlit Dashboard: http://localhost:8501
• FastAPI REST API: http://localhost:8000/docs

---

⚖️ Ethical Considerations & Limitations

1. Risk Signals vs. Misconduct: Model predictions represent operational risk scores to guide human audit selection. They must not be used as automated proof of employee misconduct.
2. Synthetic Fraud Ground Truth: Fraud labels are synthetically generated due to absence of verified historical fraud labels. Real-world deployment requires human-in-the-loop audit verification.
3. Human-in-the-Loop Review: All flagged high-risk records must be verified by independent Ministry of Water auditors before administrative action is taken.

---

📜 License

Distributed under the MIT License.