Skip to content

About

This project sets up DVC for wine prediction dataset that uses S3 as Remote cloud storage service and git to track pointers to dataset

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

Wine Prediction using DVC

CI - Continuous Integration & Training CD - Continuous Deployment Model Testing

A machine learning project for wine quality prediction with MLOps best practices, featuring Data Version Control (DVC), MLflow tracking, and automated CI/CD pipelines.

📋 Overview

This project demonstrates a complete MLOps workflow for predicting wine quality using machine learning. It incorporates:

  • Data Version Control (DVC) for dataset management
  • MLflow for experiment tracking and model registry
  • Scikit-learn for model training
  • AWS S3 as remote storage for DVC
  • GitHub Actions for CI/CD automation
  • Continuous Training in CI pipeline

🚀 Features

  • ✅ Version-controlled datasets with DVC
  • ✅ Remote storage integration (AWS S3)
  • ✅ Experiment tracking with MLflow
  • ✅ Automated model training pipeline
  • ✅ CI/CD workflows for continuous integration and deployment
  • ✅ Reproducible ML workflows

📁 Project Structure

Wine-prediction-using-dvc/
├── .dvc/                    # DVC configuration
│   └── config              # DVC remote storage settings
├── .github/                 # GitHub Actions workflows (if configured)
│   └── workflows/
│       ├── ci.yml          # Continuous Integration
│       └── cd.yml          # Continuous Deployment
├── data/                    # Data directory (tracked by DVC)
│   └── wine_sample.csv.dvc # DVC metadata file
├── requirements.txt         # Python dependencies
├── .dvcignore              # DVC ignore patterns
├── .gitignore              # Git ignore patterns
└── README.md               # This file

🛠️ Prerequisites

  • Python 3.8+
  • Git
  • AWS account (for S3 remote storage)
  • AWS credentials configured

📦 Installation

  1. Clone the repository

    git clone https://github.com/Vidon212/Wine-prediction-using-dvc.git
    cd Wine-prediction-using-dvc
  2. Create a virtual environment

    python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
  3. Install dependencies

    pip install -r requirements.txt

🔧 DVC Configuration

Remote Storage Setup

This project uses AWS S3 for remote data storage:

# Configure DVC remote (already set up)
dvc remote list
# Output: wineremote s3://mlops-wine-dvc-bucket

AWS Credentials

Ensure your AWS credentials are configured:

# Option 1: AWS CLI
aws configure

# Option 2: Environment variables
export AWS_ACCESS_KEY_ID=your_access_key
export AWS_SECRET_ACCESS_KEY=your_secret_key
export AWS_DEFAULT_REGION=your_region

📊 Data Management

Pull Data from Remote Storage

# Pull the wine dataset from S3
dvc pull

This will download wine_sample.csv to the data/ directory.

Push Data to Remote Storage

# After adding or modifying data
dvc add data/wine_sample.csv
dvc push
git add data/wine_sample.csv.dvc .gitignore
git commit -m "Update dataset"

🤖 Model Training

Local Training

# Train the model locally
python train.py  # (if train.py exists)

CI/CD Training

The project includes continuous training in the CI workflow:

  • Automated Training: Model training runs automatically on each push to the repository
  • Artifact Storage: Trained models are uploaded as GitHub Actions artifacts
  • Reproducibility: Ensures consistent training environment across runs

🔄 CI/CD Workflows

Continuous Integration (CI)

The CI workflow (ci.yml) includes:

  • Code checkout
  • Python environment setup
  • Dependency installation
  • Model training
  • Artifact upload

Continuous Deployment (CD)

The CD workflow (cd.yml) handles:

  • Model deployment
  • Service updates
  • Production releases

📚 Dependencies

Key packages (see requirements.txt):

  • pandas: Data manipulation
  • scikit-learn: Machine learning algorithms
  • joblib: Model serialization
  • numpy: Numerical computing
  • mlflow: Experiment tracking
  • dvc: Data version control
  • dvc-s3: S3 remote storage support

🧪 Usage

  1. Pull the latest data

    dvc pull
  2. Train the model

    python train.py
  3. Track experiments with MLflow

    mlflow ui
    # Open http://localhost:5000 in your browser
  4. Version your changes

    git add .
    git commit -m "Your message"
    git push

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

📝 License

This project is open source and available under the MIT License.

🙏 Acknowledgments

  • DVC for data version control
  • MLflow for experiment tracking
  • Scikit-learn for ML algorithms
  • AWS S3 for remote storage

📧 Contact

Vidon - @Vidon212

Project Link: https://github.com/Vidon212/Wine-prediction-using-dvc


Note: Make sure to configure your AWS credentials before running DVC commands that interact with the S3 remote storage.

About

This project sets up DVC for wine prediction dataset that uses S3 as Remote cloud storage service and git to track pointers to dataset

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages