Skip to content
Het836Public

About

This repository contains a Spring Boot-based Netflix-style media pipeline demo composed of multiple microservices. The project models a basic video lifecycle: movie metadata is stored in a catalog service, videos are uploaded to S3, processed by an encoding service using FFmpeg, and delivered through a streaming service with signed HLS URLs.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Netflix Microservices Demo

This repository contains a Spring Boot-based Netflix-style media pipeline demo composed of multiple microservices. The project models a basic video lifecycle: movie metadata is stored in a catalog service, videos are uploaded to S3, processed by an encoding service using FFmpeg, and delivered through a streaming service with signed HLS URLs.

Overview

The codebase includes four independent Java services:

  • content-service: manages movie metadata and catalog queries.
  • video-service: uploads raw video files to AWS S3 and publishes Kafka events.
  • encoding-service: consumes upload events, transcodes media to HLS/playlist files, and publishes success or failure events.
  • streaming-service: serves signed streaming URLs from S3 and caches playlist metadata in Redis.

A browser demo page, netflix-player.html, can be used to load a movie by ID and play the resulting HLS stream.

Key Features

  • Movie catalog API with CRUD-style read/search endpoints
  • Video upload endpoint using multipart form data
  • Kafka-based event-driven pipeline between services
  • FFmpeg-based HLS encoding for multiple quality levels
  • Secure S3 object access through presigned URLs
  • Redis caching for generated streaming URLs and playlist metadata
  • MySQL persistence for movie records
  • Docker Compose setup for Redis, MySQL, Zookeeper, and Kafka

Technology Stack

The project is built with the following verified technologies:

  • Java 17
  • Spring Boot 4.0.9
  • Spring Web MVC
  • Spring Data JPA
  • Spring Data Redis
  • Spring Kafka
  • MySQL Connector/J
  • AWS SDK for S3
  • Lombok
  • FFmpeg for media transcode to HLS
  • Docker Compose for local infrastructure

High-Level Architecture

Video upload flow
-----------------
video-service --> S3 upload --> Kafka topic: video.uploaded --> encoding-service
                                                                       |
                                                                       v
                                                            FFmpeg HLS encoding
                                                                       |
                                                                       v
                                              S3 encoded files + Kafka topic: video.encoded
                                                                       |
                                                                       v
                                            content-service updates movie status
                                            streaming-service stores playlist key and signs URLs
                                                                       |
                                                                       v
                                                    browser/player requests stream via netflix-player.html

Project Structure

Netflix/
├── docker-compose.yml
├── netflix-player.html
├── content-service/
│   ├── pom.xml
│   ├── mvnw
│   ├── src/main/java/com/netflix/content_service/
│   │   ├── controller/
│   │   ├── dto/
│   │   ├── model/
│   │   ├── repository/
│   │   └── service/
│   └── src/main/resources/application.yaml
├── video-service/
│   ├── pom.xml
│   ├── mvnw
│   ├── src/main/java/com/netflix/video_service/
│   └── src/main/resources/application.yaml
├── encoding-service/
│   ├── pom.xml
│   ├── mvnw
│   ├── src/main/java/com/netflix/encoding_service/
│   └── src/main/resources/application.yaml
├── streaming-service/
│   ├── pom.xml
│   ├── mvnw
│   ├── src/main/java/com/netflix/streaming_service/
│   └── src/main/resources/application.yaml
└── test-video.mp4

Prerequisites

Before running the project, make sure you have the following installed:

  • Java 17 or newer
  • Maven (or use the included Maven wrappers in each service)
  • Docker Desktop or Docker Compose
  • FFmpeg installed and available on the system path, or configured via ffmpeg.path
  • Access to an AWS S3 bucket for video storage
  • A MySQL instance (the project includes Docker Compose for MySQL)

Local Infrastructure

The root docker-compose.yml provisions the following services:

  • Redis on localhost:6379
  • MySQL on localhost:3306
  • Zookeeper on localhost:2181
  • Kafka on localhost:9092

Start the infrastructure from the project root:

docker compose up -d

Configuration and Environment Variables

Service-specific application.yaml and application.yml files are excluded from Git. Each service has an example.application.yml in its src/main/resources/ directory with all configuration keys and blank values. Copy the matching example to application.yml and fill in the values for your local setup. Spring does not load .env files automatically.

Common runtime configuration

The video, encoding, and streaming services read these properties from YAML and environment variables:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_REGION
  • AWS_BUCKET_NAME

These are used by the S3 clients in video-service, encoding-service, and streaming-service. The access key ID and secret are required; region defaults to ap-southeast-2.

Encoding service-specific configuration

The encoding service also reads:

  • FFMPEG_PATH (default in code: C:/ffmpeg-9.0.2-essentials_build/bin/ffmpeg.exe)
  • TEMP_DIR (default in code: C:/temp/encoding)

The defaults shown here are defined in the project configuration and may need to be adjusted for your machine.

MySQL database configuration

content-service/src/main/resources/application.yaml currently points to:

jdbc:mysql://localhost:3306/netflix_content_db

The root Docker Compose file currently creates a database named content_db instead. This mismatch is present in the codebase and may require updates to the datasource URL or database creation step to match the service configuration.

Running the Project

Each service is an independent Spring Boot application and can be started from its own directory.

1) Start the infrastructure

docker compose up -d

2) Start the content service

From content-service/:

./mvnw spring-boot:run

On Windows PowerShell:

./mvnw.cmd spring-boot:run

This service listens on port 8081.

3) Start the video service

From video-service/:

./mvnw spring-boot:run

This service listens on port 8082.

4) Start the encoding service

From encoding-service/:

./mvnw spring-boot:run

This service listens on port 8083.

5) Start the streaming service

From streaming-service/:

./mvnw spring-boot:run

This service listens on port 8084.

6) Open the player demo

Open netflix-player.html in a browser and enter a valid movieId returned by the content service.

Available Scripts and Maven Commands

This project does not define custom npm scripts. The available standard commands are the Maven lifecycle commands provided by each service's wrapper:

./mvnw clean package
./mvnw test
./mvnw spring-boot:run

Typical build in each service directory:

./mvnw -DskipTests package

API Endpoints

Content Service (http://localhost:8081)

Add a movie

POST /api/v1/movies
Content-Type: application/json

Example request body:

{
  "title": "Example Movie",
  "description": "A demo movie",
  "genre": "ACTION",
  "director": "Jane Director",
  "cast": "Actor One, Actor Two",
  "releaseYear": 2025,
  "rating": 8.4,
  "thumbnailUrl": "https://example.com/thumbnail.jpg",
  "durationMinutes": 118
}

Get all movies

GET /api/v1/movies

Get a movie by ID

GET /api/v1/movies/{movieId}

Get movies by genre

GET /api/v1/movies/genre/{genre}

Search movies by title

GET /api/v1/movies/search?title=example

Video Upload Service (http://localhost:8082)

Upload a video file

POST /api/v1/videos/upload/{movieId}
Content-Type: multipart/form-data

Example:

curl -F "file=@test-video.mp4" http://localhost:8082/api/v1/videos/upload/123e4567-e89b-12d3-a456-426614174000

The service uploads the file to S3 and publishes a Kafka event on video.uploaded.

Streaming Service (http://localhost:8084)

Get a signed HLS playlist URL

GET /api/v1/stream/{movieId}

Returns a JSON payload like:

{
  "movieId": "movie-id",
  "streamingURL": "https://.../master.m3u8?...",
  "quality": "1080p, 720p, 480p, 360p",
  "expiredInMinutes": 60
}

Get signed playlist content for HLS player requests

GET /api/v1/stream/{movieId}/playlist?path=encoded/movieId/1080p/playlist.m3u8

Usage Workflow

A typical end-to-end flow in this project is:

  1. Create a catalog entry in the content service.
  2. Upload a video file to the video service.
  3. The video service stores the raw video in S3 and sends a Kafka video.uploaded event.
  4. The encoding service processes the upload, creates HLS outputs, and stores them in S3.
  5. The encoding service sends a Kafka video.encoded event.
  6. The streaming service stores the master playlist key in Redis and issues signed URLs.
  7. The browser player requests the stream and plays the HLS content.

Notes

  • The project is a demonstration pipeline rather than a complete production-ready Netflix clone.
  • The repository includes a sample MP4 file, test-video.mp4, that can be used for upload testing.
  • The current codebase contains hardcoded AWS credential defaults in the YAML files and is intended for local development/testing rather than production deployment.
  • The FFmpeg path and temporary encoding directory are machine-specific and should be adjusted to the local environment.

Verification

The project was verified by compiling each service with Maven wrappers successfully:

content-service: OK
encoding-service: OK
streaming-service: OK
video-service: OK

About

This repository contains a Spring Boot-based Netflix-style media pipeline demo composed of multiple microservices. The project models a basic video lifecycle: movie metadata is stored in a catalog service, videos are uploaded to S3, processed by an encoding service using FFmpeg, and delivered through a streaming service with signed HLS URLs.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages