Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
a57a21a
Step 1: Update Ruby version specification and remove ActiveSupport up…
acastro2 Dec 10, 2025
d3419e2
Step 2: Update all gem dependencies to latest compatible versions Bui…
acastro2 Dec 10, 2025
65e3664
Step 3: Address Ruby 3.x compatibility issues and update deprecated c…
acastro2 Dec 10, 2025
9b81ef1
Step 4: Update GitHub Actions CI workflow for multi-version Ruby test…
acastro2 Dec 10, 2025
5266364
Step 5: Update documentation to reflect new Ruby version requirements…
acastro2 Dec 10, 2025
b834e39
Step 6: Add Redis mocking to test suite for environments without Redi…
acastro2 Dec 10, 2025
1021d0f
Add required_ruby_version constraint to gemspec - Add spec.required_r…
acastro2 Dec 10, 2025
dc314ac
Add Redis 8.x and Valkey support with comprehensive CI testing - Add …
acastro2 Dec 10, 2025
993eb32
Fix YAML formatting in CI configuration - Remove trailing spaces - Mo…
acastro2 Dec 10, 2025
8be642f
Add comprehensive Sidekiq version testing to CI matrix - Add Sidekiq …
acastro2 Dec 10, 2025
e19b041
Fix test compatibility with Sidekiq 8.x - Add version-aware Redis moc…
acastro2 Dec 10, 2025
264b278
Use real Redis when available, fall back to mocks CRITICAL FIX: Tests…
acastro2 Dec 10, 2025
51a79f8
Update TESTING.md to document hybrid real/mock Redis approach - Expla…
acastro2 Dec 10, 2025
031d88c
Remove redis and redis-client from gemspec dependencies CRITICAL FIX:…
acastro2 Dec 10, 2025
f5cbe9c
Update README to clarify Redis version requirements Redis version dep…
acastro2 Dec 10, 2025
01d85ff
Remove bundler from development dependencies CRITICAL FIX: bundler sh…
acastro2 Dec 10, 2025
d0069e9
Fix CI matrix to respect Sidekiq Ruby version requirements CRITICAL F…
acastro2 Dec 10, 2025
2b3fe48
Update README with Sidekiq 8 Ruby version requirement Document that S…
acastro2 Dec 10, 2025
770b670
Remove 'gem install bundler' from CI workflow CRITICAL FIX: The CI wa…
acastro2 Dec 10, 2025
e9b0020
Fix CI bundler installation and remove redis-client require CRITICAL …
acastro2 Dec 10, 2025
a38b27f
Fix redis_available? to handle LoadError gracefully CRITICAL FIX: The…
acastro2 Dec 10, 2025
9db762e
Add redis as explicit dependency and fix gauge value conversion CRITI…
acastro2 Dec 10, 2025
dfbdd3d
Quote Ruby version numbers in CI matrix to prevent YAML parsing issue…
acastro2 Dec 10, 2025
684ba7a
Fix Sidekiq 4-6 compatibility issues in ServerMiddleware and tests CR…
acastro2 Dec 10, 2025
1ad7474
Handle redis gem 5.x Array return value for hgetall in Sidekiq contex…
acastro2 Dec 10, 2025
ef0da98
Fix CI matrix: Sidekiq 7-8 require Redis 6+ for redis-client HELLO co…
acastro2 Dec 10, 2025
bf5dfb7
Bump version to 0.9.0 Minor version bump for Ruby 3.3 upgrade and com…
acastro2 Dec 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 182 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
name: CI

# Test matrix strategy:
# - Sidekiq 4.x-6.x: Compatible with Ruby 2.7-3.3, Redis 4+
# - Sidekiq 7.x-8.x: Require Redis 6+ (redis-client gem needs HELLO command)
# - Sidekiq 8.x: Additionally requires Ruby >= 3.2
# - We test representative combinations of Ruby + Sidekiq + Redis/Valkey versions
# - Total: 21 Redis tests + 7 Valkey tests = 28 jobs (down from 175 invalid combinations)

on:
push:
branches:
Expand All @@ -13,24 +20,196 @@ jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
ruby-version: [2.6, 2.7, 3.0]
redis-version: [4, 5, 6, 7]
include:
# Sidekiq 4.x: Ruby 2.7-3.3, Redis 4-8
- ruby-version: '2.7.8'
sidekiq-version: '4'
redis-version: '4'
- ruby-version: '2.7.8'
sidekiq-version: '4'
redis-version: '7'
- ruby-version: '3.0'
sidekiq-version: '4'
redis-version: '5'
- ruby-version: '3.3'
sidekiq-version: '4'
redis-version: '8'

# Sidekiq 5.x: Ruby 2.7-3.3, Redis 4-8
- ruby-version: '2.7.8'
sidekiq-version: '5'
redis-version: '4'
- ruby-version: '3.1'
sidekiq-version: '5'
redis-version: '6'
- ruby-version: '3.3'
sidekiq-version: '5'
redis-version: '8'

# Sidekiq 6.x: Ruby 2.7-3.3, Redis 4-8
- ruby-version: '2.7.8'
sidekiq-version: '6'
redis-version: '4'
- ruby-version: '3.0'
sidekiq-version: '6'
redis-version: '5'
- ruby-version: '3.2'
sidekiq-version: '6'
redis-version: '7'
- ruby-version: '3.3'
sidekiq-version: '6'
redis-version: '8'

# Sidekiq 7.x: Ruby 2.7-3.3, Redis 6-8 (redis-client requires Redis 6+ for HELLO command)
- ruby-version: '2.7.8'
sidekiq-version: '7'
redis-version: '6'
- ruby-version: '3.0'
sidekiq-version: '7'
redis-version: '6'
- ruby-version: '3.1'
sidekiq-version: '7'
redis-version: '7'
- ruby-version: '3.2'
sidekiq-version: '7'
redis-version: '7'
- ruby-version: '3.3'
sidekiq-version: '7'
redis-version: '8'

# Sidekiq 8.x: Ruby 3.2+, Redis 6-8 (Sidekiq 8 requires Ruby >= 3.2, redis-client requires Redis 6+)
- ruby-version: '3.2'
sidekiq-version: '8'
redis-version: '6'
- ruby-version: '3.2'
sidekiq-version: '8'
redis-version: '7'
- ruby-version: '3.3'
sidekiq-version: '8'
redis-version: '7'
- ruby-version: '3.3'
sidekiq-version: '8'
redis-version: '8'

steps:
- name: Checkout project
uses: actions/checkout@v3

- name: Create Gemfile for specific Sidekiq version
run: |
cat > Gemfile.ci <<EOF
source 'https://rubygems.org'

gem 'sidekiq', '~> ${{ matrix.sidekiq-version }}.0'

gemspec
EOF
cat Gemfile.ci

- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
bundler-cache: false
env:
BUNDLE_GEMFILE: Gemfile.ci

- name: Install dependencies
env:
BUNDLE_GEMFILE: Gemfile.ci
run: |
bundle install --jobs 4 --retry 3
echo "=== Installed Sidekiq version ==="
bundle exec gem list sidekiq

- name: Start Redis
uses: supercharge/redis-github-action@1.5.0
with:
redis-version: ${{ matrix.redis-version }}

- name: Run tests
env:
BUNDLE_GEMFILE: Gemfile.ci
run: bundle exec rake

test-valkey:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# Sidekiq 4.x-6.x: Ruby 2.7-3.3, Valkey 7-8
- ruby-version: '2.7.8'
sidekiq-version: '4'
valkey-version: '7'
- ruby-version: '3.3'
sidekiq-version: '5'
valkey-version: '8'
- ruby-version: '3.1'
sidekiq-version: '6'
valkey-version: '7'

# Sidekiq 7.x: Ruby 2.7-3.3, Valkey 7-8
- ruby-version: '2.7.8'
sidekiq-version: '7'
valkey-version: '7'
- ruby-version: '3.2'
sidekiq-version: '7'
valkey-version: '8'

# Sidekiq 8.x: Ruby 3.2+, Valkey 7-8 (Sidekiq 8 requires Ruby >= 3.2)
- ruby-version: '3.2'
sidekiq-version: '8'
valkey-version: '7'
- ruby-version: '3.3'
sidekiq-version: '8'
valkey-version: '8'

services:
valkey:
image: valkey/valkey:${{ matrix.valkey-version }}-alpine
ports:
- 6379:6379
options: >-
--health-cmd "valkey-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

steps:
- name: Checkout project
uses: actions/checkout@v3

- name: Create Gemfile for specific Sidekiq version
run: |
cat > Gemfile.ci <<EOF
source 'https://rubygems.org'

gem 'sidekiq', '~> ${{ matrix.sidekiq-version }}.0'

gemspec
EOF
cat Gemfile.ci

- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: false
env:
BUNDLE_GEMFILE: Gemfile.ci

- name: Install dependencies
env:
BUNDLE_GEMFILE: Gemfile.ci
run: |
bundle install --jobs 4 --retry 3
echo "=== Installed Sidekiq version ==="
bundle exec gem list sidekiq

- name: Run tests
env:
BUNDLE_GEMFILE: Gemfile.ci
run: bundle exec rake
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@
/pkg/
/spec/reports/
/tmp/
sidekiq-instrument-*
sidekiq-instrument-*
.log
1 change: 1 addition & 0 deletions .ruby-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.3.10
47 changes: 47 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed

- **BREAKING**: Minimum Ruby version raised from 2.6 to 2.7.8
- Updated all gem dependencies to their latest compatible versions
- Removed upper version constraint on ActiveSupport dependency (was `< 7.2`, now `>= 5.1` with no upper limit)
- Removed upper version constraint on Sidekiq dependency (was `< 7`, now `>= 4.2` with no upper limit)
- Updated Bundler to version 4.0.1
- Migrated to Ruby 3.x modern syntax patterns (frozen_string_literal, **dir**)
- Updated test suite for Sidekiq 8.x API compatibility (Sidekiq.configure_server)

### Added

- Ruby 3.3 support and full compatibility
- CI testing for Ruby versions 2.7.8, 3.0, 3.1, 3.2, and 3.3
- Redis gem dependency (>= 4.0) for compatibility with worker metrics
- **Redis 8.x support** - CI now tests against Redis 4.x, 5.x, 6.x, 7.x, and 8.x (latest of each major)
- **Valkey support** - Full compatibility with Valkey 7.x and 8.x as a Redis-compatible alternative
- **Comprehensive Sidekiq version testing** - CI now tests against Sidekiq 4.x, 5.x, 6.x, 7.x, and 8.x (latest of each major)
- CI testing matrix: 5 Ruby versions × 5 Sidekiq versions × 5 Redis versions = 125 combinations (plus 50 Valkey combinations)
- Gemspec constraint: `required_ruby_version >= 2.7.8` to enforce minimum Ruby version at installation

### Upgraded

- Sidekiq: now supports 8.0.x (previously limited to < 7)
- ActiveSupport: now supports 8.x (previously limited to < 7.2)
- RSpec: 3.13.x
- Rubocop: 1.81.x
- Rake: 13.3.x
- dogstatsd-ruby: 5.7.x
- statsd-instrument: 3.9.x
- redis: 5.4.x
- redis-client: 0.26.x
- simplecov: 0.22.x

### Removed

- Ruby 2.6 support (end of life)
- Version constraints that prevented using latest gem versions
53 changes: 48 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,38 @@

Reports job metrics using Shopify's [statsd-instrument][statsd-instrument] library and \[optionally\] DataDog's [dogstatsd-ruby](https://github.com/DataDog/dogstatsd-ruby), incrementing a counter for each enqueue and dequeue per job type, and timing the full runtime of your perform method.

## Requirements

- **Ruby**: 2.7.8 or higher
- **Sidekiq**: 4.2 or higher (tested with Sidekiq 4.x - 8.x)
- **Note**: Sidekiq 8.x requires Ruby 3.2 or higher
- **ActiveSupport**: 5.1 or higher (no upper version constraint)
- **Redis Server**: Version depends on Sidekiq version
- Sidekiq 4-6: Redis 4.0+ (tested with Redis 4-8)
- Sidekiq 7-8: **Redis 6.0+** required (redis-client gem needs HELLO command)
- **Valkey**: 7.2+ (Redis-compatible alternative, works with Sidekiq 7+)

This gem is tested against:

- Ruby versions: 2.7.8, 3.0, 3.1, 3.2, 3.3
- Sidekiq versions: 4.x, 5.x, 6.x, 7.x, 8.x (latest of each major)
- Redis server versions: 4.x-8.x (4-5 for Sidekiq 4-6, 6-8 for Sidekiq 7-8)
- Valkey versions: 7.x, 8.x (latest of each major)

### Ruby + Sidekiq + Redis Compatibility Matrix

| Sidekiq Version | Minimum Ruby | Min Redis Server | Tested Ruby Versions |
| --------------- | ------------ | ---------------- | ------------------------- |
| 4.x - 6.x | 2.7.8 | 4.0+ | 2.7.8, 3.0, 3.1, 3.2, 3.3 |
| 7.x | 2.7.8 | **6.0+** | 2.7.8, 3.0, 3.1, 3.2, 3.3 |
| 8.x | 3.2.0 | **6.0+** | 3.2, 3.3 |

**CI Coverage:** 30 Redis test combinations + 7 Valkey test combinations = **37 valid CI test jobs** covering all supported Ruby, Sidekiq, and Redis/Valkey version combinations.

### Redis and Valkey Support

This gem works with both Redis and Valkey (a Redis-compatible alternative). Valkey is a high-performance data store that maintains protocol compatibility with Redis, making it a drop-in replacement for most Redis use cases.

## Installation

Add the following to your application's Gemfile:
Expand All @@ -13,12 +45,16 @@ gem 'dogstatsd-ruby' # optional

And then execute:

$ bundle
```sh
bundle
```

Or install the gem(s) yourself as:

$ gem install sidekiq-instrument
$ gem install dogstatsd-ruby # again, optional
```sh
gem install sidekiq-instrument
gem install dogstatsd-ruby # again, optional
```

## Usage

Expand Down Expand Up @@ -73,6 +109,7 @@ Sidekiq::Instrument::WorkerMetrics.namespace = <APP_NAME>
```

## StatsD Keys

For each job, the following metrics will be reported:

1. **shared.sidekiq._queue_._job_.schedule**: counter incremented each time a
Expand All @@ -97,13 +134,16 @@ The metric names can be changed by overriding the `statsd_metric_name`
method in your worker classes.

For each queue, the following metrics will be reported:

1. **shared.sidekiq._queue_.size**: gauge of how many jobs are in the queue
2. **shared.sidekiq._queue_.latency**: gauge of how long the oldest job has been in the queue

For each worker, the following metrics and tags will be reported:

1. **sidekiq.worker_metrics.in_queue.#{key}**: number of jobs "in queue" per worker, uses redis to track increment/decrement (**this metric is currently inaccurate**)

## DogStatsD Keys

For each job, the following metrics and tags will be reported:

1. **sidekiq.schedule (tags: {queue: _queue_, worker: _job_})**: counter incremented each time a
Expand All @@ -125,10 +165,12 @@ will have a `.retry` appended:
2. **sidekiq.dequeue.retry (tags: {queue: _queue_, worker: _job_})**

For each queue, the following metrics and tags will be reported:

1. **sidekiq.queue.size (tags: {queue: _queue_})**: gauge of how many jobs are in the queue
2. **sidekiq.queue.latency (tags: {queue: _queue_})**: gauge of how long the oldest job has been in the queue

For each worker, the following metrics and tags will be reported:

1. **sidekiq.worker_metrics.in_queue.#{key}**: number of jobs "in queue" per worker, uses redis to track increment/decrement (**this metric is currently inaccurate**)

## Worker
Expand Down Expand Up @@ -159,14 +201,15 @@ You can schedule this however you see fit. A simple way is to use [sidekiq-sched

## Development

**Note:** Development requires Ruby 2.7.8 or higher. The recommended Ruby version is 3.3.

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/enova/sidekiq-instrument.

Bug reports and pull requests are welcome on GitHub at <https://github.com/enova/sidekiq-instrument>.

## License

Expand Down
Loading