Add unit tests for merge functionality, including conflict resolution…

.github/copilot-instructions.md

@@ -5,28 +5,56 @@ Auto-generated from all feature plans. Last updated: 2026-01-06
 ## Active Technologies
 - Go 1.24.0+ + `github.com/schollz/progressbar/v3`, `golang.org/x/term` (005-console-progress-bar)
 - N/A (UI component only) (005-console-progress-bar)
-
 - Go 1.24.0+ (alpine-based Docker build) (002-containerization-documentation)
+- Go 1.24.0+ + `crypto/sha256`, `bufio.Scanner`, `encoding/json` (006-scrobble-dedup-merge)
 
 ## Project Structure
 
 ```text
 src/
 tests/
+cmd/lastfm-sync/commands/
+  - merge.go (006-scrobble-dedup-merge)
+internal/merge/
+  - deduplicator.go (006-scrobble-dedup-merge)
+  - conflict.go (006-scrobble-dedup-merge)
+  - reader.go (006-scrobble-dedup-merge)
+  - merger.go (006-scrobble-dedup-merge)
+  - strategies.go (006-scrobble-dedup-merge)
+  - checkpoint.go (006-scrobble-dedup-merge)
+  - config.go (006-scrobble-dedup-merge)
+tests/integration/merge_test.go (006-scrobble-dedup-merge)
 ```
 
 ## Commands
 
 # Add commands for Go 1.24.0+ (alpine-based Docker build)
 
+# Merge command (006-scrobble-dedup-merge)
+lastfm-sync merge [flags] <input-pattern...>
+  --output, -o: Output file path (default: merged-scrobbles.json)
+  --strategy: Deduplication strategy (default|strict|relaxed|mbid)
+  --conflict-resolution: Conflict resolution mode (completeness|first|last)
+  --checkpoint-interval: Save checkpoint every N scrobbles (default: 10000)
+  --resume: Resume from checkpoint file
+
 ## Code Style
 
 Go 1.24.0+ (alpine-based Docker build): Follow standard conventions
+Go 1.24.0+ (006-scrobble-dedup-merge): Follow standard Go conventions
+  - Use `bufio.Scanner` for NDJSON streaming
+  - Store pointers in maps: `map[string]*models.Scrobble`
+  - SHA256 keys as hex strings (64 chars)
+  - Cyclomatic complexity <10 per function
+  - 80%+ test coverage required
+  - Table-driven tests for strategy variations
 
 ## Recent Changes
+- 006-scrobble-dedup-merge: Added merge command for deduplicating and merging multiple NDJSON scrobble files. Uses in-memory hash map with SHA256 keys. Supports 4 deduplication strategies (default/strict/relaxed/mbid) and 3 conflict resolution modes (completeness/first/last). Includes checkpointing for resume capability. Performance targets: ≥10K scrobbles/sec, <500MB for 1M records. Reuses existing internal/writer, internal/progress, internal/models packages.
 - 005-console-progress-bar: Added Go 1.24.0+ + `github.com/schollz/progressbar/v3`, `golang.org/x/term`
 - 004-normalized-title-field: Adding `normalized_title` field to remove annotations (Live, Remastered, featuring, etc.) from track titles for better matching and grouping. Uses internal/normalize package with gopkg.in/yaml.v3 for configuration. DEBUG logging when titles modified.
 - 002-containerization-documentation: Added [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
 
 <!-- MANUAL ADDITIONS START -->
+If you notice any systemic issues please add the needed requirements to this file or to the constitution if that is more appropriate.
 <!-- MANUAL ADDITIONS END -->

.specify/scripts/bash/common.sh

@@ -28,7 +28,7 @@ get_current_branch() {
 
     # For non-git repos, try to find the latest feature directory
     local repo_root=$(get_repo_root)
-    local specs_dir="$repo_root/specs"
+    local specs_dir="$repo_root/.specify/specs"
 
     if [[ -d "$specs_dir" ]]; then
         local latest_feature=""
@@ -81,14 +81,14 @@ check_feature_branch() {
     return 0
 }
 
-get_feature_dir() { echo "$1/specs/$2"; }
+get_feature_dir() { echo "$1/.specify/specs/$2"; }
 
 # Find feature directory by numeric prefix instead of exact branch match
 # This allows multiple branches to work on the same spec (e.g., 004-fix-bug, 004-add-feature)
 find_feature_dir_by_prefix() {
     local repo_root="$1"
     local branch_name="$2"
-    local specs_dir="$repo_root/specs"
+    local specs_dir="$repo_root/.specify/specs"
 
     # Extract numeric prefix from branch (e.g., "004" from "004-whatever")
     if [[ ! "$branch_name" =~ ^([0-9]{3})- ]]; then

.specify/scripts/bash/create-new-feature.sh

@@ -130,7 +130,7 @@ fi
 
 cd "$REPO_ROOT"
 
-SPECS_DIR="$REPO_ROOT/specs"
+SPECS_DIR="$REPO_ROOT/.specify/specs"
 mkdir -p "$SPECS_DIR"
 
 # Function to generate branch name with stop word filtering and length filtering

.specify/specs/006-scrobble-dedup-merge/checklists/requirements.md

@@ -0,0 +1,83 @@
+# Specification Quality Checklist: Scrobble Deduplication and Merging
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning  
+**Created**: January 7, 2026  
+**Feature**: [006-scrobble-dedup-merge/spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Validation Results
+
+### Content Quality Assessment
+✅ **PASS** - Specification focuses on WHAT users need (merge, deduplicate, consolidated view) and WHY (analysis, backup, single source of truth). Written in plain language without Go-specific or framework details.
+
+✅ **PASS** - All mandatory sections present: User Scenarios & Testing, Requirements, Success Criteria, plus comprehensive additions (Assumptions, Constraints, Dependencies, Scope, Decisions, Risks, Timeline).
+
+### Requirement Completeness Assessment
+✅ **PASS** - All 51 functional requirements are specific and testable (e.g., "FR-007: System MUST identify duplicate scrobbles using configurable unique key").
+
+✅ **PASS** - All success criteria are measurable with specific metrics (e.g., "SC-002: System processes at least 10,000 scrobbles per second", "SC-006: Duplicate detection accuracy exceeds 99.9%").
+
+✅ **PASS** - Success criteria are technology-agnostic, describing outcomes from user perspective without implementation details.
+
+✅ **PASS** - 6 user stories with 5 acceptance scenarios each = 30 total acceptance scenarios covering all major flows.
+
+✅ **PASS** - Comprehensive edge case section with 12 specific scenarios and handling strategies.
+
+✅ **PASS** - Scope clearly defined with explicit "Out of Scope" section listing 15+ features deferred to future versions.
+
+✅ **PASS** - Dependencies section lists all required and optional dependencies with integration points. Assumptions section covers data, operational, and performance assumptions. Constraints section defines technical, business, UX, and data constraints.
+
+### Feature Readiness Assessment
+✅ **PASS** - Each functional requirement mapped to user scenarios; acceptance criteria defined in Given-When-Then format.
+
+✅ **PASS** - 6 prioritized user stories (P1-P3) cover: basic merge (P1), data quality handling (P2), conflict resolution (P2), preview/validation (P3), deduplication strategies (P3), long-running operations (P3).
+
+✅ **PASS** - 27 success criteria across performance, data quality, reliability, usability, cross-platform support, and testing.
+
+✅ **PASS** - Specification remains at "WHAT/WHY" level. Technical details appropriately placed in separate sections (algorithms, data structures) for implementer reference without contaminating requirements.
+
+## Notes
+
+**Specification Status**: ✅ COMPLETE AND READY
+
+This specification is **READY** to proceed to `/speckit.plan` phase. All quality criteria met:
+
+- **Zero [NEEDS CLARIFICATION] markers** - All decisions resolved in "Open Questions and Decisions" section with clear rationale
+- **Comprehensive coverage** - 51 functional requirements, 30 acceptance scenarios, 12 edge cases, 27 success criteria
+- **Technology-agnostic** - No implementation details in requirements; focuses on user outcomes and business value
+- **Well-bounded scope** - Clear "Out of Scope" section with 15+ deferred features
+- **Testable requirements** - Every requirement and success criterion is specific and measurable
+- **User-centric** - 6 prioritized user stories each independently testable and valuable
+
+**Strengths**:
+- Exceptional detail in deduplication strategies (4 options with clear use cases)
+- Comprehensive error handling scenarios with specific error codes and messages
+- Well-defined conflict resolution algorithm with clear precedence rules
+- Realistic timeline estimate (41-52 hours) with critical path and milestones
+- Thorough risk analysis with mitigations
+
+**Ready for Planning**: Yes - no specification updates required before proceeding to implementation planning.