CHANGELOG.md

Changelog

All notable changes to this project are documented in this file.

The format is based on Keep a Changelog.

1.0.0 - 2026-09-24

First release under the name translation_diff. This gem was published as deepl_diff through 2.2.0; everything here is relative to deepl_diff 2.2.0.

Breaking

• One TranslationDiff::RateLimitExceeded, whichever limiter noticed it. The two limiters used to raise two same-named classes under their own namespaces, so an application that rescued one and then switched rate_limiter from :redis to :active_record quietly stopped catching it. See Errors.

• Language validation is on by default. TranslationDiff.translate now refuses, before making a request, any source/target pair the shipped data doesn't list for that provider -- raising TranslationDiff::UnsupportedLanguageError. Data ships for DeepL, Google, Azure and ModernMT only; Amazon and LibreTranslate ship none, and a provider with no shipped data refuses nothing. Two escapes: pass assume_supported: true for one call, or set config.validate_languages = false globally. See Languages.

• Cache keys change for any document containing a pre or code element. pre and code are now opaque (see Added, below), so what gets sent to the provider changed, and what gets keyed changed with it; an entry cached under the old behaviour keeps serving what the old behaviour produced. Give the configuration a new cache_namespace, or let cache_ttl lapse, to get every such document retranslated. See Caching.

• A runtime cache_namespace change now moves the rate limiter too. cache_namespace names the limiter's own bookkeeping namespace as well as the cache store's; it used to move only the store, leaving the limiter counting silently under the old namespace. active_record_base behaves the same way and for the same reason -- the SQL-backed limiter builds its model from that class just as the store does. See Configuration.

• A provider with a blank cache_key now raises TranslationDiff::InvalidProviderError, not TranslationDiff::Translator::Error. The two are siblings under TranslationDiff::Error, not parent and child, so an application rescuing the old class specifically stops catching this failure. Rescue TranslationDiff::Error to catch both. See Errors.

• The cache stores, rate limiters and ActiveRecord plumbing moved under their registry's own namespace; nothing was left behind at the old name. TranslationDiff::MemoryCacheStore, RedisCacheStore and ActiveRecordCacheStore are now TranslationDiff::Stores::Memory, Stores::Redis and Stores::ActiveRecord; RedisRateLimiter and ActiveRecordRateLimiter are now TranslationDiff::RateLimiters::Redis and RateLimiters::ActiveRecord; TranslationDiff::ActiveRecordSupport is now TranslationDiff::ActiveRecord::Support. config.cache = :redis and the rest of the symbol-keyed configuration are unaffected -- only the constant a name resolves to changed. See Caching and SQL cache.

• Renamed the gem to translation_diff and the module to TranslationDiff.

• The provider (config.provider; see Removed below for what replaced the old TranslationDiff.api accessor) must satisfy the five-method contract (translate, optional detect, max_request_size, max_batch_size, cache_key) instead of being a raw client such as DeepL.

• translate takes keyword arguments -- translate(values, from:, to:, **options) -- and no longer accepts a positional options hash.

• Request-size and batch-size limits moved out of Chunker and into the provider (#max_request_size, #max_batch_size); they are no longer hard-coded to DeepL's numbers.

• Every cache key changes. The key now includes the provider's cache_key, a digest of the provider options, and lowercased language codes. Nothing cached by deepl_diff -- or by an earlier translation_diff prerelease -- is reused. The next translation of every sentence is a cache miss, once, everywhere.

• Dropped punkt-segmenter and, with it, its unicode_utils dependency. Sentence boundaries are now produced by config.segmenter, defaulting to TranslationDiff::Segmenters::Pragmatic, backed by the pragmatic_segmenter gem (MIT, zero dependencies of its own) -- so ox and pragmatic_segmenter are now the gem's only two runtime dependencies. Measured against the Golden Rules corpus -- the context "Golden Rules" do block of each of the 10 per-language spec files on diasks2/pragmatic_segmenter, 80 exemplars in total; a sample of the same corpus is in test/translation_diff/golden_rules_test.rb -- the default now scores 76/80 against punkt's 38/80 and the old in-house segmenter's 47/80; the gap is largest on languages with no letter case at all -- Arabic, Hindi, Armenian, Greek -- which the in-house segmenter cannot reason about by design.

• config.segmenter_instance.split_offsets now takes a second, optional language: keyword argument. pragmatic_segmenter picks its rule set by language and falls back to English rules without one, which can mis-segment other languages (Russian abbreviations, for one); from: is the only way a caller supplies it, and only when segmentation happens before language detection would need to run. Segmenters::Pragmatic normalises the code first -- downcased, region subtag dropped -- and falls back to English for anything pragmatic_segmenter does not recognise afterward. Without this, DeepL's own codes ("RU", "EN-GB") missed their rule set entirely: pragmatic_segmenter's lookup is case-sensitive and region-blind, so this gem's own flagship provider was hitting the broken path on every call.

• config.rate_limit now actually enforces the threshold you configure. TranslationDiff::RedisRateLimiter called Ratelimit#add(size), but that gem's signature is add(subject, count = 1) -- so it recorded the hit under a subject named after the character count, while exceeded? checked a subject nothing ever incremented. The limiter never limited anything, in every release back to v1.0.2 (tagged 2023-02-16, roughly three years ago). If you have rate_limit configured, your traffic has never actually been throttled; on upgrading to 3.1.0 it will be, for the first time, against a threshold you set once and have never seen fire. You changed no configuration, but your throttling behaviour changes on upgrade. Re-validate rate_limit and rate_interval before upgrading -- see "Rate limiting" in the README.

• rate_interval is silently clamped by the ratelimit gem's fixed bucket span to roughly 5-600 seconds (measured: 3600 becomes 600, 1 becomes 5). Combined with a limiter that now actually fires, an interval configured above 600 seconds is enforced over 600 seconds instead -- up to six times more eager than the configuration reads. Keep rate_interval within that range, or expect a tighter effective window than configured.

• Providers must inherit TranslationDiff::Provider. A duck-typed object is no longer accepted: the base class supplies the transport, the configuration check and the capability defaults, and a provider without them is a provider that fails in the ways this library has already been bitten by twice.

• provider.translate(texts, from:, to:, **options) is now provider.translate(request), taking a Translation::Request and returning a Translation::Response. The response carries the detected source language and, where the provider reports it, the characters billed.

• max_request_size and max_batch_size move from provider methods to Capabilities.

• TranslationDiff::Providers::Naming is gone; the registry stamps cache_key and Provider implements it.

• Every provider normalises language codes to the casing its own vendor documents and accepts either casing from the caller: DeepL upper-cases, the other five lower-case. A code carrying a script or region subtag ("zh-Hans", "pt-BR") is passed through untouched. Provider#language is the shared rule and self.language_case selects the casing, so a provider of your own gets it by inheriting. Previously only Google and DeepL normalised at all -- Amazon Translate rejected "EN"/"RU" and LibreTranslate answered 400, on every call, for anyone who followed the README's "switch provider by changing config.provider" with DeepL-style codes -- and DeepL upper-cased subtags too, corrupting "zh-Hans".

• deepl-rb and google-cloud-translate-v2 are no longer used at all. faraday and faraday-retry become runtime dependencies; aws-sigv4 is required lazily by the Amazon provider only.

• config.deepl_host is renamed config.deepl_api_base, matching the <provider>_api_base name every other provider uses. There is no alias: a configuration still setting deepl_host raises NoMethodError on TranslationDiff.configure. Rename it.

• A provider returning the wrong number of translations now raises TranslationDiff::ResponseError, not the error that used to live on Request; a rescue written to catch a short response that way no longer catches one. Both are TranslationDiff::Error, so a rescue of the base class is unaffected.

• A provider returning a well-formed response that carries no translation for one input -- Azure answers 200 for a batch where a single string failed -- also raises TranslationDiff::ResponseError, naming the position. It previously reached the spacing step and died there as NoMethodError.

• The translation pipeline is new code. TranslationDiff::Linearizer, Spacing, Chunker, Tokenizer, Cache and Request are gone as public constants. What replaces them: Document and Leaves (walking the caller's structure), Passage, Fragment and Segment (markup and prose, cut into sentences), Markup (entity references and a < that opens no tag), Batch (packing sentences into provider requests), SentenceCache (the cache key, read and write) and Translator (the coordinator TranslationDiff.translate and Context#translate now build). See How it works. If you referenced any of the six by name, that reference is now a NameError.

• TranslationDiff::Request::Error is now TranslationDiff::Translator::Error and TranslationDiff::Cache::Error is now TranslationDiff::SentenceCache::Error. There is no alias for either: this gem has never been published under the name translation_diff with those constants in it. TranslationDiff::Chunker::Error is gone with no replacement -- a single sentence too large to send now raises TranslationDiff::Batch::Error. All three remain TranslationDiff::Error, so a rescue of the base class is unaffected.

• TranslationDiff.translate and Context#translate raise ArgumentError when to: is missing or nil. The keyword still defaults to nil in the signature, and the message names it. Previously a nil target compared equal to a nil source, the call short-circuited as "same language" and your values came back untranslated, silently. If you have a caller reading to: out of a configuration that can be blank, it has been a no-op and will now raise.

• The cache instrumentation event fires once per translate call, not once per chunk. The cache is now consulted for every sentence in one read_multi before anything is batched. hits and misses still sum to the same totals over a call, so a counter that adds them up is unaffected; a counter of events, or a histogram of per-chunk hit ratios, will see the cardinality drop. request and rate_limit still fire once per batch sent.

• Two more cache keys move, beyond the entity and < fixes below. A sentence padded with Unicode whitespace -- a non-breaking space, say -- now keys as the bare sentence: the pipeline uses one Unicode-aware definition of padding everywhere, where the key used to be built with ASCII strip, which leaves a U+00A0 in place. And the whole document key format is otherwise unmoved: it is pinned by test against recorded values, and every other input in the corpus this rewrite was judged against produces the same key it did before.

Removed

• The four module-level accessors (TranslationDiff.api, .cache_store, .segmenter, .rate_limiter) and TranslationDiff::CACHE_NAMESPACE. Every setting now lives on TranslationDiff::Configuration, reached through TranslationDiff.config or TranslationDiff.configure.

Added

• Every event from one translate call now shares a call_id. Generated once per call, opaque, and never derived from the text, it lands in translate, cache, request, rate_limit, usage and cache_error alike. Before it, a subscriber receiving cache or request events had no way to tell which translate call they belonged to, short of tagging Thread.current itself -- a workaround that breaks the moment two translations share a thread. See Instrumentation.

• translate now carries characters: the total this call considered, hit or miss. A call served entirely from cache never fires a request event and used to report nothing about its size; it now reports a number there instead. request's own characters keeps its narrower meaning -- what one batch actually sent -- so the two fields share a name but not an event: summing the wrong one produces a wrong bill. See Instrumentation.

• TranslationDiff.preview predicts a translate call without making it. It answers how many sentences a call would send, how many the cache already has, and how many characters that is -- without calling a provider and without writing anything. A preview never pays for language detection, so from: is required wherever there is anything to preview; leaving it unset raises TranslationDiff::Previewer::Error. Built for an editor that wants to show "this edit will send 1 sentence" before the author saves. See Caching.

• pre and code are no longer sent for translation, and changing a configuration option at runtime now rebuilds only what it actually feeds. pre and code join script and style in config.opaque_elements, the set TranslationDiff::Passage never treats as prose -- default %i[script style pre code], widen or narrow it as needed -- after a live <pre><code> block came back from Google with jq '.meters' mangled into jq '.metros', because nothing told the pipeline that code holds language, not prose. Separately, provider and the cache, pool, rate and segmenter options each rebuild only their own collaborator now, instead of nothing: before this, switching provider at runtime meant TranslationDiff.reset! and reconfiguring from scratch, discarding a Redis pool that had no reason to go. See Configuration. Two upgrade consequences of this are filed under Breaking, above: cache keys changing for a document containing pre or code, and a runtime cache_namespace change now moving the rate limiter too.

• A usage instrumentation event, firing once per provider request, beside translate, cache, request and rate_limit. Its payload carries provider, characters (what this library sent, counted locally), billed_characters (what the provider said it charged, or nil), reported (whether the provider reports billing at all -- not that this response was billed) and model. Summing billed_characters across providers without checking reported first produces a total that is quietly too low: only three of the six built-in providers report billing at all. See Instrumentation.

• A SQL-backed cache store and rate limiter, for an application that runs Postgres or MySQL and does not want Redis for this alone. TranslationDiff::ActiveRecordCacheStore (config.cache = :active_record) and TranslationDiff::ActiveRecordRateLimiter (config.rate_limiter = :active_record) cache translations and throttle requests in the application's own database, exercised in CI against Postgres, MySQL and SQLite. Nothing on this branch is breaking: both are opt-in, the default resolution of cache and rate_limiter is untouched, and an application with redis_url set keeps getting Redis exactly as before. rails generate translation_diff:install writes the migration for both tables, idempotently -- every create_table and add_index in it carries if_not_exists: true; for anyone not on Rails, its body is in SQL cache verbatim -- the gem itself never runs DDL. rake translation_diff:prune ships inside the gem: a Railtie wires it into a Rails application's own rake tasks automatically, enhanced with :environment so it prunes that application's own configuration; a non-Rails application loads the task file itself and must configure TranslationDiff before running it, since the task gets no :environment-equivalent there. ActiveRecord 7.1 or newer is required when either is used, refused by name at build time rather than failing inside a query, and activerecord is never a dependency of this gem -- it is required lazily on first use, the same way redis already is. Four new configuration options: cache_table_name, rate_limit_table_name, active_record_base and cache_prune_probability; the last, along with cache_namespace, is now validated at configure time -- a cache_prune_probability that will not coerce to a number, or a cache_namespace over 64 characters, is refused before either reaches a query. See SQL cache.

• The SQL cache store's write joins the caller's transaction, and its errors are redacted, not silent. A rollback in the caller's transaction discards translations ActiveRecordCacheStore already wrote -- the largest behavioural difference from RedisCacheStore, which is never inside anyone's transaction. A failed write itself runs in its own savepoint, so it no longer aborts a transaction it does not own, and the TranslationDiff::Error it raises carries the adapter's error class, not the row. upsert_all inlines values rather than binding them, though, so a written translation still appears verbatim in the host application's own ActiveRecord log at debug -- unrelated to this gem's own logger option, which never prints content. The redaction now covers every ActiveRecord::ActiveRecordError the write path can raise, not only a statement failure -- an application whose GET requests are routed to a read replica by ActiveRecord::Middleware::DatabaseSelector gets ActiveRecord::ReadOnlyError there instead, and it is redacted the same way. Pointing active_record_base at a different database or a writer-role class does not exempt this store from that routing decision; see Rails replica routing for what does. See Transactions and What ends up in your log.

• A failing cache write no longer loses the translation it was caching -- for every store, not only the SQL one. Translator#fill now rescues a store failure, logs it, fires a new cache_error instrumentation event (provider and the error's class, never the text), and returns the translation regardless: the cache is an optimisation on top of a translation already paid for at the provider, and losing the write should never mean losing that. See The three write paths fail differently and Instrumentation. The rate limiter refuses rather than degrades under the same routing -- it runs before the provider is called, so nothing has been paid for yet -- but it too now raises a redacted TranslationDiff::Error rather than a raw ActiveRecord one. See Rails replica routing.

• MySQL: the migration's translation column now carries limit: 16_777_215, giving it MEDIUMTEXT instead of TEXT. TEXT caps at 65,535 bytes on MySQL; a single sentence over that size failed the whole batch it rode in with. This is a no-op on Postgres and SQLite -- neither has a length ceiling on text to begin with, and nothing else about the schema changes for either. An installation that already ran this migration on MySQL needs one statement, once: ALTER TABLE translation_diff_translations MODIFY translation MEDIUMTEXT NOT NULL; -- this gem never runs DDL, so nothing does this for you. See The migration.

• config.rate_limiter no longer requires config.rate_limit. Setting the limiter alone used to pass nil as the threshold, overriding the keyword default and crashing every check with ArgumentError: comparison of Integer with nil failed. An unset rate_limit now falls back to the limiter's own default -- 8,000 characters per rate_interval, the same for both shipped limiters. See Configuration options.

• A refused request now says what it hit. RateLimitExceeded carries a message naming the namespace, the threshold and the interval ("rate limit reached for translation-diff: 8000 characters per 60 seconds") -- never the content that tripped it. See Errors.

• cache_ttl of 0 or less now means never expires, and nil is reachable through TranslationDiff.configure. Previously nil was documented as meaningful but unreachable through the public configuration path, and 0 wrote a row whose expires_at was already in the past -- a cache entry that could never hit. Both now fold to the same nil expires_at. See cache_ttl becomes expires_at.

• ActiveRecordRateLimiter's sliding window is conservative, not exact. It sums the oldest bucket touching the trailing rate_interval seconds in full, even though that bucket is only ever partially inside the window, so the window actually enforced is rate_interval to rate_interval + rate_interval / 12 seconds -- slightly stricter than configured, never looser. See The rate limiter.

• write_multi(pairs) joins the cache store contract, as an optional method: a store that implements it gets one call carrying a whole batch of sentences instead of one call per sentence; a store that does not is still called once per sentence, exactly as before this method existed -- a custom cache store written against the older contract is unaffected. All three shipped stores implement it now: MemoryCacheStore and RedisCacheStore gain it here too, alongside ActiveRecordCacheStore. The two batching paths fail differently from the per-key one and from each other -- see The three write paths fail differently.

• Five more providers, all on the same base class and the same transport: :google (Cloud Translation v2), :azure (Azure AI Translator v3.0), :modernmt, :libretranslate and :amazon (Amazon Translate). Each declares its own options, its own limits, and what it can actually do -- see the provider table in the README, which is written from the same capabilities the library reads at runtime.

• A Google provider: config.provider = :google translates through Cloud Translation v2 (Basic) over HTTP directly, with no Google gem installed. It declares google_api_key, google_project_id and google_api_base; an API key alone is enough. With no key configured it reads TRANSLATE_KEY and then GOOGLE_CLOUD_KEY, and google_project_id falls back to TRANSLATE_PROJECT -- the variables google-cloud-translate-v2 used to read on your behalf. Application default credentials are not supported: that path lived in the gem that is gone, and an application relying on ADC must now configure an API key. The provider asks for format: :html, which the tokenizer's output requires -- a notranslate span is handed over with its tags -- and downcases bare language codes so a configuration written for DeepL ("EN") keeps working, leaving subtagged codes such as "zh-Hans" alone.

• Environment-variable credential fallbacks, read by this library now that the vendor SDKs that read them are gone: DEEPL_AUTH_KEY for deepl_api_key, TRANSLATE_KEY then GOOGLE_CLOUD_KEY for google_api_key, and TRANSLATE_PROJECT for google_project_id. Each is read on use rather than at load, so setting one after requiring the gem still works, and an explicitly configured value always wins. The Amazon provider deliberately has no environment fallback: aws-sigv4 is handed explicit credentials and this library does not implement the AWS credential chain.

• A provider declares a default for one of its options by writing key => default in configuration_options instead of a bare symbol; a callable default is evaluated on every read.

• TranslationDiff::Configuration, a declarative settings object built through the option(key, default) macro. Options fall back to their default until assigned, treat a blank string as unset, and support a callable default (evaluated on every read, not at load time). #copy carries option values into an isolated configuration without carrying already-built collaborators along with them.

• TranslationDiff::Registry, a small name -> class map backing every pluggable extension point: a registered class need only answer build(config). Three registries exist: TranslationDiff::Providers, TranslationDiff::Stores, and TranslationDiff::Segmenters.registry.

• TranslationDiff::Context, returned by TranslationDiff.context: an isolated configuration scope with the same #translate entry point as the TranslationDiff module, for multi-tenant and per-request configuration that never touches the global configuration.

• Instrumentation and logging: config.instrumenter (anything satisfying ActiveSupport::Notifications' interface) receives translate, cache, request and rate_limit events, each named <name>.translation_diff and carrying counts, language codes and provider names only -- never the text being translated, its translation, or a credential. config.logger receives a debug line per provider resolution, naming the provider class, and is held to the same guarantee: it is never handed to deepl-rb, which logs the whole request -- auth header and payload -- at DEBUG when given a logger of its own. See "Instrumentation and logging" in the README for how to opt into that logging deliberately. Both are no-ops, at no extra cost, when left unset.

• TranslationDiff::Providers.register raises when a provider declares a configuration_options name another provider already declared. The two would otherwise share one accessor on TranslationDiff::Configuration, so a credential set for one service would be sent to the other. The same provider redeclaring its own options stays silent, so a double require and Rails development reloading keep working.

• TranslationDiff::MemoryCacheStore, a bounded, in-process LRU, and the new default cache store when redis_url is not configured -- so the library works before any infrastructure does.

• TranslationDiff::Segmenters::Pragmatic, the default sentence segmenter, wrapping pragmatic_segmenter's per-language rule sets. Before segmenting, it shadows every single newline (one with no adjoining newline) to a space in a copy of the text -- pragmatic_segmenter otherwise treats almost any single newline as a sentence boundary candidate even with no punctuation at all, a false split that HTML text nodes routinely trigger via incidental source-formatting newlines -- then segments the shadow and recovers offsets against it, so the original text, newline included, reaches the output untouched. Blank-line runs (real paragraph breaks) are left alone. This costs one Golden Rules point (77 -> 76: a bare list of items separated by single newlines, with no punctuation, now segments as one unit instead of three) -- a deliberate trade, since that shape does not arise in this gem's actual input. It recovers offsets from the strings pragmatic_segmenter returns by locating each one in the shadow, in order, and keeps every offset it locates; the first sentence it cannot locate (pragmatic_segmenter's cleaner also collapses runs of three or more spaces and respaces abbreviations such as "Ph.D.", among other things it rewrites) ends the search -- but the boundary at the end of the last sentence it did locate is not discarded with the rest, since it was matched character for character too; only the genuinely unrecoverable remainder becomes one final unit. This is a coarsening, not a failure -- every offset it ever emits has been verified, so the cache unit is simply larger, never wrong. It never guesses an offset it did not verify. TranslationDiff::Segmenters::Pragmatic::Error exists for the one case that would still be silent corruption -- offsets it computed itself violating their own postcondition (start at 0, strictly increase, stay within the text) -- not for ordinary pragmatic_segmenter rewriting.

• TranslationDiff::Segmenters::Simple (formerly TranslationDiff::Segmenter, renamed and moved to its own namespace alongside Pragmatic), the zero-dependency, in-house sentence segmenter this gem shipped with before pragmatic_segmenter became the default. It is deliberately conservative: it splits only on a handful of strong signals (a terminator followed by whitespace and an uppercase or CJK next character, none of the guard conditions -- a known abbreviation, an initial, digits on both sides, or a URL/email -- matching) so that a missed sentence boundary, which only costs a cache hit, is always preferred over a false one, which sends half a sentence to the translation provider. Its central rule has no meaning in scripts without letter case, which is why it is no longer the default; it stays available for callers who want no extra dependency and translate only from cased scripts. config.segmenter is swappable the same way config.provider and .cache are.

• TranslationDiff::Providers::DeepL and TranslationDiff::Providers::Null.

• test/support/provider_contract.rb, the executable form of the provider contract; any third-party provider can include it to verify it behaves as documented.

• detect is now its own provider method. Previously api.translate(sample, nil, to) returned an object of a different shape than the same call with from: given -- one method, two return types, told apart by an argument's value. A provider with no detect makes from: required and raises a clear error when it is missing, instead of NoMethodError.

• Four new providers: TranslationDiff::Providers::Azure (:azure), TranslationDiff::Providers::ModernMT (:modernmt), TranslationDiff::Providers::LibreTranslate (:libretranslate), and TranslationDiff::Providers::Amazon (:amazon), Amazon Translate, signed with aws-sigv4 rather than headed. Every provider's limits, HTML support, notranslate handling, detection and billing reporting are declared through Capabilities and measured against the vendor rather than assumed -- see the provider table in the README.

• An error hierarchy for everything a provider's transport can do wrong: TranslationDiff::ConfigurationError, TranslationDiff::ProviderError (and its AuthenticationError, QuotaExceededError, InvalidRequestError, ServiceError and RateLimitError subclasses), TranslationDiff::TransportError, TranslationDiff::ResponseError and TranslationDiff::InvalidProviderError, all under TranslationDiff::Error.

• config.open_timeout, config.timeout and config.max_retries, read by every HTTP provider's connection and retry policy.

Changed

• Adapters renamed to Providers, and providers are now registered by name through TranslationDiff::Providers.register rather than assigned directly to a module accessor. A provider built through the registry is stamped with its registered name and needs no cache_key of its own; an object assigned straight to config.provider, bypassing the registry, must define cache_key itself or every call through it raises -- it has no registered name to fall back on.
• The cache key's options digest uses a canonical, version-stable encoding instead of Hash#inspect, which renders symbol-keyed hashes differently across Ruby 3.2-4.0.

Fixed

• A notranslate span nested inside an opaque element (pre, code, script or style) no longer silences every sentence after it. Closing the protected span used to leave the scanner's own opacity depth one too high, so nothing past it was ever handed to the segmenter again. Found while adding the pre/code opaque elements above, and fixed the same way for all four. See How it works.

• Google and DeepL translations in HTML mode no longer come back double-escaped. Both vendors return entity-escaped text -- an apostrophe as ', a quote as ", an ampersand as & -- and the pipeline decoded entities on the way in but never on the way out, so the renderer escaped the vendor's own & a second time and a reader saw didn't on the page. English is full of apostrophes, so in practice every Google or DeepL translation into English was affected somewhere. TranslationDiff::Translation::Response.build now decodes a provider's reply the same way it already decoded the source, symmetrically, for every provider -- named entities, and both the decimal (') and hex (') numeric forms, are decoded; an entity neither decoder recognizes is left exactly as it arrived. See How it works.

• Behaviour change: a literal < in a source sentence now renders as <. Decoding the fix above exposed a second bug: a provider's own < now decoded to a bare <, and a bare < in front of a letter reads as an opening tag -- a provider could inject markup into the rendered document. A translated < that is not shaped like a tag is now escaped on render instead. if a < b then stop. used to come back with the bare < exactly as written; it now comes back if a < b then stop., the correct HTML encoding of that character and identical once a browser renders it -- but visible to anything comparing output byte-for-byte against an earlier release. > is untouched: a stray > never opens anything a parser would honour. See How it works.

• A warm cache keeps serving the corrupted text after you upgrade. A cache entry's key is derived from the source sentence, not from the value stored under it, so an entry written before this fix is served exactly as it was written until it expires -- upgrading alone does not clear it. Give the configuration a new cache_namespace, or let cache_ttl lapse, to force every sentence to be retranslated under the fix. See Caching.

• notranslate works with DeepL. The provider sent no tag_handling, and per DeepL's documentation "tags are treated as regular text" without it, so a span the tokenizer had marked as protected was translated anyway. The provider now sends tag_handling: :html and tag_handling_version: "v2", both overridable per call. This has been broken since the gem moved from Google to DeepL: tag_handling appears nowhere in the repository's history before this change. It failed quietly, because DeepL leaves the tags themselves alone either way and only the protected content came back changed.

• Comments, doctypes and CDATA sections survive a translation. Tokenizer declared no handler for those three Ox SAX events, so the bytes each one covered belonged to no token: a leading <!-- ... --> or <!DOCTYPE html> disappeared from the result outright, and a comment in the middle of a sentence leaked its <! into the surrounding text and handed the comment's own contents to the translation provider as prose -- both paying for the characters and risking an internal note coming back translated in place of the comment.

• TranslationDiff::RedisRateLimiter requires ratelimit lazily, on the first check, and raises TranslationDiff::Error naming the gem to add when it is missing. Previously the bare constant surfaced a raw NameError instead of the message the Redis path already raises.

• DeepL's batch limit was declared as 300 sentences per request; DeepL documents 50. The request-size limit (1,700 escaped characters) was already correct and is unchanged.

• An entity reference no longer reaches the provider raw. Salt & pepper. was sent to the provider as the six characters &, so the provider translated the entity's spelling as if it were words -- and was billed for it. It is now sent as Salt & pepper., the text the document actually says, and re-encoded on the way out. Named entities, & and & alike are decoded; anything neither decoder knows is left as it arrived.

• A bare < no longer swallows the rest of the sentence. if a < b then stop. Fine. was parsed by ox as prose followed by an unclosed tag, so only if a was ever sent for translation and everything after the < came back untranslated. A < that no element name, closing name, declaration or instruction follows is now escaped before parsing and restored after, so the whole sentence is translated. 5 < 6 and 7 > 6. True. was sent as three fragments and is now sent as two sentences.

• These two fixes move the cache key for the documents they affect. A document containing an entity reference, or a < that opens no tag, will miss the cache once and be re-translated. That is the point: what was cached for it was translated from the wrong text.

• A known remaining limit: < still reaches a provider undecoded, and a <b then stop. Fine. still loses everything after the <. Both fall out of escaping a bare < as < to work around ox rather than replacing it with a lexer of this gem's own, which is out of scope here. <b cannot be told apart from a tag without one. Every other entity is decoded.

Security

• Configuration#inspect and Provider#inspect print [FILTERED] in place of every credential option's value, instead of the credential itself. The filtered set is derived, not hand-maintained: option names matching a sensitive pattern, plus whatever each registered provider declares in sensitive_options. A non-credential option -- a base URL, a region, cache_namespace -- stays visible in full. A URL-valued option that carries a credential in its userinfo, redis_url included, has just that part redacted (rediss://default:[FILTERED]@cache.example.upstash.io:6379); the scheme, host, port and path stay visible. See Providers.

deepl_diff

The releases below were published under the gem's former name, deepl_diff.

2.2.0 - 2026-09-07

Message-only release under the deepl_diff name. No code changes.

Added

• spec.post_install_message, pointing installers at translation_diff, the gem's new name. deepl_diff 2.2.0 is the last release under that name and keeps working for anyone pinned to it.

2.1.0 - 2026-09-07

Six bugs found while auditing the library, each reproduced against the real behaviour and covered by a regression test.

Added

• DeepLDiff::Request::Error, raised when the adapter returns fewer translations than requested, instead of letting nil reach Spacing and fail two layers away as NoMethodError.

Fixed

• The options hash passed to Request was consumed with Hash#delete, so a second call with the same hash -- or any frozen hash -- broke.
• The chunker compared raw String#size against a limit meant for the escaped request, so Cyrillic and other non-Latin text could run several times over the actual request-size limit.
• A detected source language (a String) was compared against :to (usually a Symbol) with ==, so the same-language short circuit never fired and text was translated into its own language.
• Non-string scalars (nil, Integer, Symbol) raised instead of passing through untouched.
• The chunk count limit was off by one.

Upgrade note

The chunker fix changes how non-Latin text is split into requests: chunks are smaller, so there are more requests for the same number of characters billed. A MAX_CHUNK_SIZE tuned empirically against Cyrillic now produces a very different payload than it did.

2.0.0 - 2026-09-07

Breaking

• Six unused or duck-typed runtime dependencies were dropped: dry-initializer, connection_pool, redis, deepl-rb, redis-namespace, ratelimit. Applications that relied on this gem to install them must now depend on them directly.
• RedisRateLimiter takes threshold: and interval: as keyword arguments.

Fixed

• RedisRateLimiter declared threshold and interval as positional parameters, so the keyword call the README already documented silently discarded them and fell back to the defaults (8000 / 60).

1.1.1 - 2026-09-07

No changes to lib/.

Changed

• The test suite moved from RSpec to Minitest.
• The gem is published from CI through RubyGems trusted publishing instead of a personal API key.

1.1.0 - 2026-09-07

Changed

• Every runtime dependency is now explicitly versioned; the gem requires Ruby >= 3.2.
• CI moved from Travis to GitHub Actions, running against Ruby 3.2, 3.3, 3.4 and 4.0, plus a RuboCop job.

Fixed

• cgi/escape, digest/md5, forwardable and stringio are now required explicitly instead of relying on them being pulled in transitively.
• DeepL options were passed via **options into a method that takes a positional hash, so an empty options hash passed nothing.
• Chunker::Chunk members values/size were renamed (eventually to texts/escaped_size) because they shadowed Struct#values and Struct#size.