Skip to content

CHANGELOG.md

Latest commit

 

History

History
165 lines (125 loc) · 6.25 KB

CHANGELOG.md

File metadata and controls

165 lines (125 loc) · 6.25 KB
Raw

Changelog

2.7.0 - 2026-08-15

  • Add a keywords executable for TF-IDF keyword extraction. keywords fit builds a vocabulary from files or standard input, keywords extract reads a file, keywords info prints model statistics, and a bare text argument transforms that text. The command prints term:score pairs and maps stems back to whole words, so keywords "Ruby is elegant" prints elegant:0.61 ruby:0.61. Options set the model path, the top-N terms, quiet mode, --min-df, --max-df, and --ngram. Usage errors exit 2 and other errors exit 1. The gem now installs two executables, classifier and keywords.
  • Treat each line as a separate document during keywords fit. The document count controls the inverse document frequency, so a file with many lines contributes many documents.
  • Add Classifier::Streaming::MultiIO. It reads several IO objects or file paths as one sequential stream. It opens and closes each path one at a time, so a corpus larger than the file descriptor limit still fits.
  • Add String#stem_to_word_hash. It maps each stemmed root to the most frequent original word in the text.
  • Add the min_df and max_df readers to Classifier::TFIDF.
  • Accept a MultiIO in TFIDF#fit_from_stream and Streaming::LineReader.
  • Fix Marshal support in Classifier::Bayes. The dump left out min_word_length, so the restored classifier raised ArgumentError: comparison of Integer with nil failed on its first classify. A dump written by an older version still loads, and takes the configured default.
  • Fix LSI#highest_relative_content, which returned an Enumerator rather than the documented array of documents.
  • Fix LSI#highest_ranked_stems, which repeated one stem when the document vector held equal weights. It looked up each weight by value, so tied weights all resolved to the same index.
  • Add a docs/ reference for the command line tools, each classifier, persistence, streaming, and configuration.

2.6.0 - 2026-06-25

  • Add a --search flag to the command line tool to filter local models.
  • Add a model detail view to the command line tool.
  • Read model information in advance when the tool lists local models.

2.5.0 - 2026-06-01

  • Accept keyword arguments in train_from_stream.

2.4.0 - 2026-05-19

  • Make min_word_length configurable, so a caller can keep or drop short words during tokenization.
  • Add a Claude Code plugin with a skill and slash commands.

2.3.2 - 2026-01-01

  • Force UTF-8 encoding on the HTTP response body, so a remote model loads under any locale.

2.3.1 - 2026-01-01

  • Force UTF-8 encoding when the locale is not UTF-8. Model data and user input no longer raise an encoding error.

2.3.0 - 2025-12-31

  • Add a classifier executable with a model registry. The tool trains, classifies, and manages saved models from the shell.
  • Fix the examples and the broken links in the README.
  • Add Dependabot for automated dependency updates.

2.2.0 - 2025-12-29

  • Breaking: set required_ruby_version to >= 3.1. Older Ruby versions are no longer supported.
  • Add a k-Nearest Neighbors classifier.
  • Add a Logistic Regression classifier.
  • Add a TF-IDF vectorizer.
  • Add streaming training and incremental SVD, so a corpus larger than memory can train a model.
  • Add a hash-style API to add items to LSI.
  • Add keyword arguments to Bayes#train and Bayes#untrain.
  • Accept an array of categories in the classifier constructor.
  • Fix the sentence and paragraph splits in Summary.
  • Add property-based tests for the probabilistic invariants.

2.1.0 - 2025-12-28

  • Replace the optional GSL dependency with a bundled C extension for LSI. The extension has no external dependency and falls back to pure Ruby.
  • Add pluggable persistence backends through a storage API.
  • Add save and load methods for classifier persistence.
  • Add thread safety to the Bayes and LSI classifiers.
  • Expose the LSI tuning parameters, with validation and an introspection API.
  • Cache the expensive computations in the Bayes classifier.

2.0.0 - 2025-12-27

  • Breaking: replace the fixed 0.1 constant in the Bayes classifier with add-one (Laplace) smoothing, where P(word|category) = (count + 1) / (total + vocabulary_size). The smoothing now scales with the vocabulary size and applies to seen and unseen words alike. Classification scores change as a result.
  • Fix an LSI dimension mismatch in the pure Ruby SVD.
  • Fix the numerical stability of the SVD implementation.
  • Replace the separate RBS files with inline annotations.
  • Add a GitHub Actions workflow that publishes the gem on a version tag.
  • Add an LSI benchmark that compares GSL against pure Ruby.
  • Add RuboCop and SimpleCov.

1.4.4 - 2024-07-31

  • Improve the scaling of the LSI content node.

1.4.3 - 2024-07-31

  • Require set and use the explicit ::Set namespace.
  • Refactor prepare_category_name.

1.4.2 - 2024-07-31

  • Fix the word count when remove_category runs.
  • Add the mutex_m dependency and update the fast-stemmer version.

1.4.1 - 2024-07-31

  • Add remove_category to the Bayes classifier.

1.4.0 - 2024-07-31

  • Add classify_with_confidence to the LSI classifier.
  • Require mathn only for Ruby 2.5 and later, and add cmath for Ruby 2.7 and later.
  • Silence the warnings about an uninitialized $GSL, and correct the rb-gsl URL hint.
  • Package the test files in the gem.
  • Add a Gemfile.lock.

1.3.5 - 2018-04-17

  • Use Minitest for the test suite.
  • Add the mathn dependency, which Ruby 2.5.0 removed.
  • Make the gem installable through Bundler and a git remote.
  • Fix the gemspec and the unit tests.

1.3.4 - 2013-12-31

  • Use a prior in the Bayes classifier.
  • Change the skip word list from an array to a set.
  • Reduce the number of regular expression matches during tokenization.
  • Stem only the words that the tokenizer keeps.
  • Default the word hash values to 0.
  • Add a gemspec, a README, and Travis CI.

1.3.3 - 2010-07-06

  • Use fast-stemmer for the Porter stemmer.
  • Check $GSL before a call to Matrix.diag, so the code uses GSL::Matrix.

1.3.2 - 2010-07-06

  • Fix the reported issue #1.

1.3.1 and earlier

Versions 1.0 through 1.3.1 reached RubyGems on 2009-07-25. The git history before 2010 is too sparse to attribute each change to one of these versions.