|
1 | | -## Welcome to Classifier |
| 1 | +# Classifier |
2 | 2 |
|
3 | | -Classifier is a general module to allow Bayesian and other types of classifications. |
| 3 | +[](https://badge.fury.io/rb/classifier) |
| 4 | +[](https://github.com/cardmagic/classifier/actions/workflows/ruby.yml) |
| 5 | +[](https://opensource.org/licenses/LGPL-2.1) |
4 | 6 |
|
5 | | -## Download |
| 7 | +A Ruby library for text classification using Bayesian and Latent Semantic Indexing (LSI) algorithms. |
6 | 8 |
|
7 | | -* https://github.com/cardmagic/classifier |
8 | | -* gem install classifier |
9 | | -* git clone https://github.com/cardmagic/classifier.git |
| 9 | +## Table of Contents |
10 | 10 |
|
11 | | -## Dependencies |
| 11 | +- [Installation](#installation) |
| 12 | +- [Bayesian Classifier](#bayesian-classifier) |
| 13 | +- [LSI (Latent Semantic Indexing)](#lsi-latent-semantic-indexing) |
| 14 | +- [Performance](#performance) |
| 15 | +- [Development](#development) |
| 16 | +- [Contributing](#contributing) |
| 17 | +- [License](#license) |
12 | 18 |
|
13 | | -The `fast-stemmer` gem is required: |
| 19 | +## Installation |
14 | 20 |
|
15 | | - gem install fast-stemmer |
| 21 | +Add to your Gemfile: |
| 22 | + |
| 23 | +```ruby |
| 24 | +gem 'classifier' |
| 25 | +``` |
| 26 | + |
| 27 | +Then run: |
| 28 | + |
| 29 | +```bash |
| 30 | +bundle install |
| 31 | +``` |
| 32 | + |
| 33 | +Or install directly: |
| 34 | + |
| 35 | +```bash |
| 36 | +gem install classifier |
| 37 | +``` |
16 | 38 |
|
17 | 39 | ### Optional: GSL for Faster LSI |
18 | 40 |
|
19 | | -For faster LSI classification, install the GNU Scientific Library and its Ruby bindings. |
| 41 | +For significantly faster LSI operations, install the [GNU Scientific Library](https://www.gnu.org/software/gsl/). |
| 42 | + |
| 43 | +<details> |
| 44 | +<summary><strong>Ruby 3.4+</strong></summary> |
| 45 | + |
| 46 | +The released `gsl` gem doesn't support Ruby 3.4+. Install from source: |
| 47 | + |
| 48 | +```bash |
| 49 | +# Install GSL library |
| 50 | +brew install gsl # macOS |
| 51 | +apt-get install libgsl-dev # Ubuntu/Debian |
| 52 | + |
| 53 | +# Build and install the gem |
| 54 | +git clone https://github.com/cardmagic/rb-gsl.git |
| 55 | +cd rb-gsl |
| 56 | +git checkout fix/ruby-3.4-compatibility |
| 57 | +gem build gsl.gemspec |
| 58 | +gem install gsl-*.gem |
| 59 | +``` |
| 60 | +</details> |
| 61 | + |
| 62 | +<details> |
| 63 | +<summary><strong>Ruby 3.3 and earlier</strong></summary> |
| 64 | + |
| 65 | +```bash |
| 66 | +# macOS |
| 67 | +brew install gsl |
| 68 | +gem install gsl |
| 69 | + |
| 70 | +# Ubuntu/Debian |
| 71 | +apt-get install libgsl-dev |
| 72 | +gem install gsl |
| 73 | +``` |
| 74 | +</details> |
| 75 | + |
| 76 | +When GSL is installed, Classifier automatically uses it. To suppress the GSL notice: |
| 77 | + |
| 78 | +```bash |
| 79 | +SUPPRESS_GSL_WARNING=true ruby your_script.rb |
| 80 | +``` |
| 81 | + |
| 82 | +### Compatibility |
| 83 | + |
| 84 | +| Ruby Version | Status | |
| 85 | +|--------------|--------| |
| 86 | +| 3.4 | Supported | |
| 87 | +| 3.3 | Supported | |
| 88 | +| 3.2 | Supported | |
| 89 | +| 3.1 | Supported | |
| 90 | + |
| 91 | +## Bayesian Classifier |
| 92 | + |
| 93 | +Fast, accurate classification with modest memory requirements. Ideal for spam filtering, sentiment analysis, and content categorization. |
| 94 | + |
| 95 | +### Quick Start |
| 96 | + |
| 97 | +```ruby |
| 98 | +require 'classifier' |
| 99 | + |
| 100 | +classifier = Classifier::Bayes.new('Spam', 'Ham') |
| 101 | + |
| 102 | +# Train the classifier |
| 103 | +classifier.train_spam "Buy cheap viagra now! Limited offer!" |
| 104 | +classifier.train_spam "You've won a million dollars! Claim now!" |
| 105 | +classifier.train_ham "Meeting scheduled for tomorrow at 10am" |
| 106 | +classifier.train_ham "Please review the attached document" |
| 107 | + |
| 108 | +# Classify new text |
| 109 | +classifier.classify "Congratulations! You've won a prize!" |
| 110 | +# => "Spam" |
| 111 | +``` |
| 112 | + |
| 113 | +### Persistence with Madeleine |
20 | 114 |
|
21 | | -#### Ruby 3.4+ |
| 115 | +```ruby |
| 116 | +require 'classifier' |
| 117 | +require 'madeleine' |
22 | 118 |
|
23 | | -The released `gsl` gem doesn't support Ruby 3.4+. Install from source with the compatibility fix: |
| 119 | +m = SnapshotMadeleine.new("classifier_data") { |
| 120 | + Classifier::Bayes.new('Interesting', 'Uninteresting') |
| 121 | +} |
24 | 122 |
|
25 | | - # Install GSL library |
26 | | - brew install gsl # macOS |
27 | | - apt-get install libgsl-dev # Ubuntu/Debian |
| 123 | +m.system.train_interesting "fascinating article about science" |
| 124 | +m.system.train_uninteresting "boring repetitive content" |
| 125 | +m.take_snapshot |
28 | 126 |
|
29 | | - # Build and install the gem from the compatibility branch |
30 | | - git clone https://github.com/cardmagic/rb-gsl.git |
31 | | - cd rb-gsl |
32 | | - git checkout fix/ruby-3.4-compatibility |
33 | | - gem build gsl.gemspec |
34 | | - gem install gsl-*.gem |
| 127 | +# Later, restore and use: |
| 128 | +m.system.classify "new scientific discovery" |
| 129 | +# => "Interesting" |
| 130 | +``` |
35 | 131 |
|
36 | | -#### Ruby 3.3 and earlier |
| 132 | +### Learn More |
37 | 133 |
|
38 | | - # macOS |
39 | | - brew install gsl |
40 | | - gem install gsl |
| 134 | +- [Bayesian Filtering Explained](http://www.process.com/precisemail/bayesian_filtering.htm) |
| 135 | +- [Wikipedia: Bayesian Filtering](http://en.wikipedia.org/wiki/Bayesian_filtering) |
| 136 | +- [Paul Graham: A Plan for Spam](http://www.paulgraham.com/spam.html) |
41 | 137 |
|
42 | | - # Ubuntu/Debian |
43 | | - apt-get install libgsl-dev |
44 | | - gem install gsl |
| 138 | +## LSI (Latent Semantic Indexing) |
45 | 139 |
|
46 | | -LSI works without GSL using a pure Ruby implementation. When GSL is installed, Classifier automatically uses it with no configuration needed. |
| 140 | +Semantic analysis using Singular Value Decomposition (SVD). More flexible than Bayesian classifiers, providing search, clustering, and classification based on meaning rather than just keywords. |
47 | 141 |
|
48 | | -To suppress the GSL notice when not using it: |
| 142 | +### Quick Start |
49 | 143 |
|
50 | | - SUPPRESS_GSL_WARNING=true ruby your_script.rb |
| 144 | +```ruby |
| 145 | +require 'classifier' |
51 | 146 |
|
52 | | -## Bayes |
| 147 | +lsi = Classifier::LSI.new |
53 | 148 |
|
54 | | -A Bayesian classifier by Lucas Carlson. Bayesian Classifiers are accurate, fast, and have modest memory requirements. |
| 149 | +# Add documents with categories |
| 150 | +lsi.add_item "Dogs are loyal pets that love to play fetch", :pets |
| 151 | +lsi.add_item "Cats are independent and love to nap", :pets |
| 152 | +lsi.add_item "Ruby is a dynamic programming language", :programming |
| 153 | +lsi.add_item "Python is great for data science", :programming |
55 | 154 |
|
56 | | -### Usage |
| 155 | +# Classify new text |
| 156 | +lsi.classify "My puppy loves to run around" |
| 157 | +# => :pets |
57 | 158 |
|
58 | | - require 'classifier' |
59 | | - b = Classifier::Bayes.new 'Interesting', 'Uninteresting' |
60 | | - b.train_interesting "here are some good words. I hope you love them" |
61 | | - b.train_uninteresting "here are some bad words, I hate you" |
62 | | - b.classify "I hate bad words and you" # returns 'Uninteresting' |
| 159 | +# Get classification with confidence score |
| 160 | +lsi.classify_with_confidence "Learning to code in Ruby" |
| 161 | +# => [:programming, 0.89] |
| 162 | +``` |
63 | 163 |
|
64 | | - require 'madeleine' |
65 | | - m = SnapshotMadeleine.new("bayes_data") { |
66 | | - Classifier::Bayes.new 'Interesting', 'Uninteresting' |
67 | | - } |
68 | | - m.system.train_interesting "here are some good words. I hope you love them" |
69 | | - m.system.train_uninteresting "here are some bad words, I hate you" |
70 | | - m.take_snapshot |
71 | | - m.system.classify "I love you" # returns 'Interesting' |
| 164 | +### Search and Discovery |
72 | 165 |
|
73 | | -Using Madeleine, your application can persist the learned data over time. |
| 166 | +```ruby |
| 167 | +# Find similar documents |
| 168 | +lsi.find_related "Dogs are great companions", 2 |
| 169 | +# => ["Dogs are loyal pets that love to play fetch", "Cats are independent..."] |
74 | 170 |
|
75 | | -### Bayesian Classification |
| 171 | +# Search by keyword |
| 172 | +lsi.search "programming", 3 |
| 173 | +# => ["Ruby is a dynamic programming language", "Python is great for..."] |
| 174 | +``` |
76 | 175 |
|
77 | | -* http://www.process.com/precisemail/bayesian_filtering.htm |
78 | | -* http://en.wikipedia.org/wiki/Bayesian_filtering |
79 | | -* http://www.paulgraham.com/spam.html |
| 176 | +### Learn More |
80 | 177 |
|
81 | | -## LSI |
| 178 | +- [Wikipedia: Latent Semantic Analysis](http://en.wikipedia.org/wiki/Latent_semantic_analysis) |
| 179 | +- [C2 Wiki: Latent Semantic Indexing](http://www.c2.com/cgi/wiki?LatentSemanticIndexing) |
82 | 180 |
|
83 | | -A Latent Semantic Indexer by David Fayram. Latent Semantic Indexing engines |
84 | | -are not as fast or as small as Bayesian classifiers, but are more flexible, providing |
85 | | -fast search and clustering detection as well as semantic analysis of the text that |
86 | | -theoretically simulates human learning. |
| 181 | +## Performance |
87 | 182 |
|
88 | | -### Usage |
| 183 | +### GSL vs Native Ruby |
89 | 184 |
|
90 | | - require 'classifier' |
91 | | - lsi = Classifier::LSI.new |
92 | | - strings = [ ["This text deals with dogs. Dogs.", :dog], |
93 | | - ["This text involves dogs too. Dogs! ", :dog], |
94 | | - ["This text revolves around cats. Cats.", :cat], |
95 | | - ["This text also involves cats. Cats!", :cat], |
96 | | - ["This text involves birds. Birds.",:bird ]] |
97 | | - strings.each {|x| lsi.add_item x.first, x.last} |
| 185 | +GSL provides dramatic speedups for LSI operations, especially `build_index` (SVD computation): |
98 | 186 |
|
99 | | - lsi.search("dog", 3) |
100 | | - # returns => ["This text deals with dogs. Dogs.", "This text involves dogs too. Dogs! ", |
101 | | - # "This text also involves cats. Cats!"] |
| 187 | +| Documents | build_index | Overall | |
| 188 | +|-----------|-------------|---------| |
| 189 | +| 5 | 4x faster | 2.5x | |
| 190 | +| 10 | 24x faster | 5.5x | |
| 191 | +| 15 | 116x faster | 17x | |
102 | 192 |
|
103 | | - lsi.find_related(strings[2], 2) |
104 | | - # returns => ["This text revolves around cats. Cats.", "This text also involves cats. Cats!"] |
| 193 | +<details> |
| 194 | +<summary>Detailed benchmark (15 documents)</summary> |
105 | 195 |
|
106 | | - lsi.classify "This text is also about dogs!" |
107 | | - # returns => :dog |
| 196 | +``` |
| 197 | +Operation Native GSL Speedup |
| 198 | +---------------------------------------------------------- |
| 199 | +build_index 0.1412 0.0012 116.2x |
| 200 | +classify 0.0142 0.0049 2.9x |
| 201 | +search 0.0102 0.0026 3.9x |
| 202 | +find_related 0.0069 0.0016 4.2x |
| 203 | +---------------------------------------------------------- |
| 204 | +TOTAL 0.1725 0.0104 16.6x |
| 205 | +``` |
| 206 | +</details> |
108 | 207 |
|
109 | | - lsi.classify_with_confidence "This text is also about dogs!" |
110 | | - # returns => [:dog, 1.0] |
| 208 | +### Running Benchmarks |
111 | 209 |
|
112 | | -Please see the Classifier::LSI documentation for more information. It is possible to index, search and classify |
113 | | -with more than just simple strings. |
| 210 | +```bash |
| 211 | +rake benchmark # Run with current configuration |
| 212 | +rake benchmark:compare # Compare GSL vs native Ruby |
| 213 | +``` |
114 | 214 |
|
115 | | -### Latent Semantic Indexing |
| 215 | +## Development |
116 | 216 |
|
117 | | -* http://www.c2.com/cgi/wiki?LatentSemanticIndexing |
118 | | -* http://www.chadfowler.com/index.cgi/Computing/LatentSemanticIndexing.rdoc |
119 | | -* http://en.wikipedia.org/wiki/Latent_semantic_analysis |
| 217 | +### Setup |
120 | 218 |
|
121 | | -## Benchmarks |
| 219 | +```bash |
| 220 | +git clone https://github.com/cardmagic/classifier.git |
| 221 | +cd classifier |
| 222 | +bundle install |
| 223 | +``` |
122 | 224 |
|
123 | | -Run benchmarks to compare LSI performance: |
| 225 | +### Running Tests |
124 | 226 |
|
125 | | - rake benchmark # Run with current configuration |
126 | | - rake benchmark:compare # Compare GSL vs native Ruby |
| 227 | +```bash |
| 228 | +rake test # Run all tests |
| 229 | +ruby -Ilib test/bayes/bayesian_test.rb # Run specific test file |
127 | 230 |
|
128 | | -### GSL vs Native Ruby Comparison |
| 231 | +# Test without GSL (pure Ruby) |
| 232 | +NATIVE_VECTOR=true rake test |
| 233 | +``` |
129 | 234 |
|
130 | | -| Documents | build_index | Overall Speedup | |
131 | | -|-----------|-------------|-----------------| |
132 | | -| 5 | 4x | 2.5x | |
133 | | -| 10 | 24x | 5.5x | |
134 | | -| 15 | 116x | 17x | |
| 235 | +### Console |
135 | 236 |
|
136 | | -Sample comparison (15 documents): |
| 237 | +```bash |
| 238 | +rake console |
| 239 | +``` |
137 | 240 |
|
138 | | - Operation Native GSL Speedup |
139 | | - ---------------------------------------------------------- |
140 | | - build_index 0.1412 0.0012 116.2x |
141 | | - classify 0.0142 0.0049 2.9x |
142 | | - search 0.0102 0.0026 3.9x |
143 | | - find_related 0.0069 0.0016 4.2x |
144 | | - ---------------------------------------------------------- |
145 | | - TOTAL 0.1725 0.0104 16.6x |
| 241 | +## Contributing |
146 | 242 |
|
147 | | -The `build_index` operation (SVD computation) dominates total time and benefits most from GSL. Install GSL for production use with larger document sets. |
| 243 | +1. Fork the repository |
| 244 | +2. Create your feature branch (`git checkout -b feature/amazing-feature`) |
| 245 | +3. Commit your changes (`git commit -am 'Add amazing feature'`) |
| 246 | +4. Push to the branch (`git push origin feature/amazing-feature`) |
| 247 | +5. Open a Pull Request |
148 | 248 |
|
149 | 249 | ## Authors |
150 | 250 |
|
151 | | -* Lucas Carlson (lucas@rufy.com) |
152 | | -* David Fayram II (dfayram@gmail.com) |
153 | | -* Cameron McBride (cameron.mcbride@gmail.com) |
154 | | -* Ivan Acosta-Rubio (ivan@softwarecriollo.com) |
| 251 | +- **Lucas Carlson** - *Original author* - lucas@rufy.com |
| 252 | +- **David Fayram II** - *LSI implementation* - dfayram@gmail.com |
| 253 | +- **Cameron McBride** - cameron.mcbride@gmail.com |
| 254 | +- **Ivan Acosta-Rubio** - ivan@softwarecriollo.com |
155 | 255 |
|
156 | | -This library is released under the terms of the GNU LGPL. See LICENSE for more details. |
| 256 | +## License |
157 | 257 |
|
| 258 | +This library is released under the [GNU Lesser General Public License (LGPL) 2.1](LICENSE). |
0 commit comments