Skip to content

Commit 139a213

Browse files
committed
docs: modernize README to follow Ruby open source best practices
- Add badges for gem version, CI status, and license - Add table of contents for easier navigation - Add Ruby compatibility matrix (3.1-3.4) - Improve installation with Bundler instructions - Use collapsible sections for GSL install and benchmarks - Add syntax highlighting to all code blocks - Add Development section with setup and test instructions - Add Contributing guidelines - Improve examples (spam filter, persistence, LSI search) - Reorganize structure with Quick Start sections Closes #76
1 parent ba36830 commit 139a213

1 file changed

Lines changed: 207 additions & 106 deletions

File tree

README.md

Lines changed: 207 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,157 +1,258 @@
1-
## Welcome to Classifier
1+
# Classifier
22

3-
Classifier is a general module to allow Bayesian and other types of classifications.
3+
[![Gem Version](https://badge.fury.io/rb/classifier.svg)](https://badge.fury.io/rb/classifier)
4+
[![CI](https://github.com/cardmagic/classifier/actions/workflows/ruby.yml/badge.svg)](https://github.com/cardmagic/classifier/actions/workflows/ruby.yml)
5+
[![License: LGPL](https://img.shields.io/badge/License-LGPL_2.1-blue.svg)](https://opensource.org/licenses/LGPL-2.1)
46

5-
## Download
7+
A Ruby library for text classification using Bayesian and Latent Semantic Indexing (LSI) algorithms.
68

7-
* https://github.com/cardmagic/classifier
8-
* gem install classifier
9-
* git clone https://github.com/cardmagic/classifier.git
9+
## Table of Contents
1010

11-
## Dependencies
11+
- [Installation](#installation)
12+
- [Bayesian Classifier](#bayesian-classifier)
13+
- [LSI (Latent Semantic Indexing)](#lsi-latent-semantic-indexing)
14+
- [Performance](#performance)
15+
- [Development](#development)
16+
- [Contributing](#contributing)
17+
- [License](#license)
1218

13-
The `fast-stemmer` gem is required:
19+
## Installation
1420

15-
gem install fast-stemmer
21+
Add to your Gemfile:
22+
23+
```ruby
24+
gem 'classifier'
25+
```
26+
27+
Then run:
28+
29+
```bash
30+
bundle install
31+
```
32+
33+
Or install directly:
34+
35+
```bash
36+
gem install classifier
37+
```
1638

1739
### Optional: GSL for Faster LSI
1840

19-
For faster LSI classification, install the GNU Scientific Library and its Ruby bindings.
41+
For significantly faster LSI operations, install the [GNU Scientific Library](https://www.gnu.org/software/gsl/).
42+
43+
<details>
44+
<summary><strong>Ruby 3.4+</strong></summary>
45+
46+
The released `gsl` gem doesn't support Ruby 3.4+. Install from source:
47+
48+
```bash
49+
# Install GSL library
50+
brew install gsl # macOS
51+
apt-get install libgsl-dev # Ubuntu/Debian
52+
53+
# Build and install the gem
54+
git clone https://github.com/cardmagic/rb-gsl.git
55+
cd rb-gsl
56+
git checkout fix/ruby-3.4-compatibility
57+
gem build gsl.gemspec
58+
gem install gsl-*.gem
59+
```
60+
</details>
61+
62+
<details>
63+
<summary><strong>Ruby 3.3 and earlier</strong></summary>
64+
65+
```bash
66+
# macOS
67+
brew install gsl
68+
gem install gsl
69+
70+
# Ubuntu/Debian
71+
apt-get install libgsl-dev
72+
gem install gsl
73+
```
74+
</details>
75+
76+
When GSL is installed, Classifier automatically uses it. To suppress the GSL notice:
77+
78+
```bash
79+
SUPPRESS_GSL_WARNING=true ruby your_script.rb
80+
```
81+
82+
### Compatibility
83+
84+
| Ruby Version | Status |
85+
|--------------|--------|
86+
| 3.4 | Supported |
87+
| 3.3 | Supported |
88+
| 3.2 | Supported |
89+
| 3.1 | Supported |
90+
91+
## Bayesian Classifier
92+
93+
Fast, accurate classification with modest memory requirements. Ideal for spam filtering, sentiment analysis, and content categorization.
94+
95+
### Quick Start
96+
97+
```ruby
98+
require 'classifier'
99+
100+
classifier = Classifier::Bayes.new('Spam', 'Ham')
101+
102+
# Train the classifier
103+
classifier.train_spam "Buy cheap viagra now! Limited offer!"
104+
classifier.train_spam "You've won a million dollars! Claim now!"
105+
classifier.train_ham "Meeting scheduled for tomorrow at 10am"
106+
classifier.train_ham "Please review the attached document"
107+
108+
# Classify new text
109+
classifier.classify "Congratulations! You've won a prize!"
110+
# => "Spam"
111+
```
112+
113+
### Persistence with Madeleine
20114

21-
#### Ruby 3.4+
115+
```ruby
116+
require 'classifier'
117+
require 'madeleine'
22118

23-
The released `gsl` gem doesn't support Ruby 3.4+. Install from source with the compatibility fix:
119+
m = SnapshotMadeleine.new("classifier_data") {
120+
Classifier::Bayes.new('Interesting', 'Uninteresting')
121+
}
24122

25-
# Install GSL library
26-
brew install gsl # macOS
27-
apt-get install libgsl-dev # Ubuntu/Debian
123+
m.system.train_interesting "fascinating article about science"
124+
m.system.train_uninteresting "boring repetitive content"
125+
m.take_snapshot
28126

29-
# Build and install the gem from the compatibility branch
30-
git clone https://github.com/cardmagic/rb-gsl.git
31-
cd rb-gsl
32-
git checkout fix/ruby-3.4-compatibility
33-
gem build gsl.gemspec
34-
gem install gsl-*.gem
127+
# Later, restore and use:
128+
m.system.classify "new scientific discovery"
129+
# => "Interesting"
130+
```
35131

36-
#### Ruby 3.3 and earlier
132+
### Learn More
37133

38-
# macOS
39-
brew install gsl
40-
gem install gsl
134+
- [Bayesian Filtering Explained](http://www.process.com/precisemail/bayesian_filtering.htm)
135+
- [Wikipedia: Bayesian Filtering](http://en.wikipedia.org/wiki/Bayesian_filtering)
136+
- [Paul Graham: A Plan for Spam](http://www.paulgraham.com/spam.html)
41137

42-
# Ubuntu/Debian
43-
apt-get install libgsl-dev
44-
gem install gsl
138+
## LSI (Latent Semantic Indexing)
45139

46-
LSI works without GSL using a pure Ruby implementation. When GSL is installed, Classifier automatically uses it with no configuration needed.
140+
Semantic analysis using Singular Value Decomposition (SVD). More flexible than Bayesian classifiers, providing search, clustering, and classification based on meaning rather than just keywords.
47141

48-
To suppress the GSL notice when not using it:
142+
### Quick Start
49143

50-
SUPPRESS_GSL_WARNING=true ruby your_script.rb
144+
```ruby
145+
require 'classifier'
51146

52-
## Bayes
147+
lsi = Classifier::LSI.new
53148

54-
A Bayesian classifier by Lucas Carlson. Bayesian Classifiers are accurate, fast, and have modest memory requirements.
149+
# Add documents with categories
150+
lsi.add_item "Dogs are loyal pets that love to play fetch", :pets
151+
lsi.add_item "Cats are independent and love to nap", :pets
152+
lsi.add_item "Ruby is a dynamic programming language", :programming
153+
lsi.add_item "Python is great for data science", :programming
55154

56-
### Usage
155+
# Classify new text
156+
lsi.classify "My puppy loves to run around"
157+
# => :pets
57158

58-
require 'classifier'
59-
b = Classifier::Bayes.new 'Interesting', 'Uninteresting'
60-
b.train_interesting "here are some good words. I hope you love them"
61-
b.train_uninteresting "here are some bad words, I hate you"
62-
b.classify "I hate bad words and you" # returns 'Uninteresting'
159+
# Get classification with confidence score
160+
lsi.classify_with_confidence "Learning to code in Ruby"
161+
# => [:programming, 0.89]
162+
```
63163

64-
require 'madeleine'
65-
m = SnapshotMadeleine.new("bayes_data") {
66-
Classifier::Bayes.new 'Interesting', 'Uninteresting'
67-
}
68-
m.system.train_interesting "here are some good words. I hope you love them"
69-
m.system.train_uninteresting "here are some bad words, I hate you"
70-
m.take_snapshot
71-
m.system.classify "I love you" # returns 'Interesting'
164+
### Search and Discovery
72165

73-
Using Madeleine, your application can persist the learned data over time.
166+
```ruby
167+
# Find similar documents
168+
lsi.find_related "Dogs are great companions", 2
169+
# => ["Dogs are loyal pets that love to play fetch", "Cats are independent..."]
74170

75-
### Bayesian Classification
171+
# Search by keyword
172+
lsi.search "programming", 3
173+
# => ["Ruby is a dynamic programming language", "Python is great for..."]
174+
```
76175

77-
* http://www.process.com/precisemail/bayesian_filtering.htm
78-
* http://en.wikipedia.org/wiki/Bayesian_filtering
79-
* http://www.paulgraham.com/spam.html
176+
### Learn More
80177

81-
## LSI
178+
- [Wikipedia: Latent Semantic Analysis](http://en.wikipedia.org/wiki/Latent_semantic_analysis)
179+
- [C2 Wiki: Latent Semantic Indexing](http://www.c2.com/cgi/wiki?LatentSemanticIndexing)
82180

83-
A Latent Semantic Indexer by David Fayram. Latent Semantic Indexing engines
84-
are not as fast or as small as Bayesian classifiers, but are more flexible, providing
85-
fast search and clustering detection as well as semantic analysis of the text that
86-
theoretically simulates human learning.
181+
## Performance
87182

88-
### Usage
183+
### GSL vs Native Ruby
89184

90-
require 'classifier'
91-
lsi = Classifier::LSI.new
92-
strings = [ ["This text deals with dogs. Dogs.", :dog],
93-
["This text involves dogs too. Dogs! ", :dog],
94-
["This text revolves around cats. Cats.", :cat],
95-
["This text also involves cats. Cats!", :cat],
96-
["This text involves birds. Birds.",:bird ]]
97-
strings.each {|x| lsi.add_item x.first, x.last}
185+
GSL provides dramatic speedups for LSI operations, especially `build_index` (SVD computation):
98186

99-
lsi.search("dog", 3)
100-
# returns => ["This text deals with dogs. Dogs.", "This text involves dogs too. Dogs! ",
101-
# "This text also involves cats. Cats!"]
187+
| Documents | build_index | Overall |
188+
|-----------|-------------|---------|
189+
| 5 | 4x faster | 2.5x |
190+
| 10 | 24x faster | 5.5x |
191+
| 15 | 116x faster | 17x |
102192

103-
lsi.find_related(strings[2], 2)
104-
# returns => ["This text revolves around cats. Cats.", "This text also involves cats. Cats!"]
193+
<details>
194+
<summary>Detailed benchmark (15 documents)</summary>
105195

106-
lsi.classify "This text is also about dogs!"
107-
# returns => :dog
196+
```
197+
Operation Native GSL Speedup
198+
----------------------------------------------------------
199+
build_index 0.1412 0.0012 116.2x
200+
classify 0.0142 0.0049 2.9x
201+
search 0.0102 0.0026 3.9x
202+
find_related 0.0069 0.0016 4.2x
203+
----------------------------------------------------------
204+
TOTAL 0.1725 0.0104 16.6x
205+
```
206+
</details>
108207

109-
lsi.classify_with_confidence "This text is also about dogs!"
110-
# returns => [:dog, 1.0]
208+
### Running Benchmarks
111209

112-
Please see the Classifier::LSI documentation for more information. It is possible to index, search and classify
113-
with more than just simple strings.
210+
```bash
211+
rake benchmark # Run with current configuration
212+
rake benchmark:compare # Compare GSL vs native Ruby
213+
```
114214

115-
### Latent Semantic Indexing
215+
## Development
116216

117-
* http://www.c2.com/cgi/wiki?LatentSemanticIndexing
118-
* http://www.chadfowler.com/index.cgi/Computing/LatentSemanticIndexing.rdoc
119-
* http://en.wikipedia.org/wiki/Latent_semantic_analysis
217+
### Setup
120218

121-
## Benchmarks
219+
```bash
220+
git clone https://github.com/cardmagic/classifier.git
221+
cd classifier
222+
bundle install
223+
```
122224

123-
Run benchmarks to compare LSI performance:
225+
### Running Tests
124226

125-
rake benchmark # Run with current configuration
126-
rake benchmark:compare # Compare GSL vs native Ruby
227+
```bash
228+
rake test # Run all tests
229+
ruby -Ilib test/bayes/bayesian_test.rb # Run specific test file
127230

128-
### GSL vs Native Ruby Comparison
231+
# Test without GSL (pure Ruby)
232+
NATIVE_VECTOR=true rake test
233+
```
129234

130-
| Documents | build_index | Overall Speedup |
131-
|-----------|-------------|-----------------|
132-
| 5 | 4x | 2.5x |
133-
| 10 | 24x | 5.5x |
134-
| 15 | 116x | 17x |
235+
### Console
135236

136-
Sample comparison (15 documents):
237+
```bash
238+
rake console
239+
```
137240

138-
Operation Native GSL Speedup
139-
----------------------------------------------------------
140-
build_index 0.1412 0.0012 116.2x
141-
classify 0.0142 0.0049 2.9x
142-
search 0.0102 0.0026 3.9x
143-
find_related 0.0069 0.0016 4.2x
144-
----------------------------------------------------------
145-
TOTAL 0.1725 0.0104 16.6x
241+
## Contributing
146242

147-
The `build_index` operation (SVD computation) dominates total time and benefits most from GSL. Install GSL for production use with larger document sets.
243+
1. Fork the repository
244+
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
245+
3. Commit your changes (`git commit -am 'Add amazing feature'`)
246+
4. Push to the branch (`git push origin feature/amazing-feature`)
247+
5. Open a Pull Request
148248

149249
## Authors
150250

151-
* Lucas Carlson (lucas@rufy.com)
152-
* David Fayram II (dfayram@gmail.com)
153-
* Cameron McBride (cameron.mcbride@gmail.com)
154-
* Ivan Acosta-Rubio (ivan@softwarecriollo.com)
251+
- **Lucas Carlson** - *Original author* - lucas@rufy.com
252+
- **David Fayram II** - *LSI implementation* - dfayram@gmail.com
253+
- **Cameron McBride** - cameron.mcbride@gmail.com
254+
- **Ivan Acosta-Rubio** - ivan@softwarecriollo.com
155255

156-
This library is released under the terms of the GNU LGPL. See LICENSE for more details.
256+
## License
157257

258+
This library is released under the [GNU Lesser General Public License (LGPL) 2.1](LICENSE).

0 commit comments

Comments
 (0)