You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2e2f8ec
Browse filesBrowse the repository at this point in the historyBrowse files
*FuncFrog* is a tiny library for doing **fast**, **parallel**, **lazy**`Map`, `Filter`, `Reduce` and [a bunch of other things](#api-reference) over slices and other sequences — all chained into a single pipeline. You feed it a source (a slice, a `func`, a range...), stack a few operations on top, and pull the trigger with `Do()`.
10
+
*FuncFrog* is a powerful library for doing **fast**, **parallel**, **lazy**`Map`, `Filter`, `Reduce` and [a bunch of other things](#api-reference) over slices and other sequences -- all chained into a single pipeline. You feed it a source (a slice, a `func`, a range...), stack a few operations on top, and pull the trigger with `Do()`.
11
11
12
12
Everything runs in parallel with **minimal copying and locking**, and there's built-in [error handling](#error-handling-yeti--snag) that is honestly a bit weird but very handy once it clicks (hello `Yeet`/`Snag`).
13
13
@@ -89,12 +89,12 @@ Want to poke at real code? Jump straight to the [Examples](#examples).
89
89
90
90
The whole library revolves around two interfaces:
91
91
92
-
-**`Piper[T]`**— a *lazy* sequence whose **length is known** (a slice, a `Range`, a `Repeat`...). It can do everything.
93
-
-**`PiperNoLen[T]`**— a *lazy* sequence whose **length is unknown yet** (a `Func`, a `Cycle`, ...). It can do almost everything, plus it can *become* a `Piper` via [`Take` or `Gen`](#set-the-pipe-length-take--gen).
92
+
-**`Piper[T]`**-- a *lazy* sequence whose **length is known** (a slice, a `Range`, a `Repeat`...). It can do everything.
93
+
-**`PiperNoLen[T]`**-- a *lazy* sequence whose **length is unknown yet** (a `Func`, a `Cycle`, ...). It can do almost everything, plus it can *become* a `Piper` via [`Take` or `Gen`](#set-the-pipe-length-take--gen).
94
94
95
95
Both are lazy: nothing actually runs until you call a terminal method (`Do`, `Reduce`, `Sum`, `Count`, `First`, `Any`...). Until then you're just describing the pipeline.
96
96
97
-
Because a pipe is a value, you can copy it around freely — assign it to a variable, reuse it, branch it:
97
+
Because a pipe is a value, you can copy it around freely -- assign it to a variable, reuse it, branch it:
Operations can change the element type too, but two flavors exist:
106
-
-`Pipe.Map` keeps the **same** type (`func(T) T`) — cheap and easy.
107
-
-`pipe.Map` (prefix) / `ff.Map` change the **type** (`func(SrcT) DstT`) — see [Prefix helpers](#prefix-helpers).
106
+
-`Pipe.Map` keeps the **same** type (`func(T) T`) -- cheap and easy.
107
+
-`pipe.Map` (prefix) / `ff.Map` change the **type** (`func(SrcT) DstT`) -- see [Prefix helpers](#prefix-helpers).
108
108
109
109
## API reference
110
110
111
-
Every bullet below is a real signature — copy/paste safe.
111
+
Every bullet below is a real signature -- copy/paste safe.
112
112
113
113
### Constructors
114
114
115
-
-:frog:`Slice([]T) Piper[T]`— wraps a slice. *Length known.*
116
-
-:frog:`Func(fn func(i int) (T, bool)) PiperNoLen[T]`— builds a lazy sequence from a function of the index `i`. The bool says whether the value exists (`true`) or should be skipped (`false`). *Length unknown.*
117
-
-:frog:`Fn(fn func(i int) T) PiperNoLen[T]`— like `Func`, but nothing is ever skipped. *Length unknown.*
118
-
-:frog:`FuncP(fn func(i int) (*T, bool)) PiperNoLen[T]`— like `Func`, but your function hands back pointers (handy for big structs). *Length unknown.*
119
-
-:frog:`Cycle(a []T) PiperNoLen[T]`— loops over the slice forever. *Length unknown.*
120
-
-:frog:`Range[T Integer|Float](start, finish, step T) Piper[T]`— generates `start, start+step, ...` and stops **before**`finish` (i.e. `[start, finish)`). Works for any integer or float. *Length known.*
121
-
-:frog:`Repeat(x T, n int) Piper[T]`—`x` repeated `n` times. A non-positive `n` gives an empty pipe. *Length known.*
115
+
-:frog:`Slice([]T) Piper[T]`-- wraps a slice. *Length known.*
116
+
-:frog:`Func(fn func(i int) (T, bool)) PiperNoLen[T]`-- builds a lazy sequence from a function of the index `i`. The bool says whether the value exists (`true`) or should be skipped (`false`). *Length unknown.*
117
+
-:frog:`Fn(fn func(i int) T) PiperNoLen[T]`-- like `Func`, but nothing is ever skipped. *Length unknown.*
118
+
-:frog:`FuncP(fn func(i int) (*T, bool)) PiperNoLen[T]`-- like `Func`, but your function hands back pointers (handy for big structs). *Length unknown.*
119
+
-:frog:`Cycle(a []T) PiperNoLen[T]`-- loops over the slice forever. *Length unknown.*
120
+
-:frog:`Range[T Integer|Float](start, finish, step T) Piper[T]`-- generates `start, start+step, ...` and stops **before**`finish` (i.e. `[start, finish)`). Works for any integer or float. *Length known.*
121
+
-:frog:`Repeat(x T, n int) Piper[T]`--`x` repeated `n` times. A non-positive `n` gives an empty pipe. *Length known.*
122
122
123
123
### Set the pipe length: Take & Gen
124
124
125
125
These two only exist on `PiperNoLen` and are how you turn an unknown-length pipe into a `Piper`. Only the **first**`Take`/`Gen` in a chain counts.
126
126
127
-
-:frog:`Take(n int) Piper[T]`— "I want `n` values in the end". FuncFrog keeps evaluating until it has collected `n`*surviving* values. *Length becomes known.*
128
-
-:frog:`Gen(n int) Piper[T]`— "evaluate the first `n` positions". Note the difference: it evaluates positions `[0, n)` and your `Filter` can still drop some, so the result may be shorter than `n`. *Length becomes known.*
127
+
-:frog:`Take(n int) Piper[T]`-- "I want `n` values in the end". FuncFrog keeps evaluating until it has collected `n`*surviving* values. *Length becomes known.*
128
+
-:frog:`Gen(n int) Piper[T]`-- "evaluate the first `n` positions". Note the difference: it evaluates positions `[0, n)` and your `Filter` can still drop some, so the result may be shorter than `n`. *Length becomes known.*
129
129
130
130
### Go parallel
131
131
132
-
-:frog:`Parallel(n uint16) Piper[T]` (and `PiperNoLen[T]`) — split the work across `n` goroutines. Default is `1`. Only the **first**`Parallel` in a chain is applied.
132
+
-:frog:`Parallel(n uint16) Piper[T]` (and `PiperNoLen[T]`) -- split the work across `n` goroutines. Default is `1`. Only the **first**`Parallel` in a chain is applied.
133
133
134
134
### Transforming data
135
135
136
-
-:frog:`Map(fn func(T) T) Piper[T]`— apply `fn` to every element (same type in, same type out).
137
-
-:frog:`Filter(fn func(*T) bool) Piper[T]`— keep only the elements for which `fn` returns `true`. (**Tip:** the argument is a `*T`, so remember to dereference —`*x`.)
138
-
-:frog:`MapFilter(fn func(T) (T, bool)) Piper[T]`— map and filter in one pass. If the returned bool is `false`, the value is dropped. Great for "skip this one" cases, [including error handling](#simple-error-handling).
139
-
-:frog:`Sort(less func(*T, *T) bool) Piper[T]`— sort the current values. **Requires a known length**, so `Gen`/`Slice`/`Range`/`Repeat` first.
140
-
-:frog:`Reduce(fn func(*T, *T) T) *T`— fold the whole pipe into a single value, left to right. Returns `nil` for an empty pipe. (*Postfix* reduce: type stays the same.)
141
-
-:frog:`Sum(plus func(*T, *T) T) T`— like `Reduce` but **parallel-friendly**, so it wants an associative function (`+`, `*`, `min`, `max`...).
136
+
-:frog:`Map(fn func(T) T) Piper[T]`-- apply `fn` to every element (same type in, same type out).
137
+
-:frog:`Filter(fn func(*T) bool) Piper[T]`-- keep only the elements for which `fn` returns `true`. (**Tip:** the argument is a `*T`, so remember to dereference --`*x`.)
138
+
-:frog:`MapFilter(fn func(T) (T, bool)) Piper[T]`-- map and filter in one pass. If the returned bool is `false`, the value is dropped. Great for "skip this one" cases, [including error handling](#simple-error-handling).
139
+
-:frog:`Sort(less func(*T, *T) bool) Piper[T]`-- sort the current values. **Requires a known length**, so `Gen`/`Slice`/`Range`/`Repeat` first.
140
+
-:frog:`Reduce(fn func(*T, *T) T) *T`-- fold the whole pipe into a single value, left to right. Returns `nil` for an empty pipe. (*Postfix* reduce: type stays the same.)
141
+
-:frog:`Sum(plus func(*T, *T) T) T`-- like `Reduce` but **parallel-friendly**, so it wants an associative function (`+`, `*`, `min`, `max`...).
142
142
143
143
### Grabbing a single value
144
144
145
-
-:frog:`First() *T`— the first element, or `nil` if there is nothing (yes, it's a pointer).
146
-
-:frog:`Any() *T`—*some* (effectively random under parallelism) element, or `nil` if empty.
147
-
-:frog:`Count() int`— how many elements the pipe would produce. It still runs the pipeline, but skips building the result slice — so it's the memory-light way to ask "how many?".
145
+
-:frog:`First() *T`-- the first element, or `nil` if there is nothing (yes, it's a pointer).
146
+
-:frog:`Any() *T`--*some* (effectively random under parallelism) element, or `nil` if empty.
147
+
-:frog:`Count() int`-- how many elements the pipe would produce. It still runs the pipeline, but skips building the result slice -- so it's the memory-light way to ask "how many?".
148
148
149
149
### Running the pipe: Do & Promices
150
150
151
-
-:frog:`Do() []T`—**execute** everything and return the resulting slice. This is the usual "end of pipeline" call.
152
-
-:frog:`Promices() []func() (T, bool)`— instead of running now, get one lazy thunk per position. Each thunk returns `(value, ok)`; `ok == true` means "this value exists, append it". Handy if you want to schedule the work yourself. (The funny spelling is intentional — they're "promices" :)
151
+
-:frog:`Do() []T`--**execute** everything and return the resulting slice. This is the usual "end of pipeline" call.
152
+
-:frog:`Promices() []func() (T, bool)`-- instead of running now, get one lazy thunk per position. Each thunk returns `(value, ok)`; `ok == true` means "this value exists, append it". Handy if you want to schedule the work yourself. (The funny spelling is intentional -- they're "promices" :)
153
153
154
154
### Erase & Collect
155
155
156
156
Sometimes your pipeline needs to lug around different types at different steps. `Erase` gives up the compile-time type so you can work with `any`, and `Collect` puts a type back on.
157
157
158
-
-:frog:`Erase() Piper[any]` (or `PiperNoLen[any]`) — every element becomes `any` (backed by a pointer to the original value).
159
-
-:frog:`pipe.Collect[DstT](Piper[any]) Piper[DstT]`— pull a `Piper[any]` back into a concrete `Piper[DstT]`.
160
-
-:frog:`pipe.CollectNL[DstT](PiperNoLen[any]) PiperNoLen[DstT]`— the same for a pipe with no known length.
158
+
-:frog:`Erase() Piper[any]` (or `PiperNoLen[any]`) -- every element becomes `any` (backed by a pointer to the original value).
159
+
-:frog:`pipe.Collect[DstT](Piper[any]) Piper[DstT]`-- pull a `Piper[any]` back into a concrete `Piper[DstT]`.
160
+
-:frog:`pipe.CollectNL[DstT](PiperNoLen[any]) PiperNoLen[DstT]`-- the same for a pipe with no known length.
161
161
162
-
A gotcha worth knowing: `Collect[DstT]` asserts each stored `any` to `*DstT`. So whatever you put into an erased pipe must be a **pointer** to the type you're collecting back — an `Erase()` of `Piper[int]` hands you `any` holding a `*int`, so you'd `Collect[int]`.
162
+
A gotcha worth knowing: `Collect[DstT]` asserts each stored `any` to `*DstT`. So whatever you put into an erased pipe must be a **pointer** to the type you're collecting back -- an `Erase()` of `Piper[int]` hands you `any` holding a `*int`, so you'd `Collect[int]`.
163
163
164
164
### Error handling: Yeti & Snag
165
165
166
166
This part is a little unusual, so read slowly.
167
167
168
-
-:frog:`Yeti(y internalpipe.YeetSnag) Piper[T]`— register a *yeti* on the pipe. A yeti is a little error collector you create with `pipe.NewYeti()`.
169
-
-:frog:`NewYeti() internalpipe.YeetSnag`— makes a yeti. It has `y.Yeet(err)` to throw an error into it.
170
-
-:frog:`Snag(h func(error)) Piper[T]`— attach a handler that receives whatever got `Yeet`-ed into the **last** yeti that was set via `Yeti`.
168
+
-:frog:`Yeti(y internalpipe.YeetSnag) Piper[T]`-- register a *yeti* on the pipe. A yeti is a little error collector you create with `pipe.NewYeti()`.
169
+
-:frog:`NewYeti() internalpipe.YeetSnag`-- makes a yeti. It has `y.Yeet(err)` to throw an error into it.
170
+
-:frog:`Snag(h func(error)) Piper[T]`-- attach a handler that receives whatever got `Yeet`-ed into the **last** yeti that was set via `Yeti`.
171
171
172
-
In practice: create a yeti, register it with `Yeti(y)`**before** you start yeeting at it, yeet errors from inside your functions, and let `Snag` deal with them at the end. See the [examples](#simple-error-handling)— they make it click way faster than this paragraph.
172
+
In practice: create a yeti, register it with `Yeti(y)`**before** you start yeeting at it, yeet errors from inside your functions, and let `Snag` deal with them at the end. See the [examples](#simple-error-handling)-- they make it click way faster than this paragraph.
173
173
174
174
### Ready-made helpers: pipies
175
175
176
176
Package `pipies` has the little functions you'd otherwise rewrite every time:
177
177
178
-
-:frog:`Sum[T]`— a reducer: `func(a, b *T) T { return *a + *b }`. Works for numbers and `~string`.
179
-
-:frog:`Less[T]`— the everyday comparator for `Sort`.
178
+
-:frog:`Sum[T]`-- a reducer: `func(a, b *T) T { return *a + *b }`. Works for numbers and `~string`.
179
+
-:frog:`Less[T]`-- the everyday comparator for `Sort`.
-:frog:**Negation:**`Not(fn)`, `Nott(fn)`, `Nottt(fn)`— flip a boolean-returning function with 1, 2, or 3 args (hence the extra t's).
182
+
-:frog:**Negation:**`Not(fn)`, `Nott(fn)`, `Nottt(fn)`-- flip a boolean-returning function with 1, 2, or 3 args (hence the extra t's).
183
183
184
184
### Prefix helpers
185
185
186
186
`Erase`/`Collect` work, but all those pointer conversions are annoying. The prefix functions in `pkg/pipe` are the pleasant way to change a pipe's type: pass the pipe first, the function second.
187
187
188
-
-:frog:`pipe.Map(Piper[SrcT], func(SrcT) DstT) Piper[DstT]`— map + change type, known length.
-:frog:`pipe.Reduce(Piper[SrcT], func(*DstT, *SrcT) DstT, initVal ...DstT) DstT`— prefix reduce that can also change type. Mind the argument order: the accumulator (`*DstT`) comes first, then the current element (`*SrcT`). `initVal` is optional and seeds the fold.
188
+
-:frog:`pipe.Map(Piper[SrcT], func(SrcT) DstT) Piper[DstT]`-- map + change type, known length.
-:frog:`pipe.Reduce(Piper[SrcT], func(*DstT, *SrcT) DstT, initVal ...DstT) DstT`-- prefix reduce that can also change type. Mind the argument order: the accumulator (`*DstT`) comes first, then the current element (`*SrcT`). `initVal` is optional and seeds the fold.
193
193
194
194
### The ff shortcuts
195
195
@@ -203,12 +203,12 @@ When a whole pipe is overkill and you just want to run one function over a slice
203
203
204
204
### Still cooking (TBD)
205
205
206
-
These aren't in yet — they're on the wishlist:
206
+
These aren't in yet -- they're on the wishlist:
207
207
208
-
-:seedling:`Until(fn func(*T) bool)`— evaluate one-by-one until `fn` says stop. (Single-threaded only by nature.)
209
-
-:seedling:`IsAny() bool`— does the pipe have *any* element?
210
-
-:seedling:`MoreThan(n int) bool`— does it have *more than*`n`?
211
-
-:seedling:`Reverse() Pipe`— reverse the underlying slice.
208
+
-:seedling:`Until(fn func(*T) bool)`-- evaluate one-by-one until `fn` says stop. (Single-threaded only by nature.)
209
+
-:seedling:`IsAny() bool`-- does the pipe have *any* element?
210
+
-:seedling:`MoreThan(n int) bool`-- does it have *more than*`n`?
211
+
-:seedling:`Reverse() Pipe`-- reverse the underlying slice.
212
212
213
213
## Examples
214
214
@@ -252,7 +252,7 @@ p := pipe.Func(func(i int) (v int, b bool) {
252
252
253
253
### Take vs Gen
254
254
255
-
**`Gen(n)`** evaluates the first `n` positions, then the filters get their say — so you can end up with **fewer than `n`** elements:
255
+
**`Gen(n)`** evaluates the first `n` positions, then the filters get their say -- so you can end up with **fewer than `n`** elements:
256
256
257
257
```go
258
258
p:= pipe.Func(func(iint) (vint, bbool) {
@@ -264,7 +264,7 @@ p := pipe.Func(func(i int) (v int, b bool) {
264
264
// p will be [0, 2, 4, 6, 8]
265
265
```
266
266
267
-
All ten positions `0..9` are evaluated, but only the evens survive — 5 values, not 10.
267
+
All ten positions `0..9` are evaluated, but only the evens survive - 5 values, not 10.
268
268
269
269
**`Take(n)`** keeps going until `n` values actually make it through the filter:
270
270
@@ -281,7 +281,7 @@ p := pipe.Func(func(i int) (v int, b bool) {
281
281
And here's the trap: if `Take` asks for more values than your source can ever produce, it will spin all the way up to `math.MaxInt` and then panic.
282
282
283
283
```go
284
-
// DO NOT DO THIS — it will grind until math.MaxInt and then panic
284
+
// DO NOT DO THIS - it will grind until math.MaxInt and then panic
285
285
p:= pipe.Func(func(iint) (vint, bbool) {
286
286
return i, i < 10// only the first 10 values survive
287
287
}).
@@ -364,7 +364,7 @@ Check the `pipies` package for more handy predicates and comparators.
364
364
365
365
### An infinite sequence
366
366
367
-
`Func` can be stateful, so you can describe a sequence that never ends — here's a lazy Fibonacci generator. Each value is stashed in a one-slot channel so the next call can find it. To keep the order sane, let it run on a single goroutine (the default) and cap it with `Take`:
367
+
`Func` can be stateful, so you can describe a sequence that never ends -- here's a lazy Fibonacci generator. Each value is stashed in a one-slot channel so the next call can find it. To keep the order sane, let it run on a single goroutine (the default) and cap it with `Take`:
368
368
369
369
```go
370
370
fib:=make([]chanint, 60)
@@ -482,14 +482,14 @@ This builds `256/i` for `i` from `-10` to `9`. The `0` would explode, so we `Yee
482
482
483
483
The flow is:
484
484
485
-
1.`pipe.NewYeti()`— make an error collector.
486
-
2.`Yeti(y)`— register it on the pipe. **Do this before yeeting**, or nothing will catch your error.
487
-
3.`y.Yeet(err)`— throw an error from inside your function.
488
-
4.`Snag(handler)`— attach the function that deals with whatever was yeeted. You can stack several `Snag`s; they all go to the **last** yeti registered with `Yeti`.
485
+
1.`pipe.NewYeti()`-- make an error collector.
486
+
2.`Yeti(y)`-- register it on the pipe. **Do this before yeeting**, or nothing will catch your error.
487
+
3.`y.Yeet(err)`-- throw an error from inside your function.
488
+
4.`Snag(handler)`-- attach the function that deals with whatever was yeeted. You can stack several `Snag`s; they all go to the **last** yeti registered with `Yeti`.
489
489
490
490
### Multiple error handling
491
491
492
-
Different stages of a pipeline can have their own yetis and handlers — nice for a real user-enrichment flow:
492
+
Different stages of a pipeline can have their own yetis and handlers -- nice for a real user-enrichment flow:
493
493
494
494
```go
495
495
y1, y2:= pipe.NewYeti(), pipe.NewYeti()
@@ -531,7 +531,7 @@ It compares FuncFrog against plain `for` loops for `Map`, `Filter`, `Reduce`, `A
531
531
532
532
## Is this stable?
533
533
534
-
Yep — **stable since v1.0.0**. Everything listed above is covered by unit tests. The stuff under [Still cooking](#still-cooking-tbd) will land as described and get tests before it's called stable.
534
+
Yep! **Stable since v1.0.0**. Everything listed above is covered by unit tests. The stuff under [Still cooking](#still-cooking-tbd) will land as described and get tests before it's called stable.
535
535
536
536
If I ever change a method signature, the **major version goes up**.
537
537
@@ -541,6 +541,6 @@ PRs are welcome, especially for the [TBD](#still-cooking-tbd) features. Same goe
541
541
542
542
## What's next?
543
543
544
-
A proper roadmap is coming. Until then — fork it, play with it, and enjoy.
544
+
A proper roadmap is coming. Until then -- fork it, play with it, and enjoy.
0 commit comments