Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
89eed55
v2.4.0 (#327)
singaraiona Jul 11, 2026
eb66eed
feat(query): support live inserts into parted tables
singaraiona Jul 14, 2026
bbdc419
fix(core): restore total-core -c semantics
singaraiona Jul 15, 2026
9961388
fix(parse) Fix nonstring if not defined
protocolstardust Jul 16, 2026
6cac6e8
Merge pull request #330 from protocolstardust/dev
protocolstardust Jul 16, 2026
f44ab81
Merge branch 'master' into dev
singaraiona Jul 16, 2026
49d8524
fix(store): surface FlushFileBuffers failure in journal SYNC mode (#335)
belowzeroff Jul 18, 2026
cb1c6cd
fix(hnsw): reject build dims whose vector count overflows size_t (#333)
belowzeroff Jul 18, 2026
7475949
fix(store): read full link sidecar to avoid wrong-symbol truncation (…
belowzeroff Jul 18, 2026
ebadd07
fix(hnsw): reject index files whose vector count overflows size_t (#332)
belowzeroff Jul 21, 2026
fa39d9d
fix(docs): remediate F-0001 F-0005 F-0007
singaraiona Jul 22, 2026
ffd784f
fix(docs): remediate CF-0001
singaraiona Jul 22, 2026
1f27e61
fix(docs): remediate CF-0002
singaraiona Jul 22, 2026
465e58e
chore(audit): plan CF-0003 ratification
singaraiona Jul 22, 2026
8f8033c
fix(docs): remediate CF-0003
singaraiona Jul 22, 2026
3ae79f5
feat(docs): redesign website and documentation
singaraiona Jul 22, 2026
5dc8265
Merge branch 'master' into dev
singaraiona Jul 22, 2026
4d2c8f3
fix(null): avoid f64 null casts to integers (#340)
belowzeroff Jul 23, 2026
4e49626
fix(expr): avoid null truthiness casts in fallback binary ops (#339)
belowzeroff Jul 23, 2026
a05886d
ci: make Rayforce audit PR comments best-effort
singaraiona Jul 23, 2026
6ca3d8a
ci: publish Rayforce audit comments from trusted workflow
singaraiona Jul 23, 2026
04d8f3d
ci: resolve fork PRs for audit commenter
singaraiona Jul 23, 2026
6acdc2c
perf: parallelize serial stages around group-by; unify binary-agg nul…
ser-vasilich Jul 23, 2026
a53d98e
Merge branch 'master' into dev
singaraiona Jul 23, 2026
260e018
fix(aggr): preserve slice nullability in binary groups
singaraiona Jul 23, 2026
492c5db
fix(expr): avoid f64 null cast in fallback idiv integer output (#344)
belowzeroff Jul 23, 2026
d250108
fix(group): avoid f64 null read cast in dense aggs (#343)
belowzeroff Jul 23, 2026
95f0a3d
fix(ipc): preserve boxed data list args (#346)
belowzeroff Jul 24, 2026
e10d1ab
fix(group): avoid f64 null cast in DA reads (#348)
belowzeroff Jul 24, 2026
842acc0
ci: use portable march for fuzz jobs (#347)
belowzeroff Jul 24, 2026
84cea17
chore(release): merge master into dev
singaraiona Jul 24, 2026
7a5839e
fix(pivot): preserve generic missing cells as null (#350)
belowzeroff Jul 27, 2026
57628fb
fix(xbar): avoid narrow bucket truncation (#351)
belowzeroff Jul 28, 2026
d7a1f6f
Merge branch 'master' into dev
singaraiona Jul 28, 2026
c501ae0
fix(arith): reject float temporal operands (#353)
belowzeroff Jul 29, 2026
aea9bda
fix(ops): make nested (LIST) columns usable through a parted view (fi…
singaraiona Jul 30, 2026
0605f1a
fix(query): preserve temporal arithmetic semantics (#354)
belowzeroff Jul 30, 2026
282d3a1
Merge branch 'master' into dev
singaraiona Jul 30, 2026
17bf2b8
ci: skip audit comments for cancelled runs
singaraiona Jul 30, 2026
05c8078
fix(query): preserve if temporal branch types (#359)
belowzeroff Jul 31, 2026
488a296
fix(store): reject duplicate splayed column names (#360)
belowzeroff Aug 3, 2026
5684559
fix(builtins): reject malformed strings in TIMESTAMP cast (#361)
belowzeroff Aug 3, 2026
c9b9a66
fix(store): support nested column inserts and persistence (#365)
singaraiona Aug 3, 2026
4c0fa46
fix(eval): enforce restricted mode in compiled lambdas (#366)
singaraiona Aug 3, 2026
8c9300b
feat(core): add bounded poll step for embedders (#367)
singaraiona Aug 3, 2026
3c79632
Merge branch 'master' into dev
singaraiona Aug 3, 2026
9b05f60
fix(test): rely on public runtime declarations (#369)
belowzeroff Aug 4, 2026
add3498
feat: website add consumer
singaraiona Aug 4, 2026
9d255c2
Merge branch 'master' into dev
singaraiona Aug 4, 2026
a1dcfd9
feat(core): expose restricted poll mode (#372)
singaraiona Aug 4, 2026
d5d5240
fix(aggr): preserve grouped nested first and last
singaraiona Aug 4, 2026
c35fa2b
Merge branch 'master' into dev
singaraiona Aug 4, 2026
2cfc70e
fix(join): preserve nested columns (#376)
singaraiona Aug 4, 2026
ac9d0cc
Merge branch 'master' into dev
singaraiona Aug 4, 2026
2fa0fbd
fix(csv): round-trip signed and >=24h TIME values in .csv.read (#379)
belowzeroff Aug 4, 2026
959db02
fix(str): propagate null start/length in substr instead of overflowin…
belowzeroff Aug 4, 2026
2083f24
fix(builtins): clamp out-of-range float in scalar numeric casts (#380)
belowzeroff Aug 5, 2026
5419053
Merge branch 'master' into dev
singaraiona Aug 5, 2026
c731b42
fix(builtins): validate bounds in DATE string cast (#382)
belowzeroff Aug 6, 2026
93db5ce
perf(aggr): specialize reduction scans (#383)
singaraiona Aug 6, 2026
e71798a
perf(collection): parallel radix distinct for fixed-width columns (#384)
protocolstardust Aug 6, 2026
8d6466f
fix(csv): harden import cancellation and schema handling (#387)
singaraiona Aug 7, 2026
8a1cf0c
fix(temporal): guard int64 overflow in DATE/TIMESTAMP decomposition (…
belowzeroff Aug 9, 2026
7bf522a
fix(eval): materialize lazy values at compiled boundaries (#391)
singaraiona Aug 11, 2026
caa529d
fix(arith): wrap scalar integer add/sub/mul on overflow instead of UB…
belowzeroff Aug 11, 2026
a2607ac
fix(builtins): reject malformed strings in TIME cast (#389)
belowzeroff Aug 11, 2026
a14ba5c
Merge branch 'master' into dev
singaraiona Aug 11, 2026
5aa47a5
fix(builtins): validate hex digits and dash layout in GUID string cas…
belowzeroff Aug 12, 2026
d1d256e
feat(core): address Data Vault workflow gaps (#395)
singaraiona Aug 12, 2026
f1d1b71
feat(io): add binary file read and write (#396)
singaraiona Aug 12, 2026
9a4ea60
Merge branch 'master' into dev
singaraiona Aug 12, 2026
776b290
fix(store): repair legacy string hash caches (#398)
singaraiona Aug 12, 2026
afbcf69
fix(builtins): reject trailing garbage and overflow in numeric string…
belowzeroff Aug 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions docs/docs/guides/analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,24 +231,28 @@ The key list `[Sym Ts]` specifies equality columns followed by the time column (
[12:00:00 12:00:01 12:00:02 12:00:03 12:00:04 12:00:05 12:00:06 12:00:07 12:00:08 12:00:09]
[928 528 648 914 918 626 577 817 620 698])))

; Simple window join: match on [Sym], join by Time
(window-join trades quotes [Sym] 'Time)
; TIME offsets are milliseconds: aggregate one second before through one after.
(set intervals (map-left + [-1000 1000] (at trades 'Time)))
(window-join [Sym Time] intervals trades quotes
{total_size: (sum Size)})
```

```text
┌──────┬──────────────┬───────┬───────┐
│ Sym │ Time │ Price │ Size
│ SYM │ TIME │ F64 │ I64 │
├──────┼──────────────┼───────┼───────┤
│ AAPL │ 12:00:01.000 │ 89.17 │ 528
│ AAPL │ 12:00:04.000 │ 70.5 │ 918
│ AAPL │ 12:00:06.000 │ 80.54 │ 577
├──────┴──────────────┴───────┴───────┤
│ 3 rows (3 shown) 4 columns (4 shown)│
└─────────────────────────────────────┘
┌──────┬──────────────┬───────┬────────────
│ Sym │ Time │ Price │ total_size
│ SYM │ TIME │ F64 │ I64
├──────┼──────────────┼───────┼────────────
│ AAPL │ 12:00:01.000 │ 89.17 │ 2104
│ AAPL │ 12:00:04.000 │ 70.5 │ 2458
│ AAPL │ 12:00:06.000 │ 80.54 │ 2020
├──────┴──────────────┴───────┴────────────
│ 3 rows (3 shown) 4 columns (4 shown)
└──────────────────────────────────────────
```

The `window-join` function matches rows from the right table within a time window around each left row. The equality keys (`[Sym]`) filter candidates, and the time column (`'Time`) determines the window range.
The last key in `[Sym Time]` is temporal; preceding keys are equality
partitions. `intervals` supplies a lower and upper offset for every left row,
and the final dictionary names the aggregations applied to matching right rows.

## 7. Working with CSV { #csv-workflow }

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/guides/ipc.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,14 @@ The server evaluates any valid Rayfall expression sent as a string. This means y
(.ipc.send h "(select {from: trades total: (sum qty)})")

;; Volume-weighted average price by symbol
(.ipc.send h "(select {from: trades by: sym vwap: (% (sum (* price qty)) (sum qty))})")
(.ipc.send h "(select {from: trades by: sym vwap: (/ (sum (* price qty)) (sum qty))})")
```

### Joins

```text
;; Join trades with quotes on sym
(.ipc.send h "(select {from: (left-join trades quotes 'sym)})")
(.ipc.send h "(select {from: (left-join [sym] trades quotes)})")
```

!!! note "Tip"
Expand Down
11 changes: 10 additions & 1 deletion docs/docs/language/control-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ If no error is raised, `try` returns the result of the body expression normally.

### Fallback value

If the second argument is **not** a function, it is returned as-is as the fallback value on error (evaluated only when the body fails). Because lambdas do not capture closures, this is the only way to surface an outer binding from the failure branch:
If the second argument is **not** a function, it is returned as-is as the fallback value on error (evaluated only when the body fails):

```lisp
‣ (try (raise "boom") 0)
Expand All @@ -93,6 +93,15 @@ If the second argument is **not** a function, it is returned as-is as the fallba

A handler must accept the single error argument, so only a lambda or a unary builtin is *called* with the error; any other value (including a multi-argument builtin) is treated as a fallback value.

Nested lambdas capture visible lexical bindings when they are created:

```lisp
‣ (set make-adder (fn [x] (fn [y] (+ x y))))
‣ (set add7 (make-adder 7))
‣ (add7 5)
12
```

## Early Return: return

`return` exits the innermost enclosing compiled lambda early with the given value:
Expand Down
27 changes: 17 additions & 10 deletions docs/docs/language/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ Aggregation functions are marked **aggr** and reduce vectors to scalar values. U
| `any` | unary, aggr | True if any non-null numeric element is truthy; empty/all-null returns `false` | `(any [0 0 3])``true` |
| `count` | unary, aggr | Count of elements | `(count [1 2 3])``3` |
| `avg` | unary, aggr | Arithmetic mean | `(avg [1 2 3])``2.0` |
| `min` | unary, aggr | Minimum value | `(min [3 1 2])``1` |
| `max` | unary, aggr | Maximum value | `(max [3 1 2])``3` |
| `min` | unary aggr, or binary atomic | Reduce one vector, or choose the element-wise minimum of two values | `(min [3 9 1] [2 10 4])``[2 9 1]` |
| `max` | unary aggr, or binary atomic | Reduce one vector, or choose the element-wise maximum of two values | `(max [3 9 1] 5)``[5 9 5]` |
| `med` | unary, aggr | Median value (returns f64) | `(med [1 3 2])``2.0` |
| `mode` | unary, aggr | Most frequent non-null value; ties keep the first encountered value | `(mode [1 2 2 3])``2` |
| `dev` | unary, aggr | Population standard deviation | `(dev [1 2 3])``0.816...` |
Expand Down Expand Up @@ -157,8 +157,9 @@ Operations on vectors as collections.
| `rotate` | binary | Rotate left by N positions; negative rotates right | `(rotate [1 2 3 4] 1)``[2 3 4 1]` |
| `cut` | binary | Split at sorted 0-based indices | `(cut [1 2 3 4] [2])``([1 2] [3 4])` |
| `cross` | binary | Cartesian product as pairs | `(cross [1 2] ['a 'b])``((1 'a) (1 'b) (2 'a) (2 'b))` |
| `at` | binary | Index into vector | `(at [10 20 30] 1)``20` |
| `at` | binary | Index a collection; vector keys batch dictionary lookups | `(at [10 20 30] 1)``20` |
| `find` | binary | Find index of value | `(find [10 20 30] 20)``1` |
| `fill` | binary, atomic | Replace null values; the replacement is the first argument | `(fill 0 [1 0Nl 3])``[1 0 3]` |
| `reverse` | unary | Reverse order | `(reverse [1 2 3])``[3 2 1]` |
| `til` | unary | Range [0..n) | `(til 5)``[0 1 2 3 4]` |
| `lag` | unary | Shift values one row back; first row is null/sentinel | `(lag [10 20 30])``[0Nl 10 20]` |
Expand Down Expand Up @@ -214,7 +215,7 @@ The time-series vector functions above are lazy-aware DAG operations for vector
| `key` | unary | Get column names (table) or keys (dict) | `(key trades)``[sym price size]` |
| `cols` | unary | Get table column names | `(cols trades)``[sym price size]` |
| `value` | unary | Get column data (table) or values (dict) | `(value trades)` |
| `dict` | binary | Create dictionary from keys and values | `(dict [a b] [1 2])` |
| `dict` | binary | Create a dictionary from evaluated keys and values | `(dict [a b] [1 2])` |
| `get` | binary | Lookup key in dict/table | `(get d 'a)``1` |
| `remove` | binary | Remove key from dict | `(remove d 'a)` |
| `row` | binary | Extract single row from table as dict | `(row trades 0)` |
Expand All @@ -228,7 +229,11 @@ The time-series vector functions above are lazy-aware DAG operations for vector
| `del` | variadic, special | Delete columns or rows from table | `(del trades 'temp_col)` |
| `modify` | variadic | Functional table update (returns new table) | `(modify trades 'price (fn [p] (* p 1.1)))` |

`xkey` returns a dictionary keyed by unique key column value(s). Duplicate keys are a domain error; use `xgroup` to get a dictionary whose values are grouped table slices.
`xkey` returns a dictionary keyed by unique key column value(s). Duplicate keys are a domain error; use `xgroup` to get a dictionary whose values are grouped table slices. Passing a typed vector of keys to `at` performs a batch lookup and returns a list of the corresponding values.

Dictionary literals are literals: their entries are not evaluated, just as the
elements of other literals are not evaluated. Use `(dict keys values)` to
construct a dictionary from evaluated expressions.

## Query Operations

Expand Down Expand Up @@ -322,10 +327,10 @@ Rayforce supports equi-joins, outer joins, anti-joins, and time-series-aware joi
(set products (table [product_id name] (list [10 20] [widget gadget])))
; Left join two tables on the sym column (join keys are a symbol list)
(left-join trades_j quotes [sym])
(left-join [sym] trades_j quotes)
; Inner join
(inner-join orders products [product_id])
(inner-join [product_id] orders products)
; Full outer join keeps rows from both sides
(full-join [sym] trades_j quotes)
Expand Down Expand Up @@ -430,7 +435,9 @@ Cross-temporal comparisons are supported: dates, times, and timestamps are all c
| `.csv.read` | variadic | Load CSV file into table | `(.csv.read "data.csv")` |
| `.csv.write` | variadic | Write table to CSV file | `(.csv.write trades "out.csv")` |
| `read` | unary | Read file contents as string | `(read "file.txt")` |
| `write` | binary | Write string to file | `(write "file.txt" "content")` |
| `read-bytes` | unary | Read file contents as a `U8` byte vector | `(read-bytes "file.bin")` |
| `write` | binary | Write a string to a file | `(write "file.txt" "content")` |
| `write-bytes` | binary | Write a `U8` byte vector to a file | `(write-bytes "file.bin" bytes)` |
| `load` | unary | Load and evaluate a Rayfall script | `(load "lib.rfl")` |

## Control Flow
Expand Down Expand Up @@ -509,15 +516,15 @@ Built-in support for triple stores. The EAV table has three columns: `e` (entity
|---|---|---|---|
| `union-all` | binary | Concatenate two tables (all rows) | `(union-all t1 t2)` |
| `distinct` | unary | Remove duplicate rows from a table | `(distinct t)` |
| `anti-join` | variadic | Anti-semi-join: rows in left not in right | `(anti-join t1 t2 [x])` |
| `anti-join` | variadic | Anti-semi-join: rows in left not in right | `(anti-join [x] t1 t2)` |

```lisp
; Table concatenation and deduplication
(set t1 (table [x] (list [1 2])))
(set t2 (table [x] (list [2 3])))
(union-all t1 t2) ; 4 rows: 1 2 2 3
(distinct (union-all t1 t2)) ; 3 rows: 1 2 3
(anti-join t1 t2 [x]) ; 1 row: 1
(anti-join [x] t1 t2) ; 1 row: 1
```

## Datalog
Expand Down
7 changes: 5 additions & 2 deletions docs/docs/queries/joins.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ Join two tables on shared key columns. Returns a table with all columns from bot
Join on multiple keys:

```lisp
; Key columns must be symbol (or numeric) — string key columns are not
; supported for joins.
; Key columns may be strings, symbols, or numeric values.
(set x (table [a b c]
(list (take [aa bb cc] 10)
(take [I J K] 10)
Expand All @@ -49,6 +48,10 @@ Join on multiple keys:
(inner-join [a b] x y)
```

Every named key must exist in both input tables. A misspelled or missing key is
a `domain` error instead of an empty result. A string key must be paired with a
string key on the other side.

## Left Join

The `left-join` function keeps all rows from the left table. Unmatched rows have null in the right-side columns.
Expand Down
Loading
Loading