Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions endpoints/gql/GQL_COMPILER_PROGRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Supported today:

- `MATCH` with node labels and inline property maps using typed scalar literals
- Linear relationship traversal with `Out`, `In`, and `Both` direction mapping
- Repeated node variables constrained to their original vertex binding
- Scalar comparison and `IS [NOT] NULL` `WHERE` predicates, with `AND`, `OR`, `NOT`, and parentheses
- `RETURN var`, `RETURN var.field`, and aliased property projections
- Minimal `ORDER BY` on the current traversal variable
Expand Down Expand Up @@ -52,7 +53,7 @@ This is a first-pass inventory for the compiler, not a server-wide conformance c
| Capability area | ISO classification | Compiler status | GripQL/runtime and test evidence | Next assessment |
|---|---|---|---|---|
| Scalar literals and property types | Required baseline types include `STRING`, `BOOL`, signed `INTEGER`, and `FLOAT`. | Inline maps and comparisons accept strings, integers, floats, booleans, and null. GQL quote doubling, backslash/control escapes, Unicode escapes, and `@` no-escape strings are decoded. | Compiler tests cover literal decoding; endpoint coverage exercises string and boolean predicates. | Establish the server's Unicode version/collation claim separately. |
| Graph matching and bindings | Mandatory graph-pattern capability area. | Supports labeled node patterns and linear chains with directed or undirected relationship traversal. Rejects `OPTIONAL MATCH`, path variables, unsupported directions, quantified relationships, and repeated node variables. | `Out`/`In`/`Both` cover simple traversal; endpoint tests cover multi-hop traversal. Compiler tests verify repeated-variable patterns fail explicitly. | Assess optional and quantified patterns against GripQL operations. Check optional-feature IDs before counting path extensions as required. |
| Graph matching and bindings | Mandatory graph-pattern capability area. | Supports labeled node patterns, linear chains, and repeated node variables constrained by GripQL `SameAs`. Rejects repeated edge variables, `OPTIONAL MATCH`, path variables, unsupported directions, and quantified relationships. | `Out`/`In`/`Both` cover simple traversal; endpoint tests exercise repeated node identity against a cycle. | Assess optional and quantified patterns against GripQL operations. Check optional-feature IDs before counting path extensions as required. |
| Predicates and boolean expressions | Comparisons and boolean expressions are mandatory functionality; `XOR` is optional feature GE07. | Supports property-to-scalar comparisons, `IS [NOT] NULL`, and `AND`/`OR`/`NOT` with parentheses and precedence. Requires references to the current variable; rejects `= NULL`, `XOR`, functions, arithmetic, and other operands. | Compiler represents each supported predicate's true and false cases separately, preserving `UNKNOWN` through `AND`/`OR`/`NOT` during filtering. Compiler and Badger endpoint tests cover null predicates and unknown comparisons. | Complete truth-table tests for the supported subset; do not generalize the guarantee to unsupported predicates or expressions. |
| Results, ordering, and paging | `RETURN`, ordering, and paging are in the mandatory query capability area; aggregate functions are part of the mandatory value-expression inventory. | Supports bare/property projections and aliases, current-variable field sorting, `SKIP`, and `LIMIT`. Aggregations and complex return expressions are rejected. | `Render`, `Sort`, `Skip`, and `Limit` are emitted; compiler and endpoint tests cover the basic forms. | Check GripQL aggregate/group execution and result shape before selecting aggregate syntax as the next feature. |
| Query composition and runtime services | Mandatory capability areas also cover query/conditional statements, status objects, sessions, and transactions. | The endpoint currently compiles a narrow single-query read-only form; there is no mapping for the broader statement/runtime areas. | Those concerns are not proven by compiler tests and may live outside the compiler boundary. | Audit server and client APIs separately; classify compiler gaps versus runtime/API work. |
Expand All @@ -71,15 +72,15 @@ Other optional features must be tracked separately. For example, `UNION` is opti
## Recent Implementation

- Inline property-map values preserve string, integer, float, boolean, and null types.
- GQL string literals decode their defined escape forms; repeated node-variable paths are rejected rather than silently rebound.
- GQL string literals decode their defined escape forms; repeated node variables compile to an identity constraint rather than being rebound.
- Boolean `WHERE` comparisons support `AND`, `OR`, `NOT`, and parentheses with GQL precedence; paired true/false filters preserve `UNKNOWN` for these supported forms. `XOR` remains unsupported.
- `WHERE var.field IS NULL` and `IS NOT NULL` compile through GripQL nil equality; comparisons to `NULL` are rejected.
- Compiler tests assert predicate trees, and the endpoint test exercises boolean filtering against a seeded graph.

## Refined Development Plan

1. **Complete the inventory.** The matrix above is the initial compiler view. Add evidence links/test names and audit the server/runtime rows before making any whole-product conformance statement.
2. **Continue semantic correctness.** Complete truth-table coverage for the supported predicate subset and verify null/string behavior consistently across backends. Repeated-variable paths remain explicitly rejected until GripQL can preserve their equality semantics. Broader predicate forms remain unsupported until their truth semantics can be preserved.
2. **Continue semantic correctness.** Complete truth-table coverage for the supported predicate subset and verify null/string behavior consistently across backends. Repeated edge variables remain unsupported until GripQL can preserve edge bindings. Broader predicate forms remain unsupported until their truth semantics can be preserved.
3. **Expand based on backend evidence.** Assess `OPTIONAL MATCH`, aggregates, and grouped results one at a time. Document the translation and backend limitations before adding table-driven compiler and endpoint tests.
4. **Keep scope explicit.** If the goal becomes ISO minimum conformance, separately plan graph population/management, status objects, Unicode behavior, and session/transaction APIs. Otherwise, keep the endpoint read-only and document it as a supported subset, not a conformant implementation.

Expand Down
27 changes: 26 additions & 1 deletion endpoints/gql/GQL_TO_GRIPQL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ The compiler currently emits these GripQL steps:
- `Has(gripql.And/Or/Not(...))` for supported boolean WHERE composition
- `Out(...)`, `In(...)`, and `Both(...)` for relationship traversal
- `As(...)`
- `SameAs(...)`
- `Sort(...)`
- `Render(...)`
- `Skip(...)`
Expand Down Expand Up @@ -84,6 +85,30 @@ Direction mapping, applied per hop in the chain:

Linear multi-hop chains are supported by applying the same mapping repeatedly.

### Repeated node variables

GQL:

```gql
MATCH (n)-[:FRIEND]->(next)-[:FRIEND]->(n)
RETURN n
```

GripQL constrains the final node to the original binding:

```go
gripql.NewQuery().
V().
As("n").
Out("FRIEND").
As("next").
Out("FRIEND").
SameAs("n").
Render("$n")
```

`SameAs` retains only travelers whose current vertex ID matches the vertex bound to the named variable. Repeated relationship variables are rejected until the compiler supports binding edge elements.

### MATCH with minimal WHERE predicate

GQL:
Expand Down Expand Up @@ -213,7 +238,7 @@ The compiler currently rejects queries containing any of the following:
- `WHERE XOR`, function calls, arithmetic, comparisons to `NULL`, and non-literal comparisons
- Full three-valued boolean semantics for `UNKNOWN`
- `WHERE` expressions that reference multiple variables
- Repeated node variables in a path, because GripQL cannot currently enforce GQL's repeated-binding equality semantics
- Repeated edge variables, because the current GQL mapping does not bind edge elements
- `WHERE` on a non-current traversal variable
- `WITH`
- `SET`
Expand Down
28 changes: 23 additions & 5 deletions endpoints/gql/compiler/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ type compiledWherePredicate struct {
whenFalse *gripql.HasExpression
}

func addVertexStep(q *gripql.Query, v vertexSelect) (*gripql.Query, error) {
func addVertexStep(q *gripql.Query, v vertexSelect, sameAs bool) (*gripql.Query, error) {
if len(v.label) > 0 {
q = q.HasLabel(v.label[0])
}
Expand All @@ -97,7 +97,9 @@ func addVertexStep(q *gripql.Query, v vertexSelect) (*gripql.Query, error) {
q = q.Has(gripql.Eq(key, value))
}
}
if v.name != "" {
if sameAs {
q = q.SameAs(v.name)
} else if v.name != "" {
q = q.As(v.name)
}
return q, nil
Expand Down Expand Up @@ -477,19 +479,34 @@ func (c *gqlListener) BuildQuery() (*gripql.Query, error) {
return nil, fmt.Errorf("invalid query: path length mismatch (vertices=%d, edges=%d)", len(c.vertexPath), len(c.edgePath))
}
seenVariables := make(map[string]struct{}, len(c.vertexPath))
repeatedVariables := make(map[string]bool, len(c.vertexPath))
for _, vertex := range c.vertexPath {
if vertex.name == "" {
continue
}
if _, exists := seenVariables[vertex.name]; exists {
return nil, fmt.Errorf("unsupported GQL features: repeated node variable %q", vertex.name)
repeatedVariables[vertex.name] = true
continue
}
seenVariables[vertex.name] = struct{}{}
}
seenEdgeVariables := make(map[string]struct{}, len(c.edgePath))
for _, edge := range c.edgePath {
if edge.name == "" {
continue
}
if _, exists := seenVariables[edge.name]; exists {
return nil, fmt.Errorf("unsupported GQL features: variable %q is used for both node and edge patterns", edge.name)
}
if _, exists := seenEdgeVariables[edge.name]; exists {
return nil, fmt.Errorf("unsupported GQL features: repeated edge variable %q", edge.name)
}
seenEdgeVariables[edge.name] = struct{}{}
}
q := gripql.NewQuery()
q = q.V()
var err error
q, err = addVertexStep(q, c.vertexPath[0])
q, err = addVertexStep(q, c.vertexPath[0], false)
if err != nil {
return nil, err
}
Expand All @@ -506,7 +523,8 @@ func (c *gqlListener) BuildQuery() (*gripql.Query, error) {
default:
return nil, fmt.Errorf("unsupported GQL features: relationship direction")
}
q, err = addVertexStep(q, c.vertexPath[i+1])
nextVertex := c.vertexPath[i+1]
q, err = addVertexStep(q, nextVertex, repeatedVariables[nextVertex.name])
if err != nil {
return nil, err
}
Expand Down
16 changes: 16 additions & 0 deletions endpoints/gql/test/gql_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package test

import (
"fmt"
"strings"
"testing"

"google.golang.org/protobuf/proto"
Expand Down Expand Up @@ -190,6 +191,14 @@ var pairs = []testPair{
true,
}, {
"MATCH (n)-[:FRIEND]->(n) RETURN n",
gripql.NewQuery().V().As("n").Out("FRIEND").SameAs("n").Render("$n"),
false,
}, {
"MATCH (n)-[r:FRIEND]->(friend)-[r:FRIEND]->(n) RETURN n",
nil,
true,
}, {
"MATCH (n)-[n:FRIEND]->(friend) RETURN friend",
nil,
true,
}, {
Expand Down Expand Up @@ -253,3 +262,10 @@ func TestMatch1(t *testing.T) {
}
}
}

func TestRepeatedEdgeVariableRejected(t *testing.T) {
_, err := compiler.RunParser("MATCH (n)-[r:FRIEND]->(middle)-[r:FRIEND]->(n) RETURN n")
if err == nil || !strings.Contains(err.Error(), `repeated edge variable "r"`) {
t.Fatalf("expected a repeated edge variable error, got %v", err)
}
}
27 changes: 27 additions & 0 deletions engine/core/processors.go
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,33 @@ func (m *Marker) Process(ctx context.Context, man gdbi.Manager, in gdbi.InPipe,
return ctx
}

type SameAsFilter struct {
mark string
}

func (s *SameAsFilter) Process(ctx context.Context, man gdbi.Manager, in gdbi.InPipe, out gdbi.OutPipe) context.Context {
go func() {
defer close(out)
for t := range in {
if t.IsSignal() {
out <- t
continue
}
current := t.GetCurrent()
bound := t.GetMark(s.mark)
if current == nil || bound == nil {
continue
}
currentElement := current.Get()
boundElement := bound.Get()
if currentElement != nil && boundElement != nil && currentElement.ID == boundElement.ID {
out <- t
}
}
}()
return ctx
}

////////////////////////////////////////////////////////////////////////////////

// Selector selects marks to return
Expand Down
66 changes: 66 additions & 0 deletions engine/core/sameas_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package core

import (
"context"
"strings"
"testing"

"github.com/bmeg/grip/gdbi"
"github.com/bmeg/grip/gripql"
)

func TestSameAsFilter(t *testing.T) {
matching := &gdbi.BaseTraveler{
Current: &gdbi.DataElement{ID: "same"},
Marks: map[string]*gdbi.DataElement{"bound": {ID: "same"}},
}
different := &gdbi.BaseTraveler{
Current: &gdbi.DataElement{ID: "different"},
Marks: map[string]*gdbi.DataElement{"bound": {ID: "same"}},
}
missing := &gdbi.BaseTraveler{Current: &gdbi.DataElement{ID: "same"}}

in := make(chan gdbi.Traveler, 3)
in <- matching
in <- different
in <- missing
close(in)

out := make(chan gdbi.Traveler)
(&SameAsFilter{mark: "bound"}).Process(context.Background(), nil, in, out)

var results []gdbi.Traveler
for traveler := range out {
results = append(results, traveler)
}
if len(results) != 1 || results[0] != matching {
t.Fatalf("expected only the traveler matching its binding, got %#v", results)
}
}

func TestSameAsCompileErrors(t *testing.T) {
tests := []struct {
name string
query *gripql.Query
want string
}{
{
name: "unknown binding",
query: gripql.NewQuery().V().SameAs("missing"),
want: `unknown binding "missing"`,
},
{
name: "mismatched element kind",
query: gripql.NewQuery().V().As("vertex").OutE().SameAs("vertex"),
want: `binding "vertex" has type VertexData, current element has type EdgeData`,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := NewCompiler(nil).Compile(tc.query.Statements, nil)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("expected error containing %q, got %v", tc.want, err)
}
})
}
}
4 changes: 4 additions & 0 deletions engine/core/statement_compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,10 @@ func (sc *DefaultStmtCompiler) As(stmt *gripql.GraphStatement_As, ps *gdbi.State
return &Marker{stmt.As}, nil
}

func (sc *DefaultStmtCompiler) SameAs(stmt *gripql.GraphStatement_SameAs, ps *gdbi.State) (gdbi.Processor, error) {
return &SameAsFilter{mark: stmt.SameAs}, nil
}

func (sc *DefaultStmtCompiler) Set(stmt *gripql.GraphStatement_Set, ps *gdbi.State) (gdbi.Processor, error) {
return &ValueSet{key: stmt.Set.Key, value: stmt.Set.Value.AsInterface()}, nil
}
Expand Down
1 change: 1 addition & 0 deletions gdbi/pipeline.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ type StatementCompiler interface {
Count(gs *gripql.GraphStatement_Count, ps *State) (Processor, error)
Distinct(gs *gripql.GraphStatement_Distinct, ps *State) (Processor, error)
As(gs *gripql.GraphStatement_As, ps *State) (Processor, error)
SameAs(gs *gripql.GraphStatement_SameAs, ps *State) (Processor, error)
Set(gs *gripql.GraphStatement_Set, ps *State) (Processor, error)
Increment(gs *gripql.GraphStatement_Increment, ps *State) (Processor, error)
Mark(gs *gripql.GraphStatement_Mark, ps *State) (Processor, error)
Expand Down
22 changes: 22 additions & 0 deletions gdbi/statement_processor.go
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,28 @@ func StatementProcessor(
ps.MarkTypes[stmt.As] = ps.LastType
return sc.As(stmt, ps)

case *gripql.GraphStatement_SameAs:
if ps.LastType != VertexData && ps.LastType != EdgeData {
return nil, fmt.Errorf(`"sameAs" statement is only valid for edge or vertex types not: %s`, ps.LastType.String())
}
if stmt.SameAs == "" {
return nil, fmt.Errorf(`"sameAs" statement cannot have an empty name`)
}
if err := gripql.ValidateFieldName(stmt.SameAs); err != nil {
return nil, fmt.Errorf(`"sameAs" statement invalid; %v`, err)
}
if stmt.SameAs == tpath.CURRENT {
return nil, fmt.Errorf(`"sameAs" statement invalid; uses reserved name %s`, tpath.CURRENT)
}
markType, ok := ps.MarkTypes[stmt.SameAs]
if !ok {
return nil, fmt.Errorf(`"sameAs" statement references unknown binding %q`, stmt.SameAs)
}
if markType != ps.LastType {
return nil, fmt.Errorf(`"sameAs" statement binding %q has type %s, current element has type %s`, stmt.SameAs, markType.String(), ps.LastType.String())
}
return sc.SameAs(stmt, ps)

case *gripql.GraphStatement_Set:
return sc.Set(stmt, ps)

Expand Down
3 changes: 1 addition & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ require (
github.com/Shopify/sarama v1.38.1
github.com/Workiva/go-datastructures v1.1.5
github.com/akrylysov/pogreb v0.10.2
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10
github.com/antlr4-go/antlr/v4 v4.13.1
github.com/bmeg/benchtop v0.0.0-20260306193933-7cdd75fcb2fb
github.com/bmeg/jsonpath v0.0.0-20210207014051-cca5355553ad
github.com/bmeg/jsonschema/v6 v6.0.4
Expand Down Expand Up @@ -62,7 +62,6 @@ require (
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.2 // indirect
github.com/DataDog/zstd v1.5.7 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/apache/arrow/go/v18 v18.0.0-20241007013041-ab95a4d25142 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ github.com/akrylysov/pogreb v0.10.2 h1:e6PxmeyEhWyi2AKOBIJzAEi4HkiC+lKyCocRGlnDi
github.com/akrylysov/pogreb v0.10.2/go.mod h1:pNs6QmpQ1UlTJKDezuRWmaqkgUE2TuU0YTWyqJZ7+lI=
github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10 h1:yL7+Jz0jTC6yykIK/Wh74gnTJnrGr5AyrNMXuA0gves=
github.com/antlr/antlr4/runtime/Go/antlr v1.4.10/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY=
github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
github.com/apache/arrow/go/v18 v18.0.0-20241007013041-ab95a4d25142 h1:6EtsUpu9/vLtVl6oVpFiZe9GRax7STd2bG55VNwsRdI=
Expand Down
Loading
Loading