This is an automated email from the ASF dual-hosted git repository. colegreer pushed a commit to branch TINKERPOP-3107 in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit e78ca26dae90d655c333e81af4cd3a4022a7cc50 Author: Cole Greer <[email protected]> AuthorDate: Tue Apr 14 08:59:33 2026 -0700 docs and tests --- CHANGELOG.asciidoc | 8 + docs/src/reference/gremlin-applications.asciidoc | 144 +++++++++++--- docs/src/upgrade/release-4.x.x.asciidoc | 89 +++++++++ .../conf/gremlin-server-modern-readonly.yaml | 2 +- .../server/GremlinServerConfigIntegrateTest.java | 112 +++++++++++ .../tinkerpop/gremlin/server/SettingsTest.java | 78 ++++++++ .../server/util/ServerGremlinExecutorTest.java | 147 ++++++++++++++ .../server/util/TinkerFactoryDataLoaderTest.java | 216 +++++++++++++++++++++ .../src/test/resources/conf/remote-objects.yaml | 3 +- .../gremlin/server/gremlin-server-minimal.yaml} | 19 +- .../gremlin-server-with-traversal-sources.yaml} | 25 +-- 11 files changed, 778 insertions(+), 65 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 032033ad5e..5d391b003f 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -25,6 +25,14 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima [[release-4-0-0]] === TinkerPop 4.0.0 (Release Date: NOT OFFICIALLY RELEASED YET) +* Added declarative `traversalSources` configuration to Gremlin Server YAML for creating `TraversalSource` instances with optional strategy configuration queries. +* Added Java-based `lifecycleHooks` configuration to Gremlin Server YAML, replacing Groovy init script `LifeCycleHook` creation. +* Added `TinkerFactoryDataLoader` `LifeCycleHook` implementation for loading sample datasets without Groovy. +* Added auto-creation of `TraversalSource` bindings from `graphs` configuration (`graph` maps to `g`, others to `g_<name>`). +* Added `GraphManager` to `LifeCycleHook.Context` for Java-based hooks to access configured graphs. +* Deprecated Groovy-based `LifeCycleHook` and `TraversalSource` creation via init scripts in favor of YAML configuration. +* Updated all default Gremlin Server configs to remove Groovy dependency from initialization. + [[release-4-0-0-beta-2]] === TinkerPop 4.0.0-beta.2 (April 1, 2026) diff --git a/docs/src/reference/gremlin-applications.asciidoc b/docs/src/reference/gremlin-applications.asciidoc index 8039afff90..412612b42a 100644 --- a/docs/src/reference/gremlin-applications.asciidoc +++ b/docs/src/reference/gremlin-applications.asciidoc @@ -449,10 +449,10 @@ $ bin/gremlin-server.sh conf/gremlin-server-modern.yaml [INFO] DefaultGraphManager - Graph [graph] was successfully configured via [conf/tinkergraph-empty.properties]. [INFO] ServerGremlinExecutor - Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-* [INFO] ServerGremlinExecutor - Initialized GremlinExecutor and preparing GremlinScriptEngines instances. -[INFO] ServerGremlinExecutor - Initialized gremlin-groovy GremlinScriptEngine and registered metrics -[INFO] ServerGremlinExecutor - A GraphTraversalSource is now bound to [g] with graphtraversalsource[tinkergraph[vertices:0 edges:0], standard] +[INFO] ServerGremlinExecutor - A GraphTraversalSource is now auto-bound to [g] for graph [graph] +[INFO] ServerGremlinExecutor - Instantiated LifeCycleHook: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader [INFO] GremlinServer - Executing start up LifeCycleHook -[INFO] Logger$info - Loading 'modern' graph data. +[INFO] TinkerFactoryDataLoader - TinkerFactoryDataLoader loaded [modern] dataset into graph [graph] [INFO] GremlinServer - idleConnectionTimeout was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled [INFO] GremlinServer - keepAliveInterval was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled [INFO] AbstractChannelizer - Configured application/vnd.gremlin-v4.0+json with org.apache.tinkerpop.gremlin.util.ser.GraphSONMessageSerializerV4 @@ -469,31 +469,83 @@ That file tells Gremlin Server many things such as: * Thread pool sizes * Where to report metrics gathered by the server * The serializers to make available -* The Gremlin `ScriptEngine` instances to expose and external dependencies to inject into them * `Graph` instances to expose +* `TraversalSource` bindings (auto-created or explicitly declared) +* `LifeCycleHook` implementations for startup/shutdown logic The log messages that printed above show a number of things, but most importantly, there is a `Graph` instance named `graph` that is exposed in Gremlin Server. This graph is an in-memory TinkerGraph and was empty at the start of the -server. An initialization script at `scripts/generate-modern.groovy` was executed during startup. Its contents are -as follows: +server. A `TinkerFactoryDataLoader` lifecycle hook loaded the "modern" dataset into it during startup, and a +`TraversalSource` named `g` was auto-created from the `graph` entry. -[source,groovy] +[[server-auto-traversal-sources]] +==== Auto-Created TraversalSources + +When Gremlin Server starts, it automatically creates a `TraversalSource` for each graph in the `graphs` configuration +that does not have an explicit entry in the `traversalSources` section. The naming convention is: + +* A graph named `graph` gets a `TraversalSource` named `g` +* All other graphs get `g_<name>` (e.g. a graph named `modern` gets `g_modern`) + +This means a minimal configuration like the following is fully functional: + +[source,yaml] +---- +graphs: { + graph: conf/tinkergraph-empty.properties} +---- + +[[server-traversal-sources]] +==== Declarative TraversalSources + +For more control, the `traversalSources` YAML section allows explicit `TraversalSource` creation with optional +strategy configuration via a Gremlin query: + +[source,yaml] +---- +traversalSources: { + g: {graph: graph}, + gReadOnly: {graph: graph, query: "g.withStrategies(ReadOnlyStrategy)"}} +---- + +Each entry supports: + +* `graph` (required) — references a key in the `graphs` section +* `query` (optional) — a Gremlin expression evaluated with a base traversal source bound as `g`; the result becomes + the final `TraversalSource` +* `language` (optional) — which script engine to use for query evaluation; defaults to `gremlin-lang` or the single + configured non-`gremlin-lang` engine + +Graphs with explicit `traversalSources` entries are excluded from auto-creation. + +[[server-lifecycle-hooks]] +==== LifeCycleHooks + +The `lifecycleHooks` YAML section configures Java-based `LifeCycleHook` implementations that execute during server +startup and shutdown: + +[source,yaml] ---- -include::{basedir}/gremlin-server/scripts/generate-modern.groovy[] +lifecycleHooks: + - className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader + config: {graph: graph, dataset: modern} ---- -The script above initializes a `Map` and assigns two key/values to it. The first, assigned to "hook", defines a -`LifeCycleHook` for Gremlin Server. The "hook" provides a way to tie script code into the Gremlin Server startup and -shutdown sequences. The `LifeCycleHook` has two methods that can be implemented: `onStartUp` and `onShutDown`. -These events are called once at Gremlin Server start and once at Gremlin Server stop. This is an important point -because code outside of the "hook" is executed for each `ScriptEngine` creation (multiple may be created when -"sessions" are enabled) and therefore the `LifeCycleHook` provides a way to ensure that a script is only executed a -single time. In this case, the startup hook loads the "modern" graph into the empty TinkerGraph instance, preparing -it for use. The second key/value pair assigned to the `Map`, named "g", defines a `TraversalSource` from the `Graph` -bound to the "graph" variable in the YAML configuration file. This variable `g`, as well as any other variable -assigned to the `Map`, will be made available as variables for future remote script executions. In more general -terms, any key/value pairs assigned to a `Map` returned from the initialization script will become variables that -are global to all requests. In addition, any functions that are defined will be cached for future use. +Each entry specifies: + +* `className` (required) — the fully qualified class name of a `LifeCycleHook` implementation +* `config` (optional) — a `Map` of key/value pairs passed to the hook's `init(Map)` method + +The built-in `TinkerFactoryDataLoader` loads TinkerFactory sample datasets into a graph. It accepts two config +parameters: `graph` (the name of the graph as defined in `graphs`) and `dataset` (one of `modern`, `classic`, `crew`, +`grateful`, or `sink`). + +Custom `LifeCycleHook` implementations must have a no-arg constructor and implement the `onStartUp(Context)` and +`onShutDown(Context)` methods. The `Context` provides access to a `Logger` and the `GraphManager`. + +IMPORTANT: Creating `LifeCycleHook` and `TraversalSource` instances via Groovy init scripts is deprecated. Use the +`lifecycleHooks` and `traversalSources` YAML sections instead. Existing Groovy scripts continue to work but will +produce a deprecation warning at startup. WARNING: Transactions on graphs in initialization scripts are not closed automatically after the script finishes executing. It is up to the script to properly commit or rollback transactions in the script itself. @@ -842,8 +894,15 @@ The following table describes the various YAML configuration options that Gremli |scriptEngines |A `Map` of `ScriptEngine` implementations to expose through Gremlin Server, where the key is the name given by the `ScriptEngine` implementation. The key must match the name exactly for the `ScriptEngine` to be constructed. The value paired with this key is itself a `Map` of configuration for that `ScriptEngine`. If this value is not set, it will default to "gremlin-lang". |_gremlin-lang_ |scriptEngines.<name>.imports |A comma separated list of classes/packages to make available to the `ScriptEngine`. |_none_ |scriptEngines.<name>.staticImports |A comma separated list of "static" imports to make available to the `ScriptEngine`. |_none_ -|scriptEngines.<name>.scripts |A comma separated list of script files to execute on `ScriptEngine` initialization. `Graph` and `TraversalSource` instance references produced from scripts will be stored globally in Gremlin Server, therefore it is possible to use initialization scripts to add Traversal Strategies or create entirely new `Graph` instances all together. Instantiating a `LifeCycleHook` in a script provides a way to execute scripts when Gremlin Server starts and stops.|_none_ +|scriptEngines.<name>.scripts |A comma separated list of script files to execute on `ScriptEngine` initialization. Deprecated — use `traversalSources` and `lifecycleHooks` instead.|_none_ |scriptEngines.<name>.config |A `Map` of configuration settings for the `ScriptEngine`. These settings are dependent on the `ScriptEngine` implementation being used. |_none_ +|traversalSources |A `Map` of `TraversalSource` configurations keyed by binding name. Each entry specifies a `graph` reference and optionally a `query` to configure strategies. See <<server-traversal-sources>>. |_none (auto-created from graphs)_ +|traversalSources.<name>.graph |The name of the graph (as defined in `graphs`) to create the traversal source from. |_none_ +|traversalSources.<name>.query |An optional Gremlin query evaluated with a base traversal source bound as `g`. The result becomes the final `TraversalSource`. |_none_ +|traversalSources.<name>.language |The script engine language to use for evaluating `query`. Falls back to the single configured non-`gremlin-lang` engine or `gremlin-lang`. |_auto-detected_ +|lifecycleHooks |A `List` of Java-based `LifeCycleHook` implementations to instantiate and execute during server startup and shutdown. See <<server-lifecycle-hooks>>. |_none_ +|lifecycleHooks[X].className |The fully qualified class name of the `LifeCycleHook` implementation. |_none_ +|lifecycleHooks[X].config |A `Map` of configuration passed to the hook's `init(Map)` method. |_none_ |evaluationTimeout |The amount of time in milliseconds before a request evaluation and iteration of result times out. This feature can be turned off by setting the value to `0`. |30000 |serializers |A `List` of `Map` settings, where each `Map` represents a `MessageSerializer` implementation to use along with its configuration. If this value is not set, then Gremlin Server will configure with GraphSON and GraphBinary but will not register any `ioRegistries` for configured graphs. |_empty_ |serializers[X].className |The full class name of the `MessageSerializer` implementation. |_none_ @@ -1278,14 +1337,20 @@ consult the documentation of the graph you are using to determine what authoriza Gremlin Server supports three mechanisms to configure authorization: -. With the `ScriptFileGremlinPlugin` a groovy script is configured that instantiates the `GraphTraversalSources` that -can be accessed by client requests. Using the `withStrategies()` gremlin -link:https://tinkerpop.apache.org/docs/x.y.z/reference/#start-steps[start step], one can apply so-called -link:https://tinkerpop.apache.org/docs/x.y.z/reference/#traversalstrategy[TraversalStrategy instances] to these -`GraphTraversalSource` instances, some of which can serve for authorization purposes (`ReadOnlyStrategy`, +. With the `traversalSources` YAML configuration, one can declare `TraversalSource` instances with +link:https://tinkerpop.apache.org/docs/x.y.z/reference/#traversalstrategy[TraversalStrategy instances] applied via +a Gremlin query, some of which can serve for authorization purposes (`ReadOnlyStrategy`, `LambdaRestrictionStrategy`, `VertexProgramRestrictionStrategy`, `SubgraphStrategy`, `PartitionStrategy`, `EdgeLabelVerificationStrategy`), provided that users are not allowed to remove or modify these `TraversalStrategy` -instances afterwards. The `ScriptFileGremlinPlugin` is found in the yaml configuration file for Gremlin Server: +instances afterwards. For example: ++ +[source,yaml] +---- +traversalSources: { + g: {graph: graph, query: "g.withStrategies(ReadOnlyStrategy)"}} +---- ++ +Alternatively, the deprecated `ScriptFileGremlinPlugin` approach can still be used with a Groovy script: + [source,yaml] ---- @@ -1314,12 +1379,13 @@ gives an overview. [width="95%",cols="5,2,2,4",options="header"] |========================================================= |Type (mechanism) |GraphTraversalSources |Groups |Bytecode analysis -|Implicit (init script) | all accessible |one |`withStrategies()` +|Implicit (YAML config or init script) | all accessible |one |`withStrategies()` |Passive (pass/deny) | selected access |few |hybrid |Active (inject) |selected access |many |hybrid |========================================================= -With implicit authorization (only adding restricting `TraversalStrategy` instances in the initialization script of +With implicit authorization (configuring restricting `TraversalStrategy` instances via the `traversalSources` YAML +section or in a Groovy initialization script of Gremlin Server) all authenticated users can access all hosted `GraphTraversalSources` and all face the same restrictions. One would need separate Gremlin Server instances for each authorization policy and apply an authenticator that restricts access to a group of users (that is, supports in authorization). @@ -1330,7 +1396,8 @@ the most flexible and can support an almost unlimited number of authorization po implement. In particular, applying the `SubgraphStrategy` requires knowledge about the schema of the graph. The passive authorization solution perhaps provides a middle ground to start implementing authorization. This -solution assumes that the `SubgraphStrategy` is applied in the Gremlin Server initialization script, because compliance +solution assumes that the `SubgraphStrategy` is applied in the Gremlin Server `traversalSources` configuration (or +initialization script), because compliance with a subgraph restriction can only be determined during the actual execution of the gremlin traversal. Note that the same graph can be reused with different `SubgraphStrategies`. Now, authorization policies can be defined in terms of accessible `GraphTraversalSources` and the authorizer can simply match the requested access to a `GraphTraversalSource` @@ -1589,6 +1656,11 @@ groupsandbox: [usersandbox, marko] [[script-execution]] ==== Protecting Script Execution +NOTE: As of 4.0.0, Groovy is no longer required for server initialization. The `scriptEngines` configuration +described in this section is only needed when Groovy script execution is explicitly enabled for remote script +submission. See <<server-lifecycle-hooks>> and <<server-traversal-sources>> for the preferred YAML-based +initialization approach. + It is important to remember that Gremlin Server exposes `GremlinScriptEngine` instances that allows for remote execution of arbitrary code on the server. Obviously, this situation can represent a security risk or, more minimally, provide ways for "bad" scripts to be inadvertently executed. A simple example of a "valid" Gremlin script that would cause @@ -1982,7 +2054,15 @@ Another option, `all`, can be used to indicate that all properties should be ret In some cases it can be inconvenient to load Elements with properties due to large data size or for compatibility reasons. That can be solved by utilizing `ReferenceElementStrategy` when creating the out-of-the-box `GraphTraversalSource`. As the name suggests, this means that elements will be detached by reference and will therefore not have properties -included. The relevant configuration from the Gremlin Server initialization script looks like this: +included. The relevant configuration from the Gremlin Server YAML looks like this: + +[source,yaml] +---- +traversalSources: { + g: {graph: graph, query: "g.withStrategies(ReferenceElementStrategy)"}} +---- + +Alternatively, using the deprecated Groovy init script approach: [source,groovy] ---- @@ -2233,7 +2313,7 @@ NOTE: This plugin is typically only useful to the Gremlin Console and is enabled The Server Plugin for remoting with the Gremlin Console should not be confused with a plugin of similar name that is used by the server. `GremlinServerGremlinPlugin` is typically only configured in Gremlin Server and provides a number -of imports that are required for writing <<starting-gremlin-server,initialization scripts>>. +of imports that are required for writing Groovy initialization scripts (if Groovy is enabled). [[spark-plugin]] === Spark Plugin diff --git a/docs/src/upgrade/release-4.x.x.asciidoc b/docs/src/upgrade/release-4.x.x.asciidoc index ebc2bbb1cc..c7c13e021b 100644 --- a/docs/src/upgrade/release-4.x.x.asciidoc +++ b/docs/src/upgrade/release-4.x.x.asciidoc @@ -30,6 +30,95 @@ image::gremlins-wildest-dreams.png[width=185] Please see the link:https://github.com/apache/tinkerpop/blob/4.0.0/CHANGELOG.asciidoc#release-4-0-0[changelog] for a complete list of all the modifications that are part of this release. +=== Upgrading for Users + +==== Gremlin Server Initialization Without Groovy + +Gremlin Server no longer requires Groovy for server initialization. Three new YAML configuration mechanisms replace +the Groovy init scripts that were previously used to create `TraversalSource` bindings, load data, and run lifecycle +hooks. + +===== Auto-Created TraversalSources + +Any graph defined in the `graphs` section that does not have an explicit `traversalSources` entry will automatically +get a `TraversalSource` binding. A graph named `graph` is bound to `g`; all others are bound to `g_<name>`. + +A minimal server configuration is now: + +[source,yaml] +---- +graphs: { + graph: conf/tinkergraph-empty.properties} +---- + +This automatically creates a `g` binding — no init script needed. + +===== Declarative `traversalSources` + +A new `traversalSources` YAML section allows explicit `TraversalSource` creation with optional strategy configuration: + +[source,yaml] +---- +traversalSources: { + g: {graph: graph}, + gReadOnly: {graph: graph, query: "g.withStrategies(ReadOnlyStrategy)"}} +---- + +Each entry specifies: + +- `graph` (required) — references a key in the `graphs` section +- `query` (optional) — a Gremlin expression evaluated with a base traversal source bound as `g` +- `language` (optional) — which script engine to use for the query (defaults to `gremlin-lang`, or the single + configured non-`gremlin-lang` engine if only one exists) + +===== Java-Based `lifecycleHooks` + +A new `lifecycleHooks` YAML section replaces Groovy-based `LifeCycleHook` creation: + +[source,yaml] +---- +lifecycleHooks: + - className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader + config: {graph: graph, dataset: modern} +---- + +Each entry specifies a `className` implementing `LifeCycleHook` and an optional `config` map passed to the hook's +`init()` method. The built-in `TinkerFactoryDataLoader` supports datasets: `modern`, `classic`, `crew`, `grateful`, +and `sink`. + +===== Deprecation of Groovy Init Scripts + +Creating `TraversalSource` and `LifeCycleHook` instances via Groovy init scripts is now deprecated. Existing scripts +continue to work, but a deprecation warning is logged at startup. Migrate to the YAML-based configuration described +above. + +===== Migration Examples + +*Before (Groovy init script):* + +[source,groovy] +---- +def globals = [:] +globals << [hook : [ + onStartUp: { ctx -> + org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory.generateModern(graph) + } +] as LifeCycleHook] +globals << [g : traversal().withEmbedded(graph)] +---- + +*After (YAML only):* + +[source,yaml] +---- +graphs: { + graph: conf/tinkergraph-empty.properties} +lifecycleHooks: + - { className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader, config: {graph: graph, dataset: modern}} +---- + +The `g` binding is auto-created from the `graph` entry. No `scriptEngines` section is needed. + == TinkerPop 4.0.0-beta.2 *Release Date: April 1, 2026* diff --git a/gremlin-server/conf/gremlin-server-modern-readonly.yaml b/gremlin-server/conf/gremlin-server-modern-readonly.yaml index 975caed215..e0c87103eb 100644 --- a/gremlin-server/conf/gremlin-server-modern-readonly.yaml +++ b/gremlin-server/conf/gremlin-server-modern-readonly.yaml @@ -23,7 +23,7 @@ graphs: { traversalSources: { g: {graph: graph, query: "g.withStrategies(ReadOnlyStrategy)"}} lifecycleHooks: - - { className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader, config: {graph: graph, dataset: classic}} + - { className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader, config: {graph: graph, dataset: modern}} serializers: - { className: org.apache.tinkerpop.gremlin.util.ser.GraphSONMessageSerializerV3, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3] }} # application/json - { className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 } # application/vnd.graphbinary-v1.0 diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerConfigIntegrateTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerConfigIntegrateTest.java new file mode 100644 index 0000000000..647bbba964 --- /dev/null +++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerConfigIntegrateTest.java @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.server; + +import org.apache.tinkerpop.gremlin.driver.Client; +import org.apache.tinkerpop.gremlin.driver.Cluster; +import org.apache.tinkerpop.gremlin.driver.Result; +import org.apache.tinkerpop.gremlin.tinkergraph.structure.AbstractTinkerGraph; +import org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.File; +import java.io.FileInputStream; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.List; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.is; + +/** + * Validates that each shipped server config can start successfully and serve a basic query. + * Configs requiring SSL or authentication infrastructure are excluded. + */ +@RunWith(Parameterized.class) +public class GremlinServerConfigIntegrateTest { + + private static final Logger logger = LoggerFactory.getLogger(GremlinServerConfigIntegrateTest.class); + private static final int TEST_PORT = 45950; + + @Parameterized.Parameter + public String configName; + + @Parameterized.Parameters(name = "{0}") + public static Collection<String> configs() { + return Arrays.asList( + "gremlin-server.yaml", + "gremlin-server-min.yaml", + "gremlin-server-modern.yaml", + "gremlin-server-classic.yaml", + "gremlin-server-modern-readonly.yaml", + "gremlin-server-rest-modern.yaml", + "gremlin-server-transaction.yaml" + ); + } + + @Test + public void shouldStartAndServeQuery() throws Exception { + final File confDir = new File(System.getProperty("build.dir"), "../conf"); + final Settings settings = Settings.read(new FileInputStream(new File(confDir, configName))); + + settings.port = TEST_PORT; + ServerTestHelper.rewritePathsInGremlinServerSettings(settings); + + // ensure a V4 serializer is available so the V4 client can connect + if (!hasV4Serializer(settings)) { + final Settings.SerializerSettings v4 = new Settings.SerializerSettings(); + v4.className = GraphBinaryMessageSerializerV4.class.getName(); + final List<Settings.SerializerSettings> serializers = new ArrayList<>(settings.serializers); + serializers.add(v4); + settings.serializers = serializers; + } + + final GremlinServer server = new GremlinServer(settings); + try { + server.start().join(); + logger.info("Started server with config: {}", configName); + + final Cluster cluster = Cluster.build("localhost").port(TEST_PORT).create(); + final Client client = cluster.connect(); + try { + final List<Result> results = client.submit("g.inject(1)").all().get(); + assertThat(results.size(), is(1)); + assertThat(results.get(0).getInt(), is(1)); + } finally { + client.close(); + cluster.close(); + } + } finally { + server.getServerGremlinExecutor().getGraphManager().getAsBindings().values().stream() + .filter(g -> g instanceof AbstractTinkerGraph) + .forEach(g -> ((AbstractTinkerGraph) g).clear()); + server.stop().join(); + } + } + + private static boolean hasV4Serializer(final Settings settings) { + return settings.serializers.stream() + .anyMatch(s -> s.className.contains("V4")); + } +} diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/SettingsTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/SettingsTest.java index d399a5ad65..3eedf8f24c 100644 --- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/SettingsTest.java +++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/SettingsTest.java @@ -24,6 +24,11 @@ import org.yaml.snakeyaml.constructor.Constructor; import java.io.InputStream; +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.hasKey; +import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.notNullValue; +import static org.hamcrest.Matchers.nullValue; import static org.junit.Assert.assertEquals; public class SettingsTest { @@ -56,4 +61,77 @@ public class SettingsTest { assertEquals("localhost", settings.customValue); } + + @Test + public void scriptEnginesDefaultsToGremlinLangWhenAbsentFromYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-minimal.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.scriptEngines, is(notNullValue())); + assertThat(settings.scriptEngines.size(), is(1)); + assertThat(settings.scriptEngines, hasKey("gremlin-lang")); + } + + @Test + public void scriptEnginesPopulatedWhenPresentInYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-integration.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.scriptEngines, hasKey("gremlin-groovy")); + assertThat(settings.scriptEngines, hasKey("gremlin-lang")); + } + + @Test + public void traversalSourcesParsedFromYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-with-traversal-sources.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.traversalSources.size(), is(2)); + assertThat(settings.traversalSources, hasKey("g")); + assertThat(settings.traversalSources, hasKey("gReadOnly")); + + final Settings.TraversalSourceSettings gSettings = settings.traversalSources.get("g"); + assertThat(gSettings.graph, is("graph")); + assertThat(gSettings.query, is(nullValue())); + assertThat(gSettings.language, is(nullValue())); + + final Settings.TraversalSourceSettings roSettings = settings.traversalSources.get("gReadOnly"); + assertThat(roSettings.graph, is("graph")); + assertThat(roSettings.query, is("g.withStrategies(ReadOnlyStrategy)")); + assertThat(roSettings.language, is("gremlin-groovy")); + } + + @Test + public void traversalSourcesDefaultsToEmptyMapWhenAbsentFromYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-minimal.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.traversalSources, is(notNullValue())); + assertThat(settings.traversalSources.isEmpty(), is(true)); + } + + @Test + public void lifecycleHooksParsedFromYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-with-traversal-sources.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.lifecycleHooks.size(), is(2)); + + final Settings.LifeCycleHookSettings first = settings.lifecycleHooks.get(0); + assertThat(first.className, is("org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader")); + assertThat(first.config, hasKey("graph")); + assertThat(first.config.get("dataset"), is("modern")); + + final Settings.LifeCycleHookSettings second = settings.lifecycleHooks.get(1); + assertThat(second.config.get("dataset"), is("classic")); + } + + @Test + public void lifecycleHooksDefaultsToEmptyListWhenAbsentFromYaml() throws Exception { + final InputStream stream = SettingsTest.class.getResourceAsStream("gremlin-server-minimal.yaml"); + final Settings settings = Settings.read(stream); + + assertThat(settings.lifecycleHooks, is(notNullValue())); + assertThat(settings.lifecycleHooks.isEmpty(), is(true)); + } } diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutorTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutorTest.java new file mode 100644 index 0000000000..03320268f7 --- /dev/null +++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/ServerGremlinExecutorTest.java @@ -0,0 +1,147 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.server.util; + +import org.apache.tinkerpop.gremlin.server.Settings; +import org.junit.After; +import org.junit.Test; + +import java.io.InputStream; +import java.lang.reflect.Method; +import java.util.HashMap; +import java.util.LinkedHashMap; +import java.util.Map; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.notNullValue; +import static org.hamcrest.Matchers.nullValue; + +public class ServerGremlinExecutorTest { + + private ServerGremlinExecutor serverGremlinExecutor; + + @After + public void tearDown() { + if (serverGremlinExecutor != null) { + serverGremlinExecutor.getGremlinExecutorService().shutdownNow(); + serverGremlinExecutor.getGraphManager().getGraphNames().forEach(name -> { + try { + serverGremlinExecutor.getGraphManager().getGraph(name).close(); + } catch (Exception ignored) { + } + }); + } + } + + private Settings readSettings(final String resource) { + final InputStream stream = ServerGremlinExecutorTest.class.getResourceAsStream("../" + resource); + final Settings settings = Settings.read(stream); + settings.gremlinPool = 1; + return settings; + } + + @Test + public void shouldAutoCreateTraversalSourceForSingleGraph() { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + // graph named "graph" should auto-create traversal source named "g" + assertThat(serverGremlinExecutor.getGraphManager().getTraversalSource("g"), is(notNullValue())); + } + + @Test + public void shouldAutoCreateTraversalSourceWithPrefixForNonDefaultGraph() { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + settings.graphs.put("myGraph", settings.graphs.get("graph")); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + // "graph" -> "g", "myGraph" -> "g_myGraph" + assertThat(serverGremlinExecutor.getGraphManager().getTraversalSource("g"), is(notNullValue())); + assertThat(serverGremlinExecutor.getGraphManager().getTraversalSource("g_myGraph"), is(notNullValue())); + } + + @Test + public void shouldNotAutoCreateTraversalSourceWhenExplicitEntryExists() { + final Settings settings = readSettings("gremlin-server-with-traversal-sources.yaml"); + // this YAML has explicit traversalSources for "g" referencing "graph" + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + // "g" should exist from the explicit config + assertThat(serverGremlinExecutor.getGraphManager().getTraversalSource("g"), is(notNullValue())); + // "g_graph" should NOT exist because "graph" had an explicit traversalSources entry + assertThat(serverGremlinExecutor.getGraphManager().getTraversalSource("g_graph"), is(nullValue())); + } + + @Test + public void shouldInstantiateLifecycleHooksFromYaml() { + final Settings settings = readSettings("gremlin-server-with-traversal-sources.yaml"); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + assertThat(serverGremlinExecutor.getHooks().size(), is(2)); + assertThat(serverGremlinExecutor.getHooks().get(0) instanceof TinkerFactoryDataLoader, is(true)); + assertThat(serverGremlinExecutor.getHooks().get(1) instanceof TinkerFactoryDataLoader, is(true)); + } + + @Test + public void shouldHaveEmptyHooksWhenNoneConfigured() { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + settings.lifecycleHooks.clear(); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + assertThat(serverGremlinExecutor.getHooks().isEmpty(), is(true)); + } + + @Test + public void resolveLanguageShouldReturnExplicitLanguage() throws Exception { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + final Method resolveLanguage = ServerGremlinExecutor.class.getDeclaredMethod("resolveLanguage", String.class); + resolveLanguage.setAccessible(true); + + assertThat(resolveLanguage.invoke(serverGremlinExecutor, "gremlin-groovy"), is("gremlin-groovy")); + } + + @Test + public void resolveLanguageShouldFallBackToGremlinLangWhenNoExplicitLanguage() throws Exception { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + // minimal YAML has no scriptEngines, so constructor default is just gremlin-lang + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + final Method resolveLanguage = ServerGremlinExecutor.class.getDeclaredMethod("resolveLanguage", String.class); + resolveLanguage.setAccessible(true); + + assertThat(resolveLanguage.invoke(serverGremlinExecutor, (String) null), is("gremlin-lang")); + assertThat(resolveLanguage.invoke(serverGremlinExecutor, ""), is("gremlin-lang")); + } + + @Test + public void resolveLanguageShouldUseSingleNonLangEngine() throws Exception { + final Settings settings = readSettings("gremlin-server-minimal.yaml"); + settings.scriptEngines.put("gremlin-groovy", new Settings.ScriptEngineSettings()); + serverGremlinExecutor = new ServerGremlinExecutor(settings, null, null); + + final Method resolveLanguage = ServerGremlinExecutor.class.getDeclaredMethod("resolveLanguage", String.class); + resolveLanguage.setAccessible(true); + + // with one non-gremlin-lang engine configured, should resolve to it + assertThat(resolveLanguage.invoke(serverGremlinExecutor, (String) null), is("gremlin-groovy")); + } +} diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/TinkerFactoryDataLoaderTest.java b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/TinkerFactoryDataLoaderTest.java new file mode 100644 index 0000000000..9a06fd29b5 --- /dev/null +++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/util/TinkerFactoryDataLoaderTest.java @@ -0,0 +1,216 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.tinkerpop.gremlin.server.util; + +import org.apache.tinkerpop.gremlin.server.GraphManager; +import org.apache.tinkerpop.gremlin.structure.Graph; +import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph; +import org.junit.Test; +import org.slf4j.LoggerFactory; + +import java.util.HashMap; +import java.util.Map; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.greaterThan; +import static org.hamcrest.Matchers.is; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +public class TinkerFactoryDataLoaderTest { + + private LifeCycleHook.Context createContext(final GraphManager graphManager) { + return new LifeCycleHook.Context(LoggerFactory.getLogger(TinkerFactoryDataLoaderTest.class), graphManager); + } + + @Test(expected = IllegalArgumentException.class) + public void initShouldThrowWhenGraphMissing() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("dataset", "modern"); + loader.init(config); + } + + @Test(expected = IllegalArgumentException.class) + public void initShouldThrowWhenDatasetMissing() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + loader.init(config); + } + + @Test(expected = IllegalArgumentException.class) + public void initShouldThrowWhenConfigEmpty() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + loader.init(new HashMap<>()); + } + + @Test + public void onStartUpShouldHandleMissingGraph() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "nonexistent"); + config.put("dataset", "modern"); + loader.init(config); + + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("nonexistent")).thenReturn(null); + + // should not throw — just logs a warning + loader.onStartUp(createContext(gm)); + } + + @Test + public void onStartUpShouldHandleNonTinkerGraph() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "modern"); + loader.init(config); + + final Graph mockGraph = mock(Graph.class); + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(mockGraph); + + // should not throw — just logs a warning + loader.onStartUp(createContext(gm)); + } + + @Test + public void onStartUpShouldHandleUnknownDataset() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "bogus"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + // should not throw — just logs a warning + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), is(0)); + } finally { + graph.close(); + } + } + + @Test + public void onStartUpShouldLoadModern() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "modern"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), is(6)); + assertThat((int) graph.traversal().E().count().next().longValue(), is(6)); + } finally { + graph.close(); + } + } + + @Test + public void onStartUpShouldLoadClassic() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "classic"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), is(6)); + } finally { + graph.close(); + } + } + + @Test + public void onStartUpShouldLoadCrew() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "crew"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), greaterThan(0)); + } finally { + graph.close(); + } + } + + @Test + public void onStartUpShouldLoadGrateful() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "grateful"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), greaterThan(0)); + } finally { + graph.close(); + } + } + + @Test + public void onStartUpShouldLoadSink() { + final TinkerFactoryDataLoader loader = new TinkerFactoryDataLoader(); + final Map<String, Object> config = new HashMap<>(); + config.put("graph", "graph"); + config.put("dataset", "sink"); + loader.init(config); + + final TinkerGraph graph = TinkerGraph.open(); + try { + final GraphManager gm = mock(GraphManager.class); + when(gm.getGraph("graph")).thenReturn(graph); + + loader.onStartUp(createContext(gm)); + assertThat((int) graph.traversal().V().count().next().longValue(), greaterThan(0)); + } finally { + graph.close(); + } + } +} diff --git a/gremlin-server/src/test/resources/conf/remote-objects.yaml b/gremlin-server/src/test/resources/conf/remote-objects.yaml index a0f3318768..37e1caaa8d 100644 --- a/gremlin-server/src/test/resources/conf/remote-objects.yaml +++ b/gremlin-server/src/test/resources/conf/remote-objects.yaml @@ -23,8 +23,7 @@ # - docker/gremlin-server/conf/remote-objects.yaml # # Without such changes, the test docker server can't be started for independent -# testing with Gremlin Language Variants. Note this file's relation to -# gremlin-server/src/test/resources/scripts/test-server-start.groovy +# testing with Gremlin Language Variants. ############################################################################### hosts: [localhost] diff --git a/gremlin-server/src/test/resources/conf/remote-objects.yaml b/gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-minimal.yaml similarity index 52% copy from gremlin-server/src/test/resources/conf/remote-objects.yaml copy to gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-minimal.yaml index a0f3318768..e4cd0fe943 100644 --- a/gremlin-server/src/test/resources/conf/remote-objects.yaml +++ b/gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-minimal.yaml @@ -15,18 +15,7 @@ # specific language governing permissions and limitations # under the License. -############################################################################### -# IMPORTANT -############################################################################### -# Changes to this file need to be appropriately replicated to -# -# - docker/gremlin-server/conf/remote-objects.yaml -# -# Without such changes, the test docker server can't be started for independent -# testing with Gremlin Language Variants. Note this file's relation to -# gremlin-server/src/test/resources/scripts/test-server-start.groovy -############################################################################### - -hosts: [localhost] -port: 45940 -serializer: { className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4 } \ No newline at end of file +host: localhost +port: 8182 +graphs: { + graph: conf/tinkergraph-empty.properties} diff --git a/gremlin-server/src/test/resources/conf/remote-objects.yaml b/gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-with-traversal-sources.yaml similarity index 52% copy from gremlin-server/src/test/resources/conf/remote-objects.yaml copy to gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-with-traversal-sources.yaml index a0f3318768..d1f300a520 100644 --- a/gremlin-server/src/test/resources/conf/remote-objects.yaml +++ b/gremlin-server/src/test/resources/org/apache/tinkerpop/gremlin/server/gremlin-server-with-traversal-sources.yaml @@ -15,18 +15,13 @@ # specific language governing permissions and limitations # under the License. -############################################################################### -# IMPORTANT -############################################################################### -# Changes to this file need to be appropriately replicated to -# -# - docker/gremlin-server/conf/remote-objects.yaml -# -# Without such changes, the test docker server can't be started for independent -# testing with Gremlin Language Variants. Note this file's relation to -# gremlin-server/src/test/resources/scripts/test-server-start.groovy -############################################################################### - -hosts: [localhost] -port: 45940 -serializer: { className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV4 } \ No newline at end of file +host: localhost +port: 8182 +graphs: { + graph: conf/tinkergraph-empty.properties} +traversalSources: { + g: {graph: graph}, + gReadOnly: {graph: graph, query: "g.withStrategies(ReadOnlyStrategy)"} +lifecycleHooks: + - { className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader, config: {graph: graph, dataset: modern}} + - { className: org.apache.tinkerpop.gremlin.server.util.TinkerFactoryDataLoader, config: {graph: graph, dataset: classic}}
