GitHub user xinyuiscool opened a pull request:
https://github.com/apache/samza/pull/127
SAMZA-1204: Visualize StreamGraph and ExecutionPlan
First look: https://xinyuiscool.github.io/visualizer/plan.html. This is
based on the example graph JSON generated in TestJobGraphJsonGenerator.
For the api-test job:
For the join-test job:
How to visualize the plan (locally):
Once a Samza application (using fluent API) is deployed, an execution plan
will be generated by the ExecutionPlanner. The plan JSON will be written to a
file (plan.json) under the bin directory, which also contains the plan.html and
javscripts (js folder). So the bin dir has the following for visualization:
./bin
- plan.json
- plan.html
- /js
- d3.js
- dagre.js
- planToDagre.js
Then deployed locally, the user can directly view the plan by enter the
following in a browser:
**file:///my/samza/app/dir/bin/plan.html**
To visualize the plan remotely, the user can either scp the files into
local box and view the plan there, or bring up a simple http server to serve
the page.
In the future when we have the Samza dashboard, we can add the plan
visualization there to avoid the above manual steps.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xinyuiscool/samza SAMZA-1204
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/127.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #127
----
commit 0079c6d78e22de82c2857b20d8ae5e7436d38efa
Author: Xinyu Liu <[email protected]>
Date: 2017-04-10T23:53:19Z
SAMZA-1204: Visualize StreamGraph and ExecutionPlan
commit beaf1bde2dd2acbf891639ffd1d06a69e38ac19d
Author: Xinyu Liu <[email protected]>
Date: 2017-04-11T16:04:38Z
Minor adjustment to html
commit 40cf47d2cf08092acd4ddc540672ffce06768644
Author: Xinyu Liu <[email protected]>
Date: 2017-04-12T17:22:14Z
Add support to operator call trace
commit d78be41bd243c6fc5cafd95aaea289eac34b511f
Author: Xinyu Liu <[email protected]>
Date: 2017-04-15T01:12:32Z
Generate plan jon in the application runner
commit 16e3a4c6ee9d440418a6f40faefc1bbcafe9b893
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T19:03:28Z
Further enhancement of the visualizer
commit 6eca9f85947e25bd40617eb97a0bd87ae42d77fe
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T19:12:28Z
Merge branch 'master' into SAMZA-1204
commit 95d89211704e264533de6e0e9463f8f6d1bdc417
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T19:46:40Z
Add application name and id
commit d3c2d4249b3948bfaf12828cbdb9b3a47b609157
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T21:47:54Z
Update page title
commit 5e40fe5d3e17a5138b6ae7bcd64eeb28b1717fb5
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T23:00:11Z
Fetch partition info for json
commit 7166d690856451d5161d430c5347726953aa4f2a
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T23:26:36Z
Remove sample json
commit 7f447391e2b6a0eca04ac3990ad14dc015250a4f
Author: xinyuiscool <[email protected]>
Date: 2017-04-17T23:32:25Z
Use minified dagre
commit de3d43962306f5a213a83884c6d604cde821ada8
Author: Xinyu Liu <[email protected]>
Date: 2017-04-18T17:24:02Z
Clean up the scripts
commit 1459ba93a509441de3d8edde32ec4e16b9be3df5
Author: Xinyu Liu <[email protected]>
Date: 2017-04-18T17:24:57Z
Merge branch 'SAMZA-1204' of https://github.com/xinyuiscool/samza into
SAMZA-1204
commit 4039ba2f9f36faa004b930887a804f7907b63594
Author: Xinyu Liu <[email protected]>
Date: 2017-04-18T19:14:44Z
Clean up based on gradle build
commit 54325ab7fa9484ad442b68a7e32a59358f2beb83
Author: Xinyu Liu <[email protected]>
Date: 2017-04-18T19:22:37Z
Small javadoc change
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
