GitHub user 1ambda reopened a pull request:
https://github.com/apache/zeppelin/pull/2098
[ZEPPELIN-2217][WIP] AdvancedTransformation for Visualization
### What is this PR for?
`AdvancedTransformation` has more detailed options while providing existing
features of `PivotTransformation` and `ColumnselectorTransformation` which
Zeppelin already has
Here are some features which advanced-transformation can provide.
1. **(screenshot)** multiple sub charts
2. **(screenshot)** parameter widgets: `input`, `checkbox`, `option`,
`textarea`
3. **(screenshot)** expand/fold axis and parameter panels
4. **(screenshot)** clear axis and parameter panels
5. **(screenshot)** remove duplicated columns in an axis
6. **(screenshot)** limit column count in an axis
7. configurable char axes: `valueType`, `axisType`, `description`, ...
8. configurable chart parameters
9. lazy transformation
10. parsing parameters automatically based on their type: `int`, `float`,
`string`, `JSON`
11. multiple transformation methods
12. re-initialize whole configuration based on spec hash.
13. **(screenshot)** shared axis
#### API Details: Spec
`AdvancedTransformation` requires `spec` which includes axis and parameter
details for charts.
- Let's create 2 sub-charts called `simple-line` and `step-line`.
- Each sub chart can have different `axis` and `parameter` depending on
their requirements.
```js
constructor(targetEl, config) {
super(targetEl, config)
const spec = {
charts: {
'simple-line': {
sharedAxis: true, /** set if you want to share axes between sub
charts, default is `false` */
axis: {
'xAxis': { dimension: 'multiple', axisType: 'key', },
'yAxis': { dimension: 'multiple', axisType: 'aggregator'},
'category': { dimension: 'multiple', axisType: 'group', },
},
parameter: {
'xAxisUnit': { valueType: 'string', defaultValue: '',
description: 'unit of xAxis', },
'yAxisUnit': { valueType: 'string', defaultValue: '',
description: 'unit of yAxis', },
'dashLength': { valueType: 'int', defaultValue: 0, description:
'the length of dash', },
},
},
'step-line': {
axis: {
'xAxis': { dimension: 'single', axisType: 'unique', },
'yAxis': { dimension: 'multiple', axisType: 'value', },
},
parameter: {
'xAxisUnit': { valueType: 'string', defaultValue: '',
description: 'unit of xAxis', },
'yAxisUnit': { valueType: 'string', defaultValue: '',
description: 'unit of yAxis', },
'noStepRisers': { valueType: 'boolean', defaultValue: false,
description: 'no risers in step line', widget: 'checkbox', },
},
},
}
this.transformation = new AdvancedTransformation(config, spec)
}
```
#### API Details: Axis Spec
| Field Name | Available Values (type) | Description |
| --- | --- | --- |
|`dimension` | `single` | Axis can contains only 1 column |
|`dimension` | `multiple` | Axis can contains multiple columns |
|`axisType` | `key` | Column(s) in this axis will be used as `key` like in
`PivotTransformation`. These columns will be served in `column.key` |
|`axisType` | `aggregator` | Column(s) in this axis will be used as `value`
like in `PivotTransformation`. These columns will be served in
`column.aggregator` |
|`axisType` | `group` | Column(s) in this axis will be used as `group` like
in `PivotTransformation`. These columns will be served in `column.group` |
|`axisType` | (string) | Any string value can be used here. These columns
will be served in `column.custom` |
|`maxAxisCount` | (int) | The maximum column count that this axis can
contains. (unlimited if `undefined`) |
|`valueType` | (string) | Describe the value type just for annotation |
Here is an example.
```js
axis: {
'xAxis': { dimension: 'multiple', axisType: 'key', },
'yAxis': { dimension: 'multiple', axisType: 'aggregator'},
'category': { dimension: 'multiple', axisType: 'group',
maxAxisCount: 2, valueType: 'string', },
},
```
#### API Details: Parameter Spec
| Field Name | Available Values (type) | Description |
| --- | --- | --- |
|`valueType` | `string` | Parameter which has string value |
|`valueType` | `int` | Parameter which has int value |
|`valueType` | `float` | Parameter which has float value |
|`valueType` | `boolean` | Parameter which has boolean value used with
`checkbox` widget usually |
|`valueType` | `JSON` | Parameter which has JSON value used with `textarea`
widget usually. `defaultValue` should be `""` (empty string). This
||`defaultValue` | (any) | Default value of this parameter. `JSON` type should
have `""` (empty string) |
|`description` | (string) | Description of this parameter. This value will
be parsed as HTML for pretty output |
|`widget` | `input` | Use
[input](https://developer.mozilla.org/en/docs/Web/HTML/Element/input) widget.
This is the default widget (if `widget` is undefined)|
|`widget` | `checkbox` | Use
[checkbox](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox)
widget. |
|`widget` | `textarea` | Use
[textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea)
widget. |
|`widget` | `option` | Use [select +
option](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/select)
widget. This parameter should have `optionValues` field as well. |
|`optionValues` | (Array<string>) | Available option values used with the
`option` widget |
Here is an example.
```js
parameter: {
// string type, input widget
'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit
of xAxis', },
// boolean type, checkbox widget
'inverted': { widget: 'checkbox', valueType: 'boolean', defaultValue:
false, description: 'invert x and y axes', },
// string type, option widget with `optionValues`
'graphType': { widget: 'option', valueType: 'string', defaultValue:
'line', description: 'graph type', optionValues: [ 'line', 'smoothedLine',
'step', ], },
// HTML in `description`
'dateFormat': { valueType: 'string', defaultValue: '', description:
'format of date (<a
href="https://docs.amcharts.com/3/javascriptcharts/AmGraph#dateFormat";>doc</a>)
(e.g YYYY-MM-DD)', },
// JSON type, textarea widget
'yAxisGuides': { widget: 'textarea', valueType: 'JSON', defaultValue: '',
description: 'guides of yAxis ', },
```
#### API Details: Transformer Spec
`AdvancedTransformation` supports 3 transformation methods. The return
value will depend on the transformation method type.
```js
const spec = {
charts: {
'simple': {
/** default value of `transform.method` is the flatten cube. */
axis: { ... },
parameter: { ... }
},
'cube-group': {
transform: { method: 'cube', },
axis: { ... },
parameter: { ... },
}
'no-group': {
transform: { method: 'raw', },
axis: { ... },
parameter: { ... },
}
```
| Field Name | Available Values (type) | Description |
| --- | --- | --- |
|`method` | `object` | designed for [amcharts:
serial](https://www.amcharts.com/demos/date-based-data/) |
|`method` | `array` | designed for [highcharts:
column](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-basic/)
|
|`method` | `drill-down` | designed for [highcharts:
drill-down](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-drilldown/)
|
|`method` | `raw` | will return the original `tableData.rows` |
Whatever you specified as `transform.method`, the `transformer` value will
be always function for lazy computation.
```js
// advanced-transformation.util#getTransformer
if (transformSpec.method === 'raw') {
transformer = () => { return rows; }
} else if (transformSpec.method === 'array') {
transformer = () => {
...
return { ... }
}
}
```
#### Feature Details: Automatic parameter parsing
Advanced transformation will parse parameter values automatically based on
their type: `int`, `float`, `string`, `JSON`
- See also `advanced-transformation-util.js#parseParameter`
#### Feature Details: re-initialize the whole configuration based on spec
hash
```js
// advanced-transformation-util#initializeConfig
const currentVersion = JSON.stringify(spec)
if (!config.spec || !config.spec.version || config.spec.version !==
currentVersion) {
spec.version = currentVersion
// reset config...
}
```
#### Feature Details: Shared Axes
If you set `sharedAxis` to `true` in chart specification, then these charts
will share their axes. (default is `false`)
```js
const spec = {
charts: {
'column': {
transform: { method: 'array', },
sharedAxis: true,
axis: { ... },
parameter: { ... },
},
'stacked': {
transform: { method: 'array', },
sharedAxis: true,
axis: { ... }
parameter: { ... },
},
```
#### API Details: Usage in Visualization#render()
Let's assume that we want to create 2 sub-charts called `basic` and
`no-group`.
- https://github.com/1ambda/zeppelin-ultimate-line-chart (an practical
example)
```js
drawBasicChart(parameter, column, transformer) {
const { ... } = transformer()
}
drawNoGroupChart(parameter, column, transformer) {
const { ... } = transformer()
}
render(data) {
const { chart, parameter, column, transformer, } = data
if (chart === 'basic') {
this.drawBasicChart(parameter, column, transformer)
} else if (chart === 'no-group') {
this.drawNoGroupChart(parameter, column, transformer)
}
}
```
### What type of PR is it?
[Feature]
### Todos
NONE
### What is the Jira issue?
[ZEPPELIN-2217](https://issues.apache.org/jira/browse/ZEPPELIN-2217)
### How should this be tested?
1. Clone https://github.com/1ambda/zeppelin-ultimate-line-chart
2. Create a symbolic link `ultimate-line-chart.json` into
`$ZEPPELIN_HOME/helium`
3. Modify the `artifact` value to proper absolute path considering your
local machine.
4. Install the above visualization in `localhost:9000/#helium`
5. Test it
### Screenshots (if appropriate)
#### 1. *(screenshot)* multiple sub charts
#### 2. *(screenshot)* parameter widgets: `input`, `checkbox`, `option`,
`textarea`
#### 3. *(screenshot)* expand/fold axis and parameter panels
#### 4. *(screenshot)* clear axis and parameter panels
#### 5. *(screenshot)* remove duplicated columns in an axis
#### 6. *(screenshot)* limit column count in an axis
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/1ambda/zeppelin
ZEPPELIN-2217/advanced-transformation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2098.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2098
----
commit 10c80fc8beb8731ad416449350cda255b59de107
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-06T05:39:48Z
feat: AdvancedTransformation
commit 91ae54d1c151667c57186913cf03feb56a7a17d6
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-11T20:48:35Z
fix: Overflow issue in single aggr
commit 9768ecfcc576ba37c8f7f1bd12b4cf95f0c6705c
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-12T07:46:59Z
feat: Add groupBase axis option
commit a8a4fb1ac53ca106830fa7b460032775baa0ea8f
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-12T07:55:12Z
refactor: Add getAxisInSingleDimension func
commit 9fb398e92d2f7f623a8dfe37adc6fb5ef6a6bf3a
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-12T08:16:41Z
feat: Add clearConfig
commit 676bd7e422df4cfb4d7d0101a75f3d2f7b452d69
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-12T08:59:28Z
refactor: Refine transform API
commit 97be62933685d5494397f736f69f0d6c465c7371
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-12T09:06:40Z
fix: Add singleDimensionAggregatorChanged
commit e1fcc2e234b3c68e8cef347357e71c7dbed7930b
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-13T09:26:40Z
feat: Support multiple charts
commit 75569cea6e9eb5b7deb8efe428b84bcc445888e1
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-13T11:21:40Z
feat: Support multiple charts in UI
commit d89e223768aa9945fc8f1729301e93e765d95917
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-13T12:15:38Z
feat: advanced-transformation-api
commit 49985c69056603a53ffa2ca2f596c82ea58f6f04
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-14T15:49:23Z
refactor: Refine axis, param spec
commit f0ed02fff0841af306e3f52d990332fe3d6a7719
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-14T16:17:33Z
feat: Support same axis types
commit 7a454ff8218745f5d8f5150e553488bb501be207
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T12:03:10Z
feat: Cube Transformation
commit 936901b553442fb1a014e2197496457b44e524ec
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T12:48:23Z
feat: Support undefined valueType in axisSpec
commit 0484e1ee6b57fbb327ebf94d3347779f7ae44def
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T13:15:59Z
feat: Support maxAxisCount in axisSpec
commit 019892ce00309208293b66a2eacf7241ffd5b3f3
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T15:44:04Z
feat: Support transform: flatten
commit 5b88f08be1492dc045581896e571f2953548ea46
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T15:48:24Z
fix: Modify margin of subchart radio btns
commit 74b8b4e64918a2a1f3bf50998f489808fde025df
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T15:52:35Z
fix: Duplicated radio btn id, name
commit 94d837a4f7267511b787809fefdd6e8776dafeb3
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T16:35:26Z
feat: Automatic spec versioning
commit 0dbc431e1cddef6e7a2fa12daf410c04564a3309
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T17:11:17Z
feat: Support transformer
commit 53f508c2d3cfee692f7f606d550113b3b15b693e
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T18:24:20Z
feat: custom axisSpec
commit b1d9d31b970b153d11a9b67c2cade6d2cf21ffcb
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T18:35:23Z
feat: Save and close with enter key
commit 5d3efc932f9dffb04c314f7e574b471982eabf3b
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-15T18:45:55Z
fix: Change panel header
commit 11b7eaac94a807d6c62a9ed484b3e47f67f72f3b
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T10:26:38Z
feat: option widget
commit 4e73012033e8720f4227c5fcbba23e54275fc8d5
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T10:30:15Z
feat: widget checkbox
commit bf88b4f237fd99d7d3c0a4e6d8b64933f6dcd1d0
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T10:39:31Z
feat: textare widget and update hook
commit 088705b3b83c3ea716bb396b995f305aeac0d692
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T15:32:46Z
refactor: Remove util and add Widget funcs
commit b4d774cde12e341511880a42fa2fbf7bbae4a76f
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T18:03:10Z
fix: Dont close param panel when enter
commit fcc625cb0c9c5f247ebfadc2c52720bdc907e424
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T18:54:57Z
feat: Automatic param parsing
commit 9a2d227318469383fc4015d2fc2f81752a71ee76
Author: 1ambda <1am...@gmail.com>
Date: 2017-03-16T19:11:35Z
fix: Stop event propagation in widgets
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
