Drago, It sounds like you convinced me to give D3.js a second chance :). I'll experiment with what can be achieved using force-directed graph layout combined with some composable svg object, hopefully this will save me from placing objects on the canvas on my own.
I've read the barricade_Spec.js several times and part of barricade.js. Code is very interesting and allowed me to refresh some JavaScript knowledge I used a looong ago :). The topic I definitely haven't fully grasped is deferred/deferrable/referenced objects. The thing I've understood is that if some scheme includes '@ref' key, then it tries to get value returned from the resolver function no matter what value has provided during scheme instantiation. Am I right? Is 'needs' section required for the value to be resolved? The examples in file with tests are a bit mind-bending, so I failed to imagine how it works for real use-cases. Also I'm interested whether it is possible to define a schema that allows both to provide the value directly and via reference? Among other things that inclined me to give some feedback are: * '@type' vs. '@class' - is the only difference between them that '@type' refers to primitive type and '@class' refers to Barricade.js scheme? Perhaps it could be reduced to a single word to make things simpler? * '?' vs '*' - seems they are used in different contexts, '?' is for Object and '*' for Array - are 2 distinct markers actually needed? * Is it better for objects with fixed schema to fail when unexpected key is passed to them? Currently they don't. * Pushing an element of wrong type into an arraylike scheme still creates an element with empty default value. * Is it possible to create schemas with arbitrary default values (the example from spec caused me to think that default values cannot be specified)? * 'required' property does not really force such key to be provided during schema instantiation - I presume this is going to change when the real validation arrives? * What are the conceptual changes between objects instantiated from mutable (with '?') and from immutable (with fixed keys) schema? Thank you very much for your efforts, I think that Barricade.js could provide a solid foundation for Merlin! On Thu, Aug 28, 2014 at 9:31 PM, Drago Rosson <drago.ros...@rackspace.com> wrote: > Timur, > > Composable entities can be a real need for Heat if provider templates > (which allow templates to be used as a resource, with a template’s > parameters and outputs becoming properties and attributes, respectively) > are to be included in the app. A provider template resource, since it is a > template itself, would be composed of resources which would require a > composable entity. What is great about D3’s force graph is that it’s nodes > and links can be completely arbitrary - meaning they can be any JavaScript > object (including an SVG or DOM element). Additionally, the force graph > simulation updates x and y properties on those elements and calls a > user-defined “tick” function. The tick function can use the x and y > properties in any way it wants to do the *actual* update to the position > of each element. For example, this is how multiple foci can be implemented > [1]. Lots of other customization is available, including starting and > stopping the simulation, updating the node and link data, and having > per-element control of most (all?) properties such as charge or link > distance. > > Composability can be achieved using SVG’s <g> elements to group multiple > graphical elements together. The tick function would need to update the > <g>’s transform attribute [2]. This is how it is done in my app since my > nodes and links are composed of icons, labels, backgrounds, etc. I think > that D3’s force graph is not a limiting factor since it itself does not > concern itself with graphics at all. Therefore, the question seems to be > whether D3 can do everything graphically that Merlin needs. D3 is not a > graphics API, but it does have support for graphical manipulation, > animations, and events. They have sufficed for me so far. Plus, D3 can do > these things without having to use its fancy data transformations so it > can be used as a low-level SVG library where necessary. D3 can do a lot > [3] so hopefully it could also do what Merlin needs. > > You are in luck, because I have just now open-sourced Barricade! Check it > out [4]. I am working on getting documentation written for it but to see > some ways it can be used, look at its test suite [5]. > > [1] http://bl.ocks.org/mbostock/1021953 > [2] node.attr("transform", function (d) { > return "translate(" + d.x + ", " + d.y + ")"; > }); > [3] http://christopheviau.com/d3list/ > [4] https://github.com/rackerlabs/barricade > > [5] > https://github.com/rackerlabs/barricade/blob/master/test/barricade_Spec.js > > On 8/28/14, 10:03 AM, "Timur Sufiev" <tsuf...@mirantis.com> wrote: > >>Hello, Drago! >> >>I'm extremely interested in learning more about your HOT graphical >>builder. The screenshots you had attached look gorgeous! Yours visual >>representation of Heat resources is much more concise and simple than >>I had drawn in Merlin PoC mock-ups [1]. On the other hand I have some >>suspicions that D3.js is a good fit for a general purpose UI toolkit >>Merlin aims to provide. Please don't get me wrong, D3.js is a great >>library which can do fantastic things with data - in case your >>data<->visualization use-case maps to the one of the facilities D3.js >>provides out of the box. In case it doesn't, there are 2 options: >>either change your approach to what should be visualized/how it should >>be visualized, or tweak some inner machinery of D3.js >> >>While bending the design towards the facilities of D3.js doesn't seem >>a viable choice, changing D3.js from inside can be painful too. AFAIK >>force-directed graph layout from D3.js doesn't provide the means to >>represent composable entities (which isn't a big problem for Heat, but >>is a very serious issue for Murano) out of the box. By composable I >>mean something like [2] - but with much more complex inner structure >>(imagine the Resource entity [3] having as its properties other >>Resource entities which are shown as simple rounded rectangles with >>labels on that picture, but are expanded into complex objects similar >>to [3] once the user, say, clicks on them). As far as I understand, >>you are visualizing that kind of composition via arrow links, but I'd >>like to try another design options (especially in case of Murano) and >>fear that D3.js will constrain me here. I've been thinking a bit about >>using more low-level SVG js-framework like Raphael.js - it doesn't >>offer most of the goodies D3.js does, but also it doesn't force me to >>create the design based on some data transformations in a way that >>D3.js does, providing the good old procedural API instead. Of course, >>I may be wrong, perhaps more time and efforts invested into Merlin PoC >>would allow me to realize it (or not). > >>Yet you are totally right having stressed the importance of right tool >>for implementing the underlying object model (or JSON-wrapper as you >>called it) - Barricade.js. That's the second big part of work Merlin >>had to do, and I couldn't underestimate how it would be beneficial for >>Merlin to leverage some of the facilities that Barricade.js provides. >>I'll gladly look at the demo of template builder and Barricade. Is >>there any chance I could take a look also at the source code of >>Barricade.js, so I would better understand to which extent it suits >>Merlin’s needs? I've searched through github.com and didn't found any >>traces of Barricade.js repo, so it seems like some in-house project to >>me. What are your plans for sharing this library with the community? >> >>[1] https://wiki.openstack.org/wiki/Merlin/PoC >>[2] http://mbostock.github.io/d3/talk/20111116/pack-hierarchy.html >>[3] https://wiki.openstack.org/wiki/File:Merlin_poc_3.png >> >>On Wed, Aug 27, 2014 at 6:14 PM, Drago Rosson >><drago.ros...@rackspace.com> wrote: >>> Hello Timur, >>> >>> I have been developing a graphical Heat Orchestration Template builder. >>>I >>> first heard about the Merlin project around a week ago and found that >>>what >>> I have developed in the past few months is in line with what Merlin is >>> trying to accomplish. >>> >>> Some of the features of the template builder (described further down) >>> could be used as part of the UI framework Merlin aims to produce. >>>However, >>> its most important and useful component is a JavaScript library I have >>> developed called Barricade.js, which powers the template builder’s data >>> layer. >>> >>> Barricade.js aims to solve the problem of using JSON data across a web >>> app. It creates data model objects out of JSON using a predefined schema >>> (which looks similar to the schema used in your Murano Dynamic UI). This >>> removes the disadvantages with JSON and introduces some very important >>> advantages: >>> >>> - Encapsulation. JSON values can be set to any type or deleted entirely, >>> which either causes errors when UI components expect these values to >>>exist >>> or be of a certain type, or forces the UI components to constantly check >>> for correctness. Barricade instead wraps around the JSON and provides >>> accessor methods to ensure type-safe data manipulation. Additionally, >>> Barricade objects are observable, so changes made to their data trigger >>> events that can be subscribed to by UI components. >>> >>> - Normalization. Whenever properties that are defined in the schema are >>> missing in the JSON, Barricade will fill them in with default values. >>>This >>> way, UIs will always have valid values where they expect them, making >>> their design much simpler. Optional properties are extremely common in >>> Heat templates. >>> >>> - Metadata. Anything extra attached to JSON must be handled carefully >>> (such as when converting back to the original YAML format). By wrapping >>> the JSON with Barricade, metadata and convenience methods that UI >>> components can use can be defined. For instance, the datatype of any >>>value >>> or a description to go along with each property in a Heat resource. >>> >>> - Validation. Soon, the schema will allow defining validators that will >>> run whenever a new value is attempted to be set on data. Messages about >>> failed validation will be available so that the UI can display it. This >>> system seems to be very similar to dynamic UI’s. >>> >>> What this all boils down to is that all of the logic required to ensure >>> JSON’s integrity is rolled into Barricade instead of sprinkled into all >>>of >>> the UI code. This way, UI components can be confident in the data that >>> they are working with, which makes their code more concise and faster to >>> develop. >>> >>> Barricade seems like a great fit for Merlin because the projects it >>> targets (Heat, Solum, Mistral, Murano) use YAML files that can be used >>> with Barricade once they are converted to JSON. >>> >>> >>> About the template builder (see screenshots attached): >>> >>> - Uses an interactive canvas (powered by a D3.js force-directed graph) >>>to >>> display Heat resources and the dependencies in between them. New >>>resources >>> can be drag-and-dropped from a panel onto the canvas. Different resource >>> types are attracted to different respective areas so that they >>> automatically arrange themselves in a familiar network topology >>>hierarchy. >>> A form/panel is displayed when a resource on the canvas is clicked so >>> that the resource’s properties can be edited. >>> >>> - Has two―way data binding provided by Knockout.js interfacing with >>> Barricade.js so that editing values in forms automatically updates the >>> topology on the canvas (e.g. Changing resource IDs or creating/removing >>> dependencies between resources). >>> >>> - Allows for direct-editing of the template (via text input) and loading >>> templates via URLs. >>> >>> >>> Please, let me know if you would like a demo of the template builder and >>> Barricade! >>> >>> Thanks, >>> Drago Rosson >>> >>> >>> >>> >>> On 8/18/14, 5:19 AM, "Timur Sufiev" <tsuf...@mirantis.com> wrote: >>> >>>>David, >>>> >>>>I'm happy to hear that :)! After thinking a bit, I came up with the >>>>following strategy for further Merlin development: make all the >>>>commits into a separate repository (stackforge/merlin) at least until >>>>the PoC is ready. This will allow to keep project history more >>>>granular instead of updating one large commit inside openstack/horizon >>>>gerrit (thus also lessening the burden on Horizon reviewers). Once the >>>>Merlin proceeds from the experimental/PoC phase to the implementing of >>>>a more elaborated spec, it will be just the time for it to join with >>>>the Horizon. >>>> >>>>On Wed, Aug 13, 2014 at 2:48 AM, Lyle, David <david.l...@hp.com> wrote: >>>>> On 8/6/14, 1:41 PM, "Timur Sufiev" <tsuf...@mirantis.com> wrote: >>>>> >>>>>>Hi, folks! >>>>>> >>>>>>Two months ago there was an announcement in ML about gathering the >>>>>>requirements for cross-project UI library for >>>>>>Heat/Mistral/Murano/Solum [1]. The positive feedback in related >>>>>>googledoc [2] and some IRC chats and emails that followed convinced me >>>>>>that I'm not the only person interested in it :), so I'm happy to make >>>>>>the next announcement. >>>>>> >>>>>>The project finally has got its name - 'Merlin' (making complex UIs is >>>>>>a kind of magic), Openstack wiki page [3] and all other stuff like >>>>>>stackforge repo, launchpad page and IRC channel (they are all >>>>>>referenced in [3]). For those who don't like clicking the links, here >>>>>>is quick summary. >>>>>> >>>>>>Merlin aims to provide a convenient client side framework for building >>>>>>rich UIs for Openstack projects dealing with complex input data with >>>>>>lot of dependencies and constraints (usually encoded in YAML format >>>>>>via some DSL) - projects like Heat, Murano, Mistral or Solum. The >>>>>>ultimate goal for such UI is to save users from reading comprehensive >>>>>>documentation just in order to provide correct input data, thus making >>>>>>the UI of these projects more user-friendly. If things go well for >>>>>>Merlin, it could be eventually merged into Horizon library (I¹ll spare >>>>>>another option for the end of this letter). >>>>>> >>>>>>The framework trying to solve this ambitious task is facing at least 2 >>>>>>challenges: >>>>>>(1) enabling the proper UX patterns and >>>>>>(2) dealing with complexities of different projects' DSLs. >>>>>> >>>>>>Having worked on DSL things in Murano project before, I'm planning at >>>>>>first to deal with the challenge (2) in the upcoming Merlin PoC. So, >>>>>>here is the initial plan: design an in-framework object model (OM) >>>>>>that could translated forth and back into target project's DSL. This >>>>>>OM is meant to be synchronised with visual elements shown on browser >>>>>>canvas. Target project is the Heat with its HOT templates - it has the >>>>>>most well-established syntax among other projects and comprehensive >>>>>>documentation. >>>>>> >>>>>>Considering the challenge (1), not being a dedicated UX engineer, I'm >>>>>>planning to start with some rough UI concepts [4] and gradually >>>>>>improve them relying on community feedback, and especially, Openstack >>>>>>UX group. If anybody from the UX team (or any other team!) is willing >>>>>>to be involved to a greater degree than just giving some feedback, >>>>>>you're are enormously welcome! Join Merlin, it will be fun :)! >>>>>> >>>>>>Finally, with this announcement I¹d like to start a discussion with >>>>>>Horizon community. As far as I know, Horizon in its current state >>>>>>lacks such UI toolkit as Merlin aims to provide. Would it be by any >>>>>>chance possible for the Merlin project to be developed from the very >>>>>>beginning as part of Horizon library? This choice has its pros and >>>>>>cons I¹m aware of, but I¹d like to hear the opinions of Horizon >>>>>>developers on that matter. >>>>> >>>>> I would like to see this toolset built into Horizon. That will make it >>>>> accessible to integrated projects like Heat that Horizon already >>>>>supports, >>>>> but will also allow other projects to use the horizon library as a >>>>> building block to providing managing project specific DSLs. >>>>> >>>>> David >>>>> >>>>>> >>>>>>[1] >>>>>>http://lists.openstack.org/pipermail/openstack-dev/2014-June/037054.ht >>>>>>ml >>>>>>[2] >>>>>>https://docs.google.com/a/mirantis.com/document/d/19Q9JwoO77724RyOp7Xk >>>>>>pY >>>>>>mA >>>>>>Lwmdb7JjoQHcDv4ffZ-I/edit# >>>>>>[3] https://wiki.openstack.org/wiki/Merlin >>>>>>[4] https://wiki.openstack.org/wiki/Merlin/SampleUI >>>>>> >>>>>>-- >>>>>>Timur Sufiev >>>>>> >>>>>>_______________________________________________ >>>>>>OpenStack-dev mailing list >>>>>>OpenStack-dev@lists.openstack.org >>>>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>>> _______________________________________________ >>>>> OpenStack-dev mailing list >>>>> OpenStack-dev@lists.openstack.org >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> >>>>-- >>>>Timur Sufiev >>>> >>>>_______________________________________________ >>>>OpenStack-dev mailing list >>>>OpenStack-dev@lists.openstack.org >>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> >> >>-- >>Timur Sufiev >> >>_______________________________________________ >>OpenStack-dev mailing list >>OpenStack-dev@lists.openstack.org >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur Sufiev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
