Awesome. Sounds like the editor already does a whole bunch of what you need. I was wondering if you'd want to even decouple the editor from XML, which is a goal, and not too far off, but seems like you don't need that so that's even less work.
I really hate using email for this kind of thing... :) but here goes: Στις 29 Απριλίου 2011 2:00 π.μ., ο χρήστης Thomas Broyer <[email protected]> έγραψε: > Hi Dan, > > It's really great to see several of you knowledgeable Googlers showing > interest! > > On Thu, Apr 28, 2011 at 6:54 AM, Daniel Danilatos <[email protected]> > wrote: >> Hi Thomas, >> >> As Pat said, it's great to hear you've decided the editor is a good >> fit for your project. I'd love to work with you in making the editor >> more standalone. If you are OK with it, could you send out a more >> detailed proposal of the use cases you need from the editor; then we >> could directly help you quickly identify the parts of the code that >> would need improvement in order to achieve your goals as soon as >> possible. > > What's in our specs (that our client said they want): > - a rich text editor with both "graphical", "structural" and > "semantic" markup (see below). > - the documents have to be stored in (or easily converted to) XML > with strict schemas (looks a bit like DocBook); we actually don't have > any constraint re. the storage format So, annotations out of the question? They really do solve many problems in a nicer way than XML, even without the need for concurrent editing. XML is great for structure; annotations are great for... annotating. It is also entirely possible to use annotations internally, and then export to XML. Even with with say things like bold and strikethrough, if you choose to implement them directly as elements, it's going to be a royal PITA to deal with nicely. > - text will have to be spell-checked, but we can very well turn into > a specific mode for that where editing would be turned of (like GMail, > or the spell check wizard in MSWord/LibreOffice Writer). Particularly, > we don't need real-time spellchecking, we can very well send the whole > doc to the server, get errors back, and process them. Similarly, we'll > do some basic "grammar checking" (correct usage of spaces around > punctuation, sentences start with upper-case letter; that's it.) > - "graphical" markup includes: bold, italic, underline and > line-through (yes, they consider this graphical, not semantic, even > though in other discussions they said that if something is bold, it's > for a reason, so they nevertheless attach some kind meaning to > "something being bold") > - "structural" markup includes: > - paragraph, heading (levels 1-5) and unordered list (with nesting; > lack of ordered list might be an oversight) > - tables: with caption, add/remove column/row, merge/split cells > - notes (I don't have any more details for now, I don't know if > they want "foot notes" –or HTML's <span title="">– or rather a > "decorated paragraph" –or group of paragraphs–) > - links > - images (with caption; à la HTML5's <figure>); these are linked to > "illustration" domain objects (including the caption). In the text, > the user can define the size of the thumbnail, how text flows around > the figure (float), and change the caption's text (could be done in, > say, a popup, not necessarily by directly editing the text displayed > in the editor) Image thumbnails pretty much do this already, except for the arbitrary size, but that should be simple to add. The rest is pretty much already implemented. > - "semantic" markup is > - mostly about attaching a domain-specific term/category to the > selected text, e.g. this is a person, date, building material, etc. > (there are something like 50 thesaurus/list-of-values to choose from; > the user "attaches" the selected text to one such thesaurus, and can > optionally select a specific value from the list) I don't know if > annotations would be the best fit here: we don't have the constraint > of running through several paragraphs, but it's likely that, despite > not spec'd, such semantic markup shouldn't overlap; so elements could > be used; plus, the value being optional could make it impractical to > distinguish "start of annotation without value" from "end of > annotation" (if we use the DocInitialization XML representation). annotation values are opaque strings, so you can choose to interpret them however you like. if a particular annotation has no value, but is merely a marker, then an empty string, or single letter, would suffice as marking it "present" (null being absent). > - there are also 10 cases where the annotation isn't linked to any > thesaurus (e.g. marking up a birth date, a person's given name, a > book's publish date, etc.), in this case, we could have a single > "kind" with 10 possible values (i.e. kind="birthdate"). not sure i understand - do you want an annotation with multiple values at once, or more like an enum i.e. can take on one of several possible values? (both doable, several strategies depending on use case specifics). > > Aside of the editor will be outlines: > - headings hierarchy > - tables > - figures If you want these to be in the editor, it's straight forward to write "doodads". There's a tutorial up on the wiki somewhere. Not sure where the documentation lives now, if it's been migrated to confluence now or what. (There's also a mostly-written editor design doc, you may have come across it). > - "semantic" markup (grouped by category/type) > I was thinking about periodically (every 5secs or so) scanning the > document to extract those information along with their location > (instead of trying to keep everything up-to-date at each modification) > When an item is clicked, the caret should be moved at said location > and then scrolled into view if needed. easy > > This editor will be used in a dozen places (fyi, we have something > like 100 different "forms"), with different features: > - "graphical" markup is always "on" (I thought there was one case > where it didn't, but re-reading our specs, it seems I was wrong) > - "structural" markup can be disabled (i.e. a single line/paragraph; > looks like the LinoTextEventHandler could help here), partially > enabled (paragraphs, headings and figures only; no text-align, list, > table or link; there's also one case where there are only paragraphs > and headings) or fully enabled (in a couple places only) > - "semantic" markup is always on, but the set of thesaurus/LoVs to > choose from is different each time. perfect - you just register the right set of doodads for the features you want enabled. think of the editor not as an editor, but as "editor tools". you build your editor by composing together the bits you want. put all the bits in for the fully featured version, put only some bits for the simpler version, etc. > There are two cases where the edited document can (theoretically) be > very large ("up to 80 pages"), and obviously these are the cases where > all the features are enabled: think domain-specific Google Docs! currently the editor has no pagination, i.e. it holds the whole document in memory. so there would be upper limits (though i've tested it on quite huge documents and it worked fine). > > I thought about Wave's editor because it is "schema-based" and because > of the "structural markup" constraints, particularly when pasting > things coming from, e.g. MSWord (something that's likely to be common > in our case). We specifically don't need OT or collaborative editing, > "just" a rich-text editor with the special features described above. no problem. the copy/paste code atm is the most hard coded part of the editor. we'd need to make it more pluggable so you can customize how it converts pasted rich text html into whatever data model you have. the good news is, it already supports "semantic" copy/paste, which means that if you copy between two wave editors, it won't get degraded by going through an intermediate clipboard html format - we preserve the original annotations, elements, attributes, the lot. > > To limit the amount of work, I though about using the Wave's way of > marking up <line/>s, at least at the beginning; but maybe it'd be easy > to use a more HTML-like <p>...</p> and <h1>...</h1> kind of markup? > (and keeping <?a ?> annotations for styling) At first sight, it looks > like the Editor is quite tightly bound to <line/>s, but maybe it's > just me? There may be a couple of hard coded bits here and there but it's definitely not tightly bound, in fact, it used to be that we had <p>...</p> and <h1>...</h1>, but we moved away from it for OT reasons. There is still some unused code in there for working with paragraphs (though most has been deleted). > > If you could give me hints about how best to: > - initialize an Editor (use an "owned" ContentDocument or not? what > should I register in the registries?) > - put content into, and get content out of the editor: should I > rather use the "persistent document" (and if yes, which class: > PersistentContentDoc, ContentRawDocument? I must confess I'm lost with > all these classes and interfaces) or a "doc initialization"? > - make my own schema(s), and how to map the "structural" and > "semantic" markup (elements? annotations?) > It'd help me start customizing the editor to my needs and see what > could be improved to make it a standalone component (starting with > writing some docs! ;-) ) i don't know if these have already been migrated to confluence, but here's something better than nothing: http://www.waveprotocol.org/code/tutorials/writing-a-doodad http://www.waveprotocol.org/protocol/design-proposals/editor http://danilatos.com/Wave_Summit_2010_Editor.pdf (this might be published somewhere else properly, but i couldn't find it, so i just put it here for now). The above should give you some idea, but a few pointers: The persistent view is what is shared with the server and hence what is affected by operations. It is updated using MutableDocument, whose mutator methods map to the underlying document operations. The "full" view contains all the extra local nodes (see presentation slides for more info). These are unaffected by operations; as such, they have a different mutation api for modifying them (and it's more powerful, because the restrictions of operations don't apply). Annotations have something similar, there's local annotations which can have any java object as their value, whereas persistent ones must have string values. It's unlikely that you'll want to touch "Raw" documents, they are an inner implementation detail. If in doubt, do perform all accesses and mutations through something that implements MutableDocument. Probably CMutableDocument, which just lets you avoid generics and use the concrete classes ContentNode, ContentElement, etc. I have some more documentation and diagrams lying around but scared up update the wiki in case i should be updating confluence instead. I haven't been following the latest state of affairs on this, anyone know? Below is a copy/paste of some documentation Pat did for all the interfaces. It might help a bit. Around the time wave got... de-staffed, we were in the process of drastically simplifying this list, but unfortunately that's now unlikely to happen soon. Though if you're keen, we might have enough combined momentum to work together on some improvements. The good news is that most of the various interfaces are just minor variations on eachother, providing incrementally more capabilities. See how you go, I'm happy to answer more questions or help directly with coding. ------------------------ Definitions: Document: throughout this, 'Document' will refer to a DOM tree, possibly with annotations. They are parameterised (in Java through generics) by three types: <N, E extends N, T extends N> representing the Node, Element and TextNode types the document uses in its tree. Doc.N, Doc.E, Doc.T: vacuous interfaces If all node/element/text node implementations extend these, we can use the non-generic versions of interfaces (e.g. "Document", which extends "MutableDocument<Doc.N, Doc.E, Doc.T>"), for simplicity. Point<N> Representation for locations within the DOM tree, one of: (Parent Element, Node after), representing the location just before the 'after' node. If after is null, it is the location at the end of the parent (just before the closing tag). (Text node, int offset), representing a character location within a text node. The precise location is just before the offset-th character within the node, or at the end if offset is the length of the node. ReadableDocument<N, E, T> For DOM implementation details (parent/sibling/child structures, element tag names & attributes, ...) we take the pattern of letting the document define these for its contents. e.g. instead of: element.getAttribute("id") Instead each call should take place within the context of a document, so this is written: document.getAttribute(element, "id") This allows a single Doc.N node to conceptually exist in multiple documents, and have a different hierarchy position in each. ReadableDocumentView<N, E, T> (extends ReadableDocument<N, E, T>) Adds the concept of 'visibility' to nodes within the document, by presenting methods for visible definitions and traversal. This allows easy creation of a subset of the nodes, still usable as a document, but only requiring a thin layer over an already available document. TODO; contract? TODO: image? Filtered View<N, E, T> (implements ReadableDocumentView<N, E, T>) Allows simple creation of a view, by taking a document and defining a 'Skip' filter on the nodes in the document. Each node can be: Skip.NONE = node is visible in the filtered view. Skip.SHALLOW = node is invisible, but some of its subtree may be visible. Skip.DEEP = node and all descendants are invisible. Skip.INVALID = node is not part of the document at all. [not used?] RawDocument<N, E, T> (extends ReadableDocument<N, E, T>) Extends a readable document by adding methods to create/update/move/remove the nodes, using a similar API to the W3C DOM. Additionally, exposes the concept of Indexing containers, used for indexing the nodes within the document, and a TextNodeOrganiser to manage implementation-specific mutations of text nodes. ReadableAnnotationSet<V> API for reading information about annotations over a range, with String keys and values of type V. Methods exposed include things like iteration over annotations, intervals or keys within a certain range in the document, and finding places where annotation values change. LocationMapper<N> An item which translates between Point<N> within a document and integer offsets from the start of the document. This allows locations within a document to be handled more simply as integers, or more semantically as document points. ReadableWDocument<N, E, T> (extends ReadableDocument<N, E, T>, ReadableAnnotationSet<String>, LocationMapper<N>) Represents a complete annotated DOM document, by uniting a readable DOM document, a set of (String, String) annotations and a mapping of offsets between the two. Also added in this interface are a number of serialisation methods, both to Strings and also to a wave operation - it is at this is the layer where operation are introduced to documents. IndexedDocument<N, E, T> (extends ReadableWDocument<N, E, T>, TextNodeOrganiser<T>, SuperSink) A readable annotated DOM document, which has the ability to be modified through consuming operations. MutableAnnotationSet<V> (extends ReadableAnnotationSet<V>) Extends the readable annotation set to provide mutation capabilities, i.e. setting and resetting annotations over a range. Two V values are provided for convenience: .Persistent (mapping to Strings) and .Local (mapping to Objects). MutableDocument<N, E, T> (extends ReadableWDocument<N, E, T>, MutableAnnotationSet<String>) Combines a readable document with mutable annotations, and adds methods to mutate the DOM parts of the document. Use this API for an operation-backed, fully readable and writable DOM document with String-valued annotations.
