Thanks Mateusz! This branch compiled and ran fine for me. No noticeable difference in performance so far.
Cheers, Corey -Corey O'Connor coreyocon...@gmail.com http://corebotllc.com/ On Thu, Sep 18, 2014 at 11:00 AM, GitHub <nore...@github.com> wrote: > Branch: refs/heads/rope-integration > Home: https://github.com/yi-editor/yi > Commit: 07129dfeffc134117e47cd761a8467ed98fe84de > > https://github.com/yi-editor/yi/commit/07129dfeffc134117e47cd761a8467ed98fe84de > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-17 (Wed, 17 Sep 2014) > > Changed paths: > M yi-contrib/src/Yi/Config/Users/Amy.hs > M yi-contrib/src/Yi/Config/Users/Anders.hs > M yi-contrib/src/Yi/Config/Users/JP.hs > M yi-contrib/src/Yi/Config/Users/Michal.hs > M yi-contrib/src/Yi/Config/Users/Reiner.hs > M yi-contrib/src/Yi/FuzzyOpen.hs > M yi-contrib/src/Yi/Style/Misc.hs > M yi-contrib/src/Yi/Templates.hs > M yi-contrib/yi-contrib.cabal > R yi/src/library/Data/Rope.hs > M yi/src/library/System/CanonicalizePath.hs > M yi/src/library/Yi.hs > M yi/src/library/Yi/Boot.hs > M yi/src/library/Yi/Buffer/HighLevel.hs > M yi/src/library/Yi/Buffer/HighLevel.hs-boot > M yi/src/library/Yi/Buffer/Implementation.hs > M yi/src/library/Yi/Buffer/Indent.hs > M yi/src/library/Yi/Buffer/Misc.hs > M yi/src/library/Yi/Buffer/Region.hs > M yi/src/library/Yi/Buffer/TextUnit.hs > M yi/src/library/Yi/Buffer/Undo.hs > M yi/src/library/Yi/Command.hs > M yi/src/library/Yi/Completion.hs > M yi/src/library/Yi/Config/Default.hs > M yi/src/library/Yi/Config/Simple.hs > M yi/src/library/Yi/Core.hs > M yi/src/library/Yi/Debug.hs > M yi/src/library/Yi/Dired.hs > M yi/src/library/Yi/Editor.hs > M yi/src/library/Yi/Editor.hs-boot > M yi/src/library/Yi/Eval.hs > M yi/src/library/Yi/File.hs > M yi/src/library/Yi/History.hs > M yi/src/library/Yi/Hoogle.hs > M yi/src/library/Yi/IReader.hs > M yi/src/library/Yi/Interact.hs > M yi/src/library/Yi/Keymap/Cua.hs > M yi/src/library/Yi/Keymap/Emacs.hs > M yi/src/library/Yi/Keymap/Emacs/KillRing.hs > M yi/src/library/Yi/Keymap/Emacs/Utils.hs > M yi/src/library/Yi/Keymap/Keys.hs > M yi/src/library/Yi/Keymap/Vim.hs > M yi/src/library/Yi/Keymap/Vim/Common.hs > M yi/src/library/Yi/Keymap/Vim/Eval.hs > M yi/src/library/Yi/Keymap/Vim/EventUtils.hs > M yi/src/library/Yi/Keymap/Vim/Ex.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Buffer.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/BufferDelete.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Buffers.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Cabal.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Common.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Delete.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Edit.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Global.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/GotoLine.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Nohl.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Paste.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Quit.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Reload.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Substitute.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Tag.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Write.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Commands/Yi.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Eval.hs > M yi/src/library/Yi/Keymap/Vim/Ex/Types.hs > M yi/src/library/Yi/Keymap/Vim/ExMap.hs > M yi/src/library/Yi/Keymap/Vim/InsertMap.hs > M yi/src/library/Yi/Keymap/Vim/MatchResult.hs > M yi/src/library/Yi/Keymap/Vim/Motion.hs > M yi/src/library/Yi/Keymap/Vim/NormalMap.hs > M yi/src/library/Yi/Keymap/Vim/NormalOperatorPendingMap.hs > M yi/src/library/Yi/Keymap/Vim/Operator.hs > M yi/src/library/Yi/Keymap/Vim/ReplaceMap.hs > M yi/src/library/Yi/Keymap/Vim/ReplaceSingleCharMap.hs > M yi/src/library/Yi/Keymap/Vim/Search.hs > M yi/src/library/Yi/Keymap/Vim/SearchMotionMap.hs > M yi/src/library/Yi/Keymap/Vim/StateUtils.hs > M yi/src/library/Yi/Keymap/Vim/StyledRegion.hs > M yi/src/library/Yi/Keymap/Vim/Tag.hs > M yi/src/library/Yi/Keymap/Vim/Utils.hs > M yi/src/library/Yi/Keymap/Vim/VisualMap.hs > M yi/src/library/Yi/KillRing.hs > M yi/src/library/Yi/Main.hs > M yi/src/library/Yi/MiniBuffer.hs > M yi/src/library/Yi/MiniBuffer.hs-boot > M yi/src/library/Yi/Misc.hs > M yi/src/library/Yi/Mode/Abella.hs > M yi/src/library/Yi/Mode/Buffers.hs > M yi/src/library/Yi/Mode/Compilation.hs > M yi/src/library/Yi/Mode/GHCi.hs > M yi/src/library/Yi/Mode/Haskell.hs > M yi/src/library/Yi/Mode/Haskell/Dollarify.hs > M yi/src/library/Yi/Mode/IReader.hs > M yi/src/library/Yi/Mode/Interactive.hs > M yi/src/library/Yi/Mode/JavaScript.hs > M yi/src/library/Yi/Mode/Latex.hs > M yi/src/library/Yi/Modes.hs > M yi/src/library/Yi/Rectangle.hs > M yi/src/library/Yi/Search.hs > M yi/src/library/Yi/Snippets.hs > M yi/src/library/Yi/Snippets/Haskell.hs > M yi/src/library/Yi/String.hs > M yi/src/library/Yi/Syntax/Haskell.hs > M yi/src/library/Yi/Syntax/JavaScript.hs > M yi/src/library/Yi/Syntax/Latex.hs > M yi/src/library/Yi/Syntax/Paren.hs > M yi/src/library/Yi/Syntax/Strokes/Haskell.hs > M yi/src/library/Yi/Syntax/Tree.hs > M yi/src/library/Yi/Tab.hs > M yi/src/library/Yi/Tag.hs > M yi/src/library/Yi/TextCompletion.hs > M yi/src/library/Yi/UI/Pango.hs > M yi/src/library/Yi/UI/Pango/Control.hs > M yi/src/library/Yi/UI/Pango/Layouts.hs > M yi/src/library/Yi/UI/TabBar.hs > M yi/src/library/Yi/UI/Utils.hs > M yi/src/library/Yi/UI/Vty.hs > M yi/src/tests/Generic/TestUtils.hs > M yi/src/tests/Vim/EditorManipulations/BufferExCommand.hs > M yi/src/tests/Vim/TestExCommandParsers.hs > M yi/src/tests/Vim/TestPureBufferManipulations.hs > M yi/src/tests/Vim/TestPureEditorManipulations.hs > M yi/src/tests/data/haskell/3.hs > M yi/yi.cabal > > Log Message: > ----------- > Migrate most of Yi to YiString/Text combo. > > Below is the commentary of the changes. This commit is large and > encompasses a lot but due to the nature of changes, small commits were > often not viable. If there was something obvious to fix then I did it > there and then too rather than putting it off for later. > > YiString is a new rope implementation implemented in the ‘yi-rope’ > package. It allows us to delete the Data.Rope module from src/library > which I did. The fundamental difference is that it is a rope implemented > over strict Text rather than over (lazy) ByteString. Benchmarks show > nice speedup and better memory efficiency. More benchmarks showed that > the chunk size of 1200 is more suited for working with larger text sizes > such as whole buffers. These benchmarks are disjoint from Yi core itself > which may mean that in practice, chunk size might need tweaking. > > Using Text frees us from having to deal with encoding. Old > implementation converted back and forth a huge amount because any > meaningful operation had to be done over String or Char, we never want > to work with Word8. These conversions made using a rope pointless as > we'd probably get better milage just keeping a String around with cached > length. This brings me onto the rest of the changes. `yi-rope` needs few > extra functions (`span`, `break`, Hashable instance) to replace some > more Text usage but it's quite usable as it is. > > Having removed the need for constant conversions, we are now able to > refactor the rest of the code to take advantage. Ideally we want to keep > things as YiString for as long as possible to not incur conversion > penalties. I initially made everything work over Text and then had a > second pass at it, converting to YiString. This was quite a lot of work. > Note that there are exceptions where it might still be faster to convert > to Text: if we are doing something over a string we have to force > and map over anyway, it is probably faster (and more convenient) to just > convert to Text and work with it that way. Text is pretty damn fast so > this still works out much better than String. I'll try to give a very > short comment on each module I touched in hopes that when someone > inevitably git-blame's, they'll be able to at least get a basic idea why > the change is in this commit. Some changes were trivial: Haddock header > includes, import cleanups, reformatting to 80C, de-shadowing, removing > extra pragmas, fixing warnings…. These are just documented here as > ‘style’. Lastly, I have removed the Copyright field from few of the > Haddock headers. The reason is that they are not consistent: most > modules are missing them and when they are present, they only have > authors from many years ago and none after. The choice was to either > track down all the authors and years they touched the files (a ton of > work) or to remove the field all together. If anyone is interested, > git-blame the file. Also see AUTHORS. > > Spread out through the modules are a bunch of more TODOs such as places > where we should probably keep something as YiString rather than > converting to Text or places where benchmarking needs to make the call. > There are a couple of places where top level types have been fixed to > YiString or Text but they do conversion internally. This makes it > implement Text/YiString native versions of their guts. Places where > conversions to String are still done are scarce but still present, such > is when working with filepaths or external processes. Where Text worked > fine from the first pass, I did not change it specifically but in some > places it might be worthwhile to do so if it causes conversions. > > `cabal test` passes and I haven't noticed any bugs I might have caused > but there are a lot of changes so I have no doubt something sneaked in. > Please report if you see anything wrong. > > Data.Rope: deleted, use Yi.Rope from `yi-rope` instead > > System.CanonicalizePath: Turned a bunch of stuff to work over Text. It > talks with System.FilePath which works over String and character-based > processing is done here which means that YiString would just incur extra > penalties. We should move to `system-filepath` package anyway which uses > its own type all together rather than an alias for String. > > Yi: style only, but we should remove the `Data.Prototypes` export now > that it's in a separate package, `oo-prototypes`. > > Yi.Boot: Follow changes from other modules. > > Yi.Buffer.Highlevel: String-returning functions got the boot in favour > of YiString. Also many of the duplicate functions (over String and Rope) > are gone and instead only YiString is exposed. `bufInfoPercent` field is > now Text. > > Yi.Buffer.Implementation: Similar to `Yi.Buffer.Highlevel`. > Unfortunatelly, there is still the problem that we need a String stream > for things like Vty. That String version is now under `getStream'`. > Also, `mem` is now exposed so that we `elemsB` can directly return > buffer content rather than stupidly taking `maxBound` from 0. > > Yi.Buffer.Indent: `yi-rope` needs `break` and `span` to get rid of the > Text conversion inside the keyword functions. > > Yi.Buffer.Misc: Get rid of a few duplicate functions that only did > conversions. Add `streamB'` for the same reason as > `Yi.Buffer.Implementation.getStream'`. modeName and modeModeLine are now > Text. > > Yi.Buffer.Region: Most notably, `replaceRegionClever` needs an > implementation for `getGroupedDiff` as the existing TODO stated. At the > moment we have to convert to and from String to call it. A benchmark may > show that it's actually faster to replace the whole region than > calculate the diff as splicing into the YiString should not be that > expensive. > > Yi.Buffer.TextUnit: You'll notice `genBoundary` still uses String. An > easy way to see just how ineffecient this makes it is to hold M-q for a > bit with the emacs keymap and look at the profiling. The reason is its > use of `streamB'`. An optimisation should be fairly easy here however, > unlike in the Vty scenario. > > Yi.Buffer.Undo: fix on of many badly placed Haddock comments, allowing > docs to build. All other such changes won't be commented on. > > Yi.Command: The minibuffers now run on Text so we just follow the > changes here. It's unclear if YiString gives us much benefit here as we > need to convert to String anyway to shell commands. > > Yi.Completion: Now runs on Text as we need to do a bunh of filtering and > prefixing in here anyway. > > Yi.Config.Default: `defaultPublishedActions` still run on String as > `yi-rope` needs a Hashable instance first. > > Yi.Config.Simple: Mostly style > > Yi.Core: Follows changes to Text > > Yi.Debug: debug functions now work over Text as most everything else is > now either Text or YiString. > > Yi.Dired: Follow other module changes but nothing fundamental changed > otherwise. > > Yi.Editor: `fst` of Status is now list of Text. Extra CPP for Typeable. > > Yi.Eval: Mostly just follow changes. Raises a problem as we use > regex-tdfa which uses String. There is `regex-tdfa-text` on Hackage > which we should probably use even though it's just a thin wrapper. > > Yi.File: Mostly adapt to Text. > > Yi.History: Replace fields with Text but I feel YiString would do better > here. > > Yi.Hoogle: Now works over YiString > > Yi.IReader: Just followed changes. I have no idea why this works over > ByteString to begin with so I did not touch it. Needs investigating. > > Yi.Interact: accepted spits out Text now. > > Yi.Keymap.Cua: work over YiString > > Yi.Keymap.Emacs: Surprisingly barely any changes, mostly imports. > > Yi.Keymap.Emacs.KillRing: Kllring now works over YiString which saves > conversions on every M-w and C-y. > > Yi.Keymap.Emacs.Utils: Follow minibuffer and Tag changes. > > Yi.Keymap.Vim: accomodate for EventString change from String to a > newtype over Text. > > Yi.Keymap.Vim.Common: EventString and OperatorName are now newtypes over > Text instead of String aliases. This was a lot of work because none of > the Vim modules actually used the aliases making it quite a chase. > Newtyping this exposed a couple of places where conversion from > EventString to OperatorName was necessary. Perhaps this indicates a bug? > A notable thing here is that the command parsers all parse a spit out > String. We should instead use Attoparsec here and directly work and get > Text. We'd much better performance due to no type conversions and > because attoparsec is damn fast although I doubt speed was ever a > concern in those modules. > > Yi.Keymap.Vim.Ex.Types: ExCommand now uses Text instead of String. > > Yi.Keymap.Vim.InsertMap: VimState still works with String (for now) so > there are a lot of conversions here from EventString to String, for > VimBindingE/Y. > > Yi.KillRing: String → YiString > > Yi.MiniBuffer: now works over Text. This is an artifact of the first > pass but considering what we do with minibuffers, it may be better to > not migrate to YiString here: the strings manipulated are never huge and > we need to do splitting, matching and all that jazz so I suspect it's > more efficient to stick to Text here. There are some conversions for > functions dumping content to buffer, such as results of the typing > callback. > > Yi.Misc: deals with filepaths a bit so some conversions are necessary > but looks like Text to the outside users. > > Yi.Haskell.Dollarify: Now uses YiString > > Yi.Search: searches are now over YiString which is nice due to replaces > but not nice because we need to unpack to String for regex. > > Yi.Snippets: ValuedMark is now over YiString. > > yi-contrib: > > Only one notable change here: JP.hs was importing a file from inside > Yi's test directory. I pulled out the content imported and inlined it > into the config. The module should be moved out of test directory if > people want to use it. The definition inside look useful enough to do > that. Other changes are simply String → Text/YiString conversions + > style. > > Summary: There are still quite a few ‘easy’ conversions to get us to the > 90%. ‘The second 90%’ requires tests and benchmarks which is why I'm > submitting this before being completely satisified. I don't want to > diverge from the main branch too much and make it difficult to merge > later, especially if I start piling testing and other tools on top of > this. > > > Commit: 1f0579565ae3ce06e053b0b3ce819e41d4f1c08e > > https://github.com/yi-editor/yi/commit/1f0579565ae3ce06e053b0b3ce819e41d4f1c08e > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-18 (Thu, 18 Sep 2014) > > Changed paths: > M yi/example-configs/yi-cua.hs > M yi/example-configs/yi-simple.hs > M yi/example-configs/yi-vim-colemak.hs > M yi/example-configs/yi-vim.hs > M yi/example-configs/yi.hs > M yi/yi.cabal > > Log Message: > ----------- > Fix example-configs (hopefully) > > > Commit: ec15a540ae9b464ddbc8d465fb05c05dfb5634c7 > > https://github.com/yi-editor/yi/commit/ec15a540ae9b464ddbc8d465fb05c05dfb5634c7 > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-18 (Thu, 18 Sep 2014) > > Changed paths: > M yi/src/library/Yi/Buffer/HighLevel.hs > > Log Message: > ----------- > Fix comment/uncomment > > > Commit: 9631e4dc1dd5e48cb73941056cf258cbb383707a > > https://github.com/yi-editor/yi/commit/9631e4dc1dd5e48cb73941056cf258cbb383707a > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-18 (Thu, 18 Sep 2014) > > Changed paths: > M .travis.yml > > Log Message: > ----------- > Disable example-configs building. > > The --force-reinstalls makes these break. We should fix Travis first. > > > Commit: aac03dc352081b02f7004aaccca339c4ec58aa9c > > https://github.com/yi-editor/yi/commit/aac03dc352081b02f7004aaccca339c4ec58aa9c > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-18 (Thu, 18 Sep 2014) > > Changed paths: > M yi/src/library/Yi/Buffer/HighLevel.hs > > Log Message: > ----------- > Fix deleteTrailingSpaceB. > > > Commit: ec9296e4c83c8dd5a1e8d7ad55329fa08bc02c33 > > https://github.com/yi-editor/yi/commit/ec9296e4c83c8dd5a1e8d7ad55329fa08bc02c33 > Author: Mateusz Kowalczyk <fuuze...@fuuzetsu.co.uk> > Date: 2014-09-18 (Thu, 18 Sep 2014) > > Changed paths: > M yi/src/library/Yi/Keymap/Emacs/Utils.hs > > Log Message: > ----------- > Fix #613. > > Also improve line counting, in emacs 0 lines does not occur, maybe in > empty buffer. > > > Compare: https://github.com/yi-editor/yi/compare/07129dfeffc1 > ^...ec9296e4c83c > > -- > -- > Yi development mailing list > yi-devel@googlegroups.com > http://groups.google.com/group/yi-devel > --- > You received this message because you are subscribed to the Google Groups > "yi.devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to yi-devel+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- Yi development mailing list yi-devel@googlegroups.com http://groups.google.com/group/yi-devel --- You received this message because you are subscribed to the Google Groups "yi.devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to yi-devel+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
