Hi Offray, In STON #bleedingEdge you will find:
=== Name: STON-Core-SvenVanCaekenberghe.57 Author: SvenVanCaekenberghe Time: 26 April 2015, 11:53:58.308533 pm UUID: f38b0222-94ea-4a0a-9878-03649843c97e Ancestors: STON-Core-SvenVanCaekenberghe.56 Added STONReader>>#convertNewLines: and STONReader>>#newLine: to read and convert CR, LF, or CRLF inside strings and symbols as one chosen canonical newLine Added STONWriter>>#keepNewLines: to write CR, LF or CRLF inside strings and symbols unencoded as one chosen canonical newLine Add unit tests #testConvertingNewLine #testKeepingNewLines and #testIllegalCharacterEscapes Added some more documentation === Name: STON-Tests-SvenVanCaekenberghe.52 Author: SvenVanCaekenberghe Time: 26 April 2015, 11:54:29.166526 pm UUID: 8b393798-e0e0-4ca4-9f54-64e487c228cc Ancestors: STON-Tests-SvenVanCaekenberghe.51 Added STONReader>>#convertNewLines: and STONReader>>#newLine: to read and convert CR, LF, or CRLF inside strings and symbols as one chosen canonical newLine Added STONWriter>>#keepNewLines: to write CR, LF or CRLF inside strings and symbols unencoded as one chosen canonical newLine Add unit tests #testConvertingNewLine #testKeepingNewLines and #testIllegalCharacterEscapes Added some more documentation === So, if you do (STON writer on: ...) newLine: String crlf; keepNewLines: true; nextPut: ... any CR, LF or CRLF inside any String will no longer be written as \r, \n or \r\n but all as CRLF, a normal EOL. Similarly, if you do (STON reader on: ..) newLine: String crlf; convertNewLines: true; next. any CR, LF or CRLF seen while reading Strings will all be converted to the same EOL, CRLF. Please have a look and see if this can help solve your problem. And let me know how it went. Sven > On 25 Apr 2015, at 00:12, Offray Vladimir Luna Cárdenas <off...@riseup.net> > wrote: > > Hi Sven, > > I would like to retake this conversation. I wasn't able to answer before > because I was trying to get minimal functionality for grafoscopio and now > that is there, I would like to tackle the issue of long lines in STON, so it > can become Distributed Version Control Systems friendly. > > To understand better your proposal, lets try it with a real "document" stored > in STON, the one at [1]. This is the grafoscopio alpha manual in Spanish, > with long lines. I would like to test it with your second propossal, and > change the newline inside strings as CRLF. A successful test case after > applying the change would be that I can diff two commits of the manual and > compare them inside our DVCS (fossil-scm), without it getting the file > confused with a binary file. > > [1] > mutabit.com/deltas/repos.fossil/grafoscopio/doc/tip/Docs/Es/Manual/manual-grafoscopio.ston > > I imagine you need to do some change to STON to implement this, but I'm > willing to make any test from this side and report them back to you. > > Making STON DVCS friendly would open a lot of collaboration on interactive > documentation and other places where STON can be used as an storage format > (like we are doing). > > So, how can we proceed? > > Thanks, > > Offray > > El 26/11/14 a las 10:40, Sven Van Caekenberghe escribió: >> Hi Offray, >> >> I think I better understand what you are doing now, and how you are using >> STON. >> >> What I think might help you (in your use case), is to add two options: >> >> 1 - to STONWriter #keepNewlines that converts any newline inside Strings to >> a newline as specified by #newline: instead of encoding it as unprintable >> 2 - to STONReader #convertNewlines that converts any newline inside Strings >> to a newline as specified by #newline: instead of keeping it as is >> >> Now, 2 is already there, except that any newline read is kept as it is, >> while I think it would be better to convert CR, LF and CRLF to just one (to >> be specified). >> >> Similarly, 1 would convert CR, LF and CRLF to a single one (to be specified). >> >> Without these conversions, mixups between different line end conventions >> could easily happen. I think it is too much work to do this in your own >> models. >> >> If you think this would help, I'll put this on my todo. >> >> Sven >> >>> On 18 Nov 2014, at 02:50, Offray Vladimir Luna Cárdenas <off...@riseup.net> >>> wrote: >>> >>> Hi Sven, >>> >>> Sorry for my late response. The constructive comments on the list and yours >>> in particular are very valuable to my. I was finishing some details, so >>> only until now I have the time to implement your suggestions. The new code >>> for custom keys on bibtex files from pharo is published at [1] (by the >>> grace of Doru's easy publishing of playgrounds on your stfx server) >>> >>> [1] http://ws.stfx.eu/3CEKQQQ3NL2E >>> >>> By the way the article I'm writing is about a tool for open, citizen, >>> garage research and science developed in Pharo. It is published at [2] and >>> was stored nicely using STON[3] and is superb. To test the tool, the >>> article was wrote on it. >>> >>> [2] >>> http://mutabit.com/deltas/repos.fossil/grafoscopio/doc/tip/Docs/Es/Articulos/Libertadores/bootstrapping-objeto-investigacion.pdf >>> >>> [3] >>> http://mutabit.com/deltas/repos.fossil/grafoscopio/doc/tip/Docs/Es/Articulos/Libertadores/bootstrapping-objeto-investigacion.ston >>> >>> [4] >>> http://mutabit.com/deltas/repos.fossil/grafoscopio/doc/tip/Docs/Es/Articulos/Libertadores/bootstrapping-objeto-investigacion.markdown >>> >>> The only thing I would add to STON would be an option to support line >>> breaks so long character sequences can be broken to make the format DVCS >>> friendly (git, fossil, etc) and support collaboration and changes tracking. >>> At this moment, because of the long lines in STON, the files are treated as >>> binaries by fossil :-/. >>> >>> Thanks for STON and your lessons, >>> >>> Offray >>> >>> El 22/10/14 a las #4, Sven Van Caekenberghe escribió: >>>> >>>>> On 22 Oct 2014, at 18:42, Offray Vladimir Luna Cárdenas >>>>> <off...@riseup.net> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Thanks again. I have a small script, using Citezen which does the trick. >>>>> I can explore and modify the BibTeX File from the playground with this: >>>>> >>>>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- >>>>> | bibFile bibliography bibStream bibOutputer | >>>>> bibFile := ((FileLocator documents / 'U/Libertadores/Grafoscopio') >>>>> children >>>>> detect: [:each | each basename endsWith: 'bib' ]). >>>>> bibliography := CZBibParser parse: bibFile contents. >>>>> bibStream := '' writeStream. >>>>> 1 to: (bibliography size) do: [:index | >>>>> (((bibliography entries at: index) fields at: 2) key = 'shorttitle') >>>>> ifTrue: [ >>>>> (bibliography entries at: index) >>>>> key: ((bibliography entries at: index) fields at: 2) value]. >>>>> bibOutputer := CZBibtexOutputer new. >>>>> bibStream nextPutAll: >>>>> (bibOutputer entryToBibtexString: >>>>> (bibliography entries at: index)); cr.]. >>>>> bibliography. >>>>> bibFile writeStreamDo: [:stream | >>>>> stream nextPutAll: bibStream contents withUnixLineEndings ]. >>>>> bibStream contents. >>>>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- >>>> >>>> Some constructive comments about your code (smaller is always better, >>>> especially for interactive snippets): >>>> >>>> - you can change the #detect: to [ :each | each extension = #bib ] >>>> - you can iterate directly over the entries with #do: as in bibliography >>>> entries do: [ :each | .. ] which saves you the #at: index >>>> - there are handy unary shortcuts for accessing elements, like #first, >>>> #second and so on (up to #ninth) which also save you parenthesis >>>> - you can also construct strings using the idiom String streamContents: [ >>>> :bibStream | .. ] >>>> >>>> Sorry, these jumped to me when I saw your code, I hope you don't mind ;-) >>>> >>>>> I will put some functionality inspired by this on my prototype this >>>>> weekend. >>>>> >>>>> Cheers, >>>>> >>>>> Offray >>>>> >>>>> On 10/21/2014 01:20 AM, stepharo wrote: >>>>>> Check in the tools there is a bib writer. >>>>>> >>>>>> Stef >>>>>> >>>>>> On 21/10/14 03:33, Offray Vladimir Luna Cárdenas wrote: >>>>>>> Thanks Stef and Damien, >>>>>>> >>>>>>> I have this small script as a proof of concept: >>>>>>> >>>>>>> =================================== >>>>>>> | bibFile bibliography | >>>>>>> bibFile := ((FileLocator documents / 'U/Libertadores/Grafoscopio') >>>>>>> children >>>>>>> detect: [:each | each basename endsWith: 'bib' ]) contents. >>>>>>> bibliography := CZBibParser parse: bibFile. >>>>>>> 1 to: (bibliography size) do: [:index | >>>>>>> (((bibliography entries at: index) fields at: 2) key = 'shorttitle') >>>>>>> ifTrue: [ >>>>>>> (bibliography entries at: index) >>>>>>> key: ((bibliography entries at: index) fields at: 2) value >>>>>>> ]]. >>>>>>> bibliography. >>>>>>> =================================== >>>>>>> >>>>>>> Now I want to write back the corrected file to the .bib ... just >>>>>>> having problems finding which message does the job. I'll keep searching. >>>>>>> >>>>>>> Cheers, >>>>>>> >>>>>>> Offray >>>>>>> >>>>>>> On 10/16/2014 06:40 AM, Damien Cassou wrote: >>>>>>>> from Damien Pollet: >>>>>>>> >>>>>>>> You will need to load the .bib file from zotero (read the file however >>>>>>>> you like, then pass the stream to the CZ parser). You'll get a >>>>>>>> CZBibSet (I don't recall the name exactly) which represents the >>>>>>>> contents of the file. A Set is composed of entries, each of which has >>>>>>>> a key and a set of fields. Finally, fields accept a few different >>>>>>>> kinds of values. >>>>>>>> >>>>>>>> Your processing is just iterating a set then setting the key of each >>>>>>>> entry (or possibly removing and re-adding the entry, I don't recall if >>>>>>>> it's implemented like a dictionary or more like a list). >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Oct 13, 2014 at 2:57 AM, Offray Vladimir Luna Cárdenas >>>>>>>> <off...@riseup.net <mailto:off...@riseup.net>> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm using a Zotero collection for keeping track of several >>>>>>>> references I have >>>>>>>> found for my article about the experience of the >>>>>>>> outline/tree-like metaphor >>>>>>>> for writing inside Pharo (as soon as I have a presentable >>>>>>>> working draft I >>>>>>>> hope to share it with you). >>>>>>>> >>>>>>>> Now I want to make a post-processing of the bibtex file exported >>>>>>>> from >>>>>>>> Zotero. The idea is to use "shorttitle" field instead to replace >>>>>>>> the Zotero >>>>>>>> auto-generated one and have custom keys. So for example instead of: >>>>>>>> >>>>>>>> ======= >>>>>>>> @misc{_holistic_????, >>>>>>>> title = {Holistic software assessment (Uni Zurich - >>>>>>>> 2011) on Vimeo}, >>>>>>>> shorttitle = {Girba-holistic-2011}, >>>>>>>> url = {http://vimeo.com/42073344?__from=outro-local >>>>>>>> <http://vimeo.com/42073344?from=outro-local>}, >>>>>>>> urldate = {2014-08-19}, >>>>>>>> note = {00000} >>>>>>>> } >>>>>>>> >>>>>>>> ======= >>>>>>>> >>>>>>>> >>>>>>>> I would like to have: >>>>>>>> >>>>>>>> >>>>>>>> ======= >>>>>>>> >>>>>>>> @misc{Girba-holistic-2011, >>>>>>>> title = {Holistic software assessment (Uni Zurich - >>>>>>>> 2011) on Vimeo}, >>>>>>>> shorttitle = {Girba-holistic-2011}, >>>>>>>> url = {http://vimeo.com/42073344?__from=outro-local >>>>>>>> <http://vimeo.com/42073344?from=outro-local>}, >>>>>>>> urldate = {2014-08-19}, >>>>>>>> note = {00000} >>>>>>>> } >>>>>>>> >>>>>>>> ======= >>>>>>>> >>>>>>>> >>>>>>>> I have already installed Citizen and open it on the browser to >>>>>>>> see the code, >>>>>>>> but I can find any place to start with examples. >>>>>>>> >>>>>>>> Any advice on how to solve this issue will be appreciated. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> >>>>>>>> Offray >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Damien Cassou >>>>>>>> http://damiencassou.seasidehosting.st >>>>>>>> >>>>>>>> "Success is the ability to go from one failure to another without >>>>>>>> losing >>>>>>>> enthusiasm." >>>>>>>> Winston Churchill >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >>> >> >> >> > >
