Re: native or not
Hey, > I'm not seeing any size difference, but groff is not in the output: OK, then its normal. > That fails on master (libpaper) whereas with the patch it works, > so I guess the patch is useful on that front. > > The patch for sudo will be in the following emails. > > Is there anything else to check / test ? Well you can also check for deterministic compilation, and all the other items listed here: https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html Also be careful because there were quite a lot of cross-compilation related patches pushed on core-updates. You may want to check that you are not duplicating this work. Anyway, most of the packages that should cross-compile fail to do so because of inputs/native-inputs mix-up, so your patches are really welcome :) Thanks, Mathieu
Re: native or not
> I only build-tested them natively on/for x86_64 (and > cross built for aarch64 for the sudo one) This LGTM, I added your copyright on the two first patches and pushed! Thanks, Mathieu
Re: native or not
Hello, On Tue, Mar 31, 2020 at 9:45 AM Mathieu Othacehe wrote: > This LGTM, I added your copyright on the two first patches and pushed! Thanks for handling that. Stay tuned for the next batch. Christopher, looks like your work on data.guix.gnu.org already triggered something useful ! ;-) -- Vincent Legoll
Re: native or not
> Christopher, looks like your work on data.guix.gnu.org > already triggered something useful ! ;-) And the view for comparing with previous run is just showing me what I wanted to see, perfect ! Question: would this view show any `guix size` difference if any ? Keep up the good work ! -- Vincent Legoll
Re: native or not
Vincent, Vincent Legoll 写道： I'm not seeing any size difference, but groff is not in the output: There's some deeper confusion here: why do you expect the size to change, at all? Making ‘groff’ native was the right thing to do (and please, keep fixing bugs like this! :-) but it has nothing to do with making packages smaller. Kind regards, T G-R signature.asc Description: PGP signature
Re: native or not
Hello, On Tue, Mar 31, 2020 at 11:44 AM Tobias Geerinckx-Rice wrote: > Vincent Legoll 写道： > > I'm not seeing any size difference, but groff is not in the > > output: > > There's some deeper confusion here: why do you expect the size to > change, at all? Because I've been told so... On Mon, Mar 30, 2020 at 8:57 AM Mathieu Othacehe wrote: > Well yes it may reduce the closure size of the package (run `guix size > sudo`) to get it. But I don't expect that to happen every time (I've seen the "it may")... > Making ‘groff’ native was the right thing to do (and please, keep > fixing bugs like this! :-) but it has nothing to do with making > packages smaller. Never will ? I'm not expecting package size to shrink, but package closure (is that the right word) size to sometimes shrink. And if I've understood that may be visible for ISO, VM, container image sizes. Am I mistaken ? Still learning... -- Vincent Legoll
Thank you for your leadership Ludo
Hey Ludo! I have been hanging out in #guix recently, and I have submitted a few guix bug reports (not many). I just wanted to thank you for the community that you have built with Guix. I have done some minor volunteer work with other free software projects, but those other free software projects did not have Guix's incredible community. It is incredibly nice to feel like I can help Guix, even when it is sparingly. I also want to thank the other /incredibly/ dedicated Guix maintainers. Thanks for all that you do! Thanks again, Joshua P.S. I hope this is not consider off topic. I just recently read the first few chapters of "How to Win friends and Influence People". It recommended giving out lots of compliments. I thought that I would try it.
Re: native or not
Vincent Legoll writes: >> Making ‘groff’ native was the right thing to do (and please, keep >> fixing bugs like this! :-) but it has nothing to do with making >> packages smaller. > > Never will ? > > I'm not expecting package size to shrink, but package closure > (is that the right word) size to sometimes shrink. Unless you are cross-compiling (i.e. guix build --target=foo), native-inputs and inputs behave exactly the same; they are simply concatenated. signature.asc Description: PGP signature
Re: LHC for guixHPC?
Hello, bijan ghavami-kia skribis: > Thank you kindly for the reply! I have one question born out of my ignorance, > so please be patient with me; I am looking at the various > packages, which belong to various repos eg CRAN, TeXlive etc; > for the julia language..., is there a similar thing, or the packages are > through the julia built in package manager only (although it > seems a very decent one https://julialang.org/blog/2019/11/artifacts/)? And > if so, is there a reason and is there any loss or conflict in > relation to the guix package management interaction with these? In general, it’s possible to use language-specific package managers on top of Guix, modulo possible packaging bugs. However, we generally recommend managing packages through Guix: it brings uniformity, which is always pleasant as a user, and it brings all the nice features of Guix to all the packages one is using—reproducibility, transparency and provenance tracking, transactional upgrades and rollbacks, integration with ‘guix pack’, etc. For that, we have a set of “importers” that convert, automatically or semi-automatically, packages from those language-specific repositories: https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-import.html There’s no importer for Julia currently, but it would be a welcome addition! > Apologies if these are stupid queries, I am not experienced and I'm sure > there are simple answers These are very valid questions. Thank you, Ludo’.
Re: Linphone
Hello Guix! I have finished packaging all the linphone project's packages. \o/ PATCH: https://issues.guix.gnu.org/issue/40264 SOURCE: https://cloud.disroot.org/f/94900047 There will be no new patches and only revision patches. :-) Regards, RG.
Re: Use genimage for disk-image creation.
Hi! Vagrant Cascadian skribis: > On 2020-03-29, Danny Milosavljevic wrote: >> Hi Ludo, >> >> On Sun, 29 Mar 2020 16:44:39 +0200 >> Ludovic Courtès wrote: >> >>> Oh, really? I’m surprised partitioning causes problems (though I’m >>> not familiar with embedded dev!). >> >> Well, maybe not that bad, but it's pretty bad. >> >> It's because the boot sector for ARM, for some unfathomable reason, is not >> the first sector but has basically a vendor-dependent sector number somewhere >> in the middle of the disk :P Oh I see, I actually noticed that in (gnu bootloader u-boot) and in Buildroot, it’s teible. > I haven't really looked at buildroot at all... but I suspect buildroot > is just a collection of all these criteria applied on a board-by-board > basis, but most of these boards don't actually require specific > partition layout, per se, it's just nice to not clobber the raw offsets > of various parts of the boot process... but creating partitions for each > of those can make installation less error-prone in some cases. Yeah, I guess we could keep importing them like Danny did in the past, but of course it would be nice to have an automated process to do that. Ludo’.
Re: Proxy settings wrt guix daemon
Hi, Vincent Legoll skribis: > On Sun, Mar 29, 2020 at 5:00 PM Ludovic Courtès wrote: >> So my recommendation would be to fix [4] specifically for ‘guix-daemon’ >> by adding a ‘set-proxy’ action or something. > > Trying to understand what that would imply, I stumbled upon my > ignorance of how to test attempts at implementing that. > > Would the following be useful : > make + pre-inst guix system container in a git checkout / branch > with modifications to nix/libstore/builtins.cc::builtinDownload (for > the server part) > > Then how to implement the "asking the server to change its > proxy setting" ? Where should I look ? > > What UI (client-side) ? > > `guix daemon set-proxy` or something like that ? > > I.e. I'm willing to try, but will need guidance... I was proposing a custom action for the Shepherd service, just like the mcron Shepherd service has a custom ‘schedule’ action that one can invoke with “herd schedule mcron”. Hope that’s clearer! Ludo’.
Re: native or not
Vincent Legoll writes: >> Christopher, looks like your work on data.guix.gnu.org >> already triggered something useful ! ;-) > > And the view for comparing with previous run is just showing > me what I wanted to see, perfect ! Nice, I'm glad that you're enjoying using the Guix Data Service, and finding it useful :) > Question: would this view show any `guix size` difference if any ? So, `guix size` as I understand it looks at the size of the store item + the size of all store items referenced both directly and indirectly. The `guix size` script can use your local store, as well the data in narinfo files from substitute files to determine the size of store items, and there references. The Guix Data Service gathers up narinfo files from substitute servers, so the database should contain the necessary data on sizes and references to provide equivalent information to the `guix size` script. I like the idea of showing size changes on the comparison page, but I think a good first step to take towards this would be to show the "size" for a single item in the store. This [1] is the page for the store output for the sudo package, with your changes. It already shows the size from the narinfo files provided by bayfront and berlin (3633056 bytes). 1: http://data.guix.gnu.org/gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1 It would be nice to have a URL like [2] (which obviously doesn't exist yet) which would show similar information to running the `guix size` script [3]. 2: http://data.guix.gnu.org/gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1/size 3: guix size /gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1 Would this be something I'd be able to convince and support you to do Vincent? I'd be more than happy to help you to implement this. Chris signature.asc Description: PGP signature
Re: [GSoC 2020] [Final Proposal] Clojars importer for Guix
On Tue, 31 Mar 2020 at 15:06, Leandro Doctors wrote: > Below, I share the final version I uploaded to the GSoC website. Another bug: I didn't mention how I plan to publicly report my progress over the course of my work. Besides the normal interaction via this list, I will send (at least) three progress reports. One progress report to this list (with a post on the Guix blog) after each Evaluation. Additionally, I may also send irregular updates when I achieve important milestones. Sorry for this oversight, Leandro
Re: native or not
Hello, On Tue, Mar 31, 2020 at 3:25 PM Marius Bakke wrote: > Unless you are cross-compiling (i.e. guix build --target=foo), > native-inputs and inputs behave exactly the same; they are simply > concatenated. OK, thanks for the clarification. Christopher, I'm not a web guy, I may take a shot at it, when my todo list gets emptyish, but don't hold your breath, that may not happen in this life... Following this will come the next batch of native-inputs. Built on x86_64, tried to also target aarch64, but kind of unconclusive (one stopped at perl, another at mit-krb5...) I've searched core-updates to avoid duplicating work, but only after having done it... Got lucky to have only lost a few minutes on openldap... The graphviz patch may not be for master as it may force 2918 packages to be rebuilt. The other ones should be OK. Question: should I stay on guix-devel, should I add guix-patches or should I move to guix-patches for the following patche series ? -- Vincent Legoll
[PATCH 6/6] gnu: nethack: Make some inputs native.
* gnu/packages/games.scm (nethack)[native-inputs]: New field.
[inputs]: Move flex & bison to native-inputs.
---
gnu/packages/games.scm | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/games.scm b/gnu/packages/games.scm
index 3284459021..addadebbba 100644
--- a/gnu/packages/games.scm
+++ b/gnu/packages/games.scm
@@ -48,6 +48,7 @@
;;; Copyright © 2017, 2019 Hartmut Goebel
;;; Copyright © 2020 Alberto Eleuterio Flores Guerrero
;;; Copyright © 2020 Naga Malleswari
+;;; Copyright © 2020 Vincent Legoll
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1250,10 +1251,11 @@ watch your CPU playing while enjoying a cup of tea!")
(string-join (string-split version #\.) "")
"-src.tgz"))
(sha256
(base32 "1liyckjp34j354qnxc1zn9730lh1p2dabrg1hap24z6xnqx0rpng"
+(native-inputs
+ `(("bison" ,bison)
+("flex" ,flex)))
(inputs
`(("ncurses" ,ncurses)
-("bison" ,bison)
-("flex" ,flex)
("less" ,less)))
(build-system gnu-build-system)
(arguments
--
2.25.2
[PATCH 4/6] gnu: iwd: Make some inputs native.
* gnu/packages/networking.scm (iwd)[native-inputs]: New field.
[inputs]: Move libtool to native-inputs.
---
gnu/packages/networking.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index ec2f0b64bd..79b07e23f0 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -2712,13 +2712,13 @@ protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, and
RIP. ")
(build-system gnu-build-system)
(inputs
`(("dbus" ,dbus)
- ("libtool" ,libtool)
("ell" ,ell)
("readline" ,readline)))
(native-inputs
`(("asciidoc" ,asciidoc)
("autoconf" ,autoconf)
("automake" ,automake)
+ ("libtool" ,libtool)
("pkgconfig" ,pkg-config)
("python" ,python)
("openssl" ,openssl)))
--
2.25.2
[PATCH 1/6] gnu: cgit: Make some inputs native.
* gnu/packages/version-control.scm (cgit)[native-inputs]: New field.
[inputs]: Move groff & python-docutils to native-inputs.
---
gnu/packages/version-control.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index 8af54c6e35..0aff346e0a 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -28,6 +28,7 @@
;;; Copyright © 2020 Roel Janssen
;;; Copyright © 2020 Brice Waegeneire
;;; Copyright © 2020 John D. Boy
+;;; Copyright © 2020 Vincent Legoll
;;;
;;; This file is part of GNU Guix.
;;;
@@ -857,6 +858,8 @@ collaboration using typical untrusted file hosts or
services.")
`(("asciidoc" ,asciidoc)
("gzip" ,gzip)
("bzip2" ,bzip2)
+ ("groff" ,groff)
+ ("python-docutils" ,python-docutils)
("xz" ,xz)))
(inputs
`(;; Building cgit requires a Git source tree.
@@ -869,9 +872,7 @@ collaboration using typical untrusted file hosts or
services.")
(sha256
(base32 "09lzwa183nblr6l8ib35g2xrjf9wm9yhk3szfvyzkwivdv69c9r2"
("openssl" ,openssl)
- ("groff" ,groff)
("python" ,python)
- ("python-docutils" ,python-docutils)
("python-markdown" ,python-markdown)
("python-pygments" ,python-pygments)
("zlib" ,zlib)))
--
2.25.2
[PATCH 5/6] gnu: mailutils: Make some inputs native.
* gnu/packages/mail.scm (mailutils)[native-inputs]: New field.
[inputs]: Move dejagnu & texinfo to native-inputs.
---
gnu/packages/mail.scm | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index 36b94f409f..458d82ecce 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -29,6 +29,7 @@
;;; Copyright © 2018 Gábor Boskovits
;;; Copyright © 2018, 2019, 2020 Ricardo Wurmus
;;; Copyright © 2019 Tanguy Le Carrour
+;;; Copyright © 2020 Vincent Legoll
;;;
;;; This file is part of GNU Guix.
;;;
@@ -218,11 +219,12 @@
#:parallel-tests? #f))
(native-inputs
- `(("perl" ,perl))) ;for 'gylwrap'
+ `(("perl" ,perl)
+ ("texinfo" ,texinfo)
+ ("dejagnu" ,dejagnu))) ;for 'gylwrap'
(inputs
- `(("dejagnu" ,dejagnu)
+ `(
("m4" ,m4)
- ("texinfo" ,texinfo)
("guile" ,guile-2.2)
("gsasl" ,gsasl)
("gnutls" ,gnutls)
--
2.25.2
[PATCH 2/6] gnu: darktable: Make some inputs native.
* gnu/packages/photo.scm (darktable)[native-inputs]: New field.
[inputs]: Move intltool & pkg-config to native-inputs.
---
gnu/packages/photo.scm | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/photo.scm b/gnu/packages/photo.scm
index 585289daf1..9f6e4a3031 100644
--- a/gnu/packages/photo.scm
+++ b/gnu/packages/photo.scm
@@ -7,6 +7,7 @@
;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice
;;; Copyright © 2018 Leo Famulari
;;; Copyright © 2020 Sebastian Schott
+;;; Copyright © 2020 Vincent Legoll
;;;
;;; This file is part of GNU Guix.
;;;
@@ -483,6 +484,9 @@ photographic equipment.")
(string-append (assoc-ref inputs "ilmbase")
"/include/OpenEXR:" (or (getenv "CPATH")
"")))
#t)
+(native-inputs
+ `(("intltool" ,intltool)
+ ("pkg-config" ,pkg-config)))
(inputs
`(("libxslt" ,libxslt)
("libxml2" ,libxml2)
@@ -502,9 +506,7 @@ photographic equipment.")
("ilmbase" ,ilmbase)
("libsoup" ,libsoup)
("python-jsonschema" ,python-jsonschema)
- ("intltool" ,intltool)
("perl" ,perl)
- ("pkg-config" ,pkg-config)
("libwebp" ,libwebp)
("lensfun" ,lensfun)
("librsvg" ,librsvg)
--
2.25.2
[PATCH 3/6] gnu: graphviz: Make some inputs native.
* gnu/packages/graphviz.scm (graphviz)[native-inputs]: New field.
[inputs]: Move swig to native-inputs.
---
gnu/packages/graphviz.scm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/graphviz.scm b/gnu/packages/graphviz.scm
index 2d2bb11130..051089881a 100644
--- a/gnu/packages/graphviz.scm
+++ b/gnu/packages/graphviz.scm
@@ -7,6 +7,7 @@
;;; Copyright © 2017 Gábor Boskovits
;;; Copyright © 2018 Mathieu Lirzin
;;; Copyright © 2020 Marius Bakke
+;;; Copyright © 2020 Vincent Legoll
;;;
;;; This file is part of GNU Guix.
;;;
@@ -97,7 +98,6 @@
("gts" ,gts)
("gd" ,gd) ; FIXME: Our GD is too old
("guile" ,guile-2.0) ;Guile bindings
- ("swig" ,swig)
("pango" ,pango)
("fontconfig" ,fontconfig)
("freetype" ,freetype)
@@ -108,6 +108,7 @@
("libpng" ,libpng)))
(native-inputs
`(("bison" ,bison)
+ ("swig" ,swig)
("pkg-config" ,pkg-config)))
(outputs '("out" "doc")) ; 5 MiB of html + pdfs
(home-page "http://www.graphviz.org/";)
--
2.25.2
Re: Thank you for your leadership Ludo
Hi! I found here in Guix not only the idea that implements - free software - reproducible builds, - better control over system - and something for the 5th industrial revolution but also community and relations with the other projects. Guix uses sources from the different original places (from the maintainers). Guix connects to the reproducible builds and to the Translation project. Guix uses gnu servers and based on GNU community. This is infrastructure that is provided by GNU and Guix move it forward creating new opportunities for users' new horizons. And the word free is not only 'name' of all this work but OS Guix essential feature. That is why I like it. March 31, 2020 12:21 PM, "Joshua Branson" wrote: > Hey Ludo! > > I have been hanging out in #guix recently, and I have submitted a few > guix bug reports (not many). I just wanted to thank you for the > community that you have built with Guix. I have done some minor > volunteer work with other free software projects, but those other free > software projects did not have Guix's incredible community. It is > incredibly nice to feel like I can help Guix, even when it is sparingly. > > I also want to thank the other /incredibly/ dedicated Guix maintainers. > Thanks for all that you do! > > Thanks again, > > Joshua > > P.S. I hope this is not consider off topic. I just recently read the > first few chapters of "How to Win friends and Influence People". It > recommended giving out lots of compliments. I thought that I would > try it.
Re: [PATCH 1/6] gnu: cgit: Make some inputs native.
Vincent, Thank you! This list is for general Guix discussion. Although that can sometimes include rough patches (heh), please send any future contributions to our patch tracker at guix-patc...@gnu.org — after carefully reading the ‘Submitting Patches’ section of the Guix manual. Vincent Legoll 写道： * gnu/packages/version-control.scm (cgit)[native-inputs]: New field. This is no new field. Neither are those of mailutils, iwd, or graphviz! Please rewrite all commit messages as either * gnu/packages/version-control.scm (cgit)[inputs]: Move groff & python-docutils from here… [native-inputs]: …to this new field. or * gnu/packages/version-control.scm (cgit)[inputs]: Move groff & python-docutils from here… [native-inputs]: …to here. although to be honest, I only ever use the latter format unless I'm feeling particularly pedantic. [inputs]: Move groff & python-docutils to native-inputs. If we run $ guix gc --references `guix build cgit` we can see that both groff and python-docutils are part of cgit's closure. This means they must to be able to run… at run time :-o Oh dear. python-docutils's ‘rst2html.py’ is required by cgit's /lib/cgit/filters/html-converters/rst2html, and groff itself by cgit's /lib/cgit/filters/html-converters/man2html. Both of these uses look legitimate to me: they weren't accidentally captured by a stray environment variable or overzealous wrapper. Now, it's still possible that either or both of these dependents must *also* run at build time, and for that they would need to be native indeed. Unfortunately: $ guix build cgit --target=mips64el-linux-gnu guix build: error: gnu/packages/python-xyz.scm:2950:2: python-docutils@0.16: build system `python' does not support cross builds I could ignore that and pretend that all Python packages are safely architecture-independent (they're not) to continue investigating, but I'm out of time. I don't think this particular patch adds any value at this time, so let's drop it. The rest LGTM! Kind regards, T G-R signature.asc Description: PGP signature
Rethinking files as a concept -- review draft paper
I was on a while back about using IPFS or some other CAS to cache
computation and results. I iterated on this concept with metadata, then
data, then just files as a fundamental concept. It became abundantly clear
that how we use, think, and need from and about files are not what computer
files are. They are currently unstructured inconsistent blobs. We want
them to be secure, distributed, and structured. In the attached, I attempt
to create the concepts to allow such a transition from files as
unstructured blobs to how we actually want and need them for all use
cases. The grammar is incomplete, and even wrong in some ways. But I
can't just deliver the final correct and complete paper without feedback.
At the end, I want a understandable, readable, meaningful, complete paper
which is of publication quality and actionable for any skilled developer.
So I ask -- edit, review, critique, criticize, shred, and otherwise pick
out everything wrong in every way that can be done. Preferably politely.
Unifying File Formats.pdf
Description: Adobe PDF document
\documentclass[]{article}
\usepackage[normalem]{ulem}
\usepackage{titling}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{indentfirst}
\usepackage{syntax}
\title{Universal File Format and Supporting Infrastructure}
\author{
Marshall, Josh\\
\texttt{joshua.r.marshall.1...@gmail.com}
}
\date{\today}
%\setlength{\parindent}{0pt}
\setlength{\grammarparsep}{10pt plus 1pt minus 1pt} % increase separation between rules
\setlength{\grammarindent}{12em} % increase separation between LHS/RHS
\begin{document}
\maketitle
\begin{abstract}
Similar to how the most complete meaning of a 'function' is deceptively complex, so too is the handling of computer files. This fundamental process of serializing and deserializing between active memory, and a format suitable for non-volatile storage and transmission has incurred massive costs over time. Many programs have developed custom serializers and deserializers. This has resulted in disunity of file formats which has in turn led to encoding inefficiencies, lost backwards and forwards compatibility, portability failures, and redundant work. Performing this work for each program is expensive. This paper proposes and formalizes a new model to file handling to ease developer burden, increase space efficiency, improve system integration, allow for much greater compatibility, robust security, transparent sharding across disparate systems, and future extensibility while lowering overall developer burden.
\end{abstract}
\section{Introduction}
This paper is written in \LaTeX, compiled to a pdf, and will likely be read online in HTML, Markdown, or ReStructured Text. They will all encode the same fundamental information in mutually incompatible ways. Yet, a system accessing these formats will still display them similarly. In system memory, their decoded state can be made, in theory, identical. This is not to say that any of these formats should replace all of the others, but it does exemplify the complexity and divergence in performing a similar simple task in computing.
Imagine the workflow for writing this document -- a program runs where a user enters some data into memory through a text editor, that memory is transformed by some serialization function into an on-disk format which is given to the host system to store through some system provided file writing function, stored by the host system, and can then be read through some system provided function, then deserialized and interpreted by some deserialization function. This series of actions describe the fundamental actions taken on file:
\begin{enumerate}
\item interface to modify content in working memory
\item transform from working memory to non-volatile memory or to stream
\item transfer to non-volatile state or by stream
\item transform from non-volatile state or from stream
\item process for use
\end{enumerate}
It is in these steps that developer burden can be lessened. Lessening the difference between working memory and non-volatile memory in order to reduce the burden of processing input. Another is by lessening the difference of transferring to non-volatile forms and making data accessible over a network. These differences arise in part from encoding efficiency, endianness, and runtime data not relevant towards the model stored in saved files.
All in memory data structures can be practically represented by records (non-ordinal, possibly key-value), lists(ordinal), real numbers, integer numbers, booleans, null, and character strings. This has been noticed and designed around in SGML, JSON, TOML, YAML, and EDN. These formats are reliably interconvertible except for SGML because SGML allows representing identical information in semantically different ways.
Another more powerful design is available in the programming language Lisp, in particular the Scheme dialect. Given app
Re: Rethinking files as a concept -- review draft paper
Spelling and grammar fixes. Automatic tools have become insufficient to
catch everything.
On Tue, Mar 31, 2020 at 9:33 PM Josh Marshall <
joshua.r.marshall.1...@gmail.com> wrote:
> I was on a while back about using IPFS or some other CAS to cache
> computation and results. I iterated on this concept with metadata, then
> data, then just files as a fundamental concept. It became abundantly clear
> that how we use, think, and need from and about files are not what computer
> files are. They are currently unstructured inconsistent blobs. We want
> them to be secure, distributed, and structured. In the attached, I attempt
> to create the concepts to allow such a transition from files as
> unstructured blobs to how we actually want and need them for all use
> cases. The grammar is incomplete, and even wrong in some ways. But I
> can't just deliver the final correct and complete paper without feedback.
>
> At the end, I want a understandable, readable, meaningful, complete paper
> which is of publication quality and actionable for any skilled developer.
>
> So I ask -- edit, review, critique, criticize, shred, and otherwise pick
> out everything wrong in every way that can be done. Preferably politely.
>
Unifying File Formats.pdf
Description: Adobe PDF document
\documentclass[]{article}
\usepackage[normalem]{ulem}
\usepackage{titling}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{indentfirst}
\usepackage{syntax}
\title{Universal File Format and Supporting Infrastructure}
\author{
Marshall, Josh\\
\texttt{joshua.r.marshall.1...@gmail.com}
}
\date{\today}
%\setlength{\parindent}{0pt}
\setlength{\grammarparsep}{10pt plus 1pt minus 1pt} % increase separation between rules
\setlength{\grammarindent}{12em} % increase separation between LHS/RHS
\begin{document}
\maketitle
\begin{abstract}
Similar to how the most complete meaning of a 'function' is deceptively complex, so too is the handling of computer files. This fundamental process of serializing and deserializing between active memory, and a format suitable for non-volatile storage and transmission has incurred massive costs over time. Many programs have developed custom serializers and deserializers. This has resulted in disunity of file formats which has in turn led to encoding inefficiencies, lost backwards and forwards compatibility, portability failures, and redundant work. Performing this work for each program is expensive. This paper proposes and formalizes a new model of file handling to ease developer burden, increase space efficiency, improve system integration, allow for much greater compatibility, robust security, transparent sharding across disparate systems, and future extensibility.
\end{abstract}
\section{Introduction}
This paper is written in \LaTeX, compiled to a pdf, and will likely be read online in HTML, Markdown, or ReStructured Text. They will all encode the same fundamental information in mutually incompatible ways. Yet, a system accessing these formats will still display them similarly. In system memory, their decoded state can be made, in theory, identical. This is not to say that any of these formats should replace all of the others, but it does exemplify the complexity and divergence in performing a similar simple task in computing.
Imagine the workflow for writing this document -- a program runs where a user enters some data into memory through a text editor, that memory is transformed by some serialization function into an on-disk format which is given to the host system to store through some system provided file writing function, stored by the host system, and can then be read through some system provided function, then deserialized and interpreted by some deserialization function. This series of actions describe the fundamental actions taken on file:
\begin{enumerate}
\item interface to modify content in working memory
\item transform from working memory to non-volatile memory or to stream
\item transfer to non-volatile state or by stream
\item transform from non-volatile state or from stream
\item process for use
\end{enumerate}
It is in these steps that developer burden can be lessened. Lessening the difference between working memory and non-volatile memory in order to reduce the burden of processing input. Another is by lessening the difference of transferring to non-volatile forms and making data accessible over a network. These differences arise in part from encoding efficiency, endianness, and runtime data irrelevant to the model stored in saved files.
All in memory data structures can be practically represented by records (non-ordinal, possibly key-value), lists(ordinal), real numbers, integer numbers, booleans, null, and character strings. This has been noticed and designed around in SGML, JSON, TOML, YAML, and EDN. These formats are reliably interconvertible except for SGML because SGML allows representing identical
Re: Brainstorming features for issues.guix.gnu.org
Hi Björn, thanks for the list of suggestions! > * First of all, I find this mixture very confusing: Is this about bugs > or is this about patches? I really don't know, and the start page is > ambigious about it too: > > "Guix /patch/ tracker > This is a web frontend to the Guix /issue/ trackers. > /Issues/ of interest > Priority /bugs/" I updated the text, but I think it’s not a good idea to have two separate trackers, because bug reports can have patches. > * Is it intentionally that there still is a http site? I would have > expected a redirect onto https. > > * http[s]://issues.guix.info/ is also still alive. Shouldn't it redirect > to http[s]://issues.guix.gnu.org/? This should be changed in maintenance.git. I would like a redirect to https://issues.guix.gnu.org/. I haven’t done this yet as I’m only focusing on mumi itself at this point. > * If you search for something, you get away from the homepage and > tips are missing. There should be a "help/tooltip" button near the > search input that explains the query language. The search results page now also has the search input widget with hints. Not sure how to handle the little search field in the menu bar, though. > * If you enter a query, that query should be kept in the input field. > Currently, it is discarded. That would make it more easy to update your > query. Yes, this has annoyed me too. The query is now kept. > * It is not clear to me how long a closed issue is still visible on the > home-page. Is it still an "Issue of interest" if it is closed? The idea was that you may want to look at issues that have just been closed to catch up (or to reopen them). But I guess closed issues could just be hidden. > * Sorting issues by columns would be cool. Done. > * Mumi has no paging: It only presents one page of issues, but it > doesn't say how many are there in total nor does it page through all > other pages of issues. Yes, this is a restriction due to debbugs. We can’t ask it for all issues and implementing paging is a little annoying. I think this will change as I’m making more and more features independent of debbugs requests. > * In that list, it is not clear what a red/orange colored bug-number > means: A tool-tip would be nice. I never liked those colors. There’s a little icon now with a tool-tip. > * A long desired feature is having general tags. It took me a very long > time until I realized that Debbugs allows user-tags. What about using a > common email address like "issues AT guix.gnu.org" and add user-tags > for that address, like "python", "core-updates", "release-1.1.x", etc. Sounds like a good idea. I don’t know if it even has to be an email address or if we can fake it. This deserves a closer look. > * I still don't see that all bugs and patches from the mailing list are > part of the mumi issues list? There was a problem with the worker process that fetched and indexed messages from Debbugs. > * The HTML could be responsive. It should be responsive with recent changes. > * The query syntax examples are only on the home page. Please repeat > them also on the help page. That I don’t understand. The help page has *more* examples, no? > * Either I don't understand the "date:.." semantics or it > is broken. The range yesterday..today lists no issues after 2015. Not > the "yesterday" I expected...: > > https://issues.guix.gnu.org/search?query=date%3Ayesterday..today Yeah, this is embarrassing… “yesterday” isn’t actually valid. I only just noticed that after a much longer debugging session than I feel comfortable admitting. The “proper” way to do this is to write “date:2d..”. I updated the help. > With numbers, it gets even more strange: > https://issues.guix.gnu.org/search?query=date%3A2020-03-29..2020-03-30 > --> Nothing found. This seems to be correct now. > Some more, but why are these selected?: > https://issues.guix.gnu.org/search?query=date%3A2020-03-29..2020-03-30 …this is the same query as above. They are selected because the discussion contains messages in the given range. > * Mdate does not find anything? > https://issues.guix.gnu.org/search?query=mdate%3A2w..12h mdate no longer exists (it was a debbugs search filter), but maybe it should be added again to distinguish between message dates and submission dates. I’ll add this to my list. Thanks again! -- Ricardo PS: When I say that something’s fixed I mean that it’s fixed in the repository, not necessarily in the public instance.
Kernel module separation
Hello. As of now, building a system requires that the kernel package includes the lib/modules directory, probably with at least some loadable modules, which is not necessary the case for custom packages. This can be fixed trivially with a check in operating-system-directory-base-entries: (packages->manifest (append (if (file-exists? #$(file-append kernel "/lib/modules")) (list kernel) '()) modules)) But may be it would be better to make a separate output for the linux-libre modules and make kernel-loadable-modules include it by default.
Re: [GSoC 2020] [Final Proposal] Clojars importer for Guix
Hi Leandro, A quick message to let you know that I'm very excited by your project! I've recently started developing with Clojure and I'd love to see a Clojars importer become reality! Good luck! -- Pierre Neidhardt https://ambrevar.xyz/ signature.asc Description: PGP signature
