The progress in this discussion is not as fast as I'd hoped it would be, but
this demonstrates how hard a problem this is to solve. I propose a solution in
this mail, that I hope will move the decision a bit closer to reality.
We have a variety of criteria to the scripting language that need to be
addressed. These are very different in nature, and thus it seems hard to come
up with an existing language that satisfies us all.
Therefor, I reluctantly proposed the idea of inventing our own Evil Scripting
Language, but that has not be welcomed with enthuisiasme (and I was not
enthuisiastic about it either), mostly because this would take away resources
from other things with limited benefits.
The main remaining problems with the so far best candiate language, Python,
are:
1) It's not bundable.
2) It's not clear how the security concern could be solved. (And this prevents
us from using the external approach.)
3) It's still a bit unclear how easy it is to link it. (Although the vim people
seem to have a solution.)
These problems go very much against adopting Python as the "official" scripting
language. (At least it seems that everybody agree that we should have an
"official" scripting language.)
The other serious candidate scripting language so far is Scheme.
And the only known disadvantage of this is that some feel that it's not
appropriate for beginners. This feeling is based on syntax, and the fact that
Scheme is not an imperative language.
Therefor I propose that we do this:
Make Scheme the primary candidate language, in the form of a very small and
thus bundable scheme, like SIOD. So we would declare Scheme the "official"
scripting language, that all LyX come with.
Then, next to this, we have a language that is designed to be simple and easy
for beginners. Specifically it will be a language with a syntax which is
designed to be as "natural" as possible. Also, it will be a imperative
language, because this is the most natural thing for a power user: A list of
commands that are executed after each other, just like the commands would
appear when they are recorded by LyX.
This language would be very small, and might not be very expressive/powerful in
itself. It would however feature a hook into Scheme, such that any Scheme
expression could be evaluated, and thus, the language would, in a technical
sense, be as powerful as Scheme, although it would arguably be cumbersome to
use this power. But it would be, and the main goal of this simple language is
to cater for non-programmers that want to write small macros and other
non-complicated stuff.
And now we turn to the main idea for realizing this proposal: This simple,
"natural", imperative scripting language would be implemented in Scheme, and
thus bundled. And furthermore: It would be feasible to do this. It's a piece
of cake to write an interpreter in Scheme.
So the neat part with this dual solution is that we target both non-programmers
(LyX power-users) and the skilled programmers on the development team.
We have a powerful language that we can use for implementing essential LyX
features.
And we have a simple language for the "novices".
Alejandro has a list of requirements for our scripting language.
Here, I try to list the appropriate comments for this dual scripting language
approach.
> - compatibility between versions
The fundamental language of Scheme is fixed.
We totally control the compatibility of the simple scripting language.
> - what is necessary to, both, use it embeded or external
We would only use embedded languages for starters. (Of course, we can provide
sockets for other external languages, but this would be similar to how we
provide the LyXServer now. In other words: A LyXServer on steroids. In
particular, we might provide a HOWTO tutorial on how to use Python for external
scripting, and other languages such as TCL and Perl.)
> - how easy is to learn it for novices
It's disputable how easy Scheme is to learn for users. But we know that at
least that some of the developers like it, and programmers that have hacked
Emacs will feel at home. Instantly, we gain a large market of potential
hackers.
Our own language would be designed to be easy to learn for non-programmers.
> - simplicity (scheme syntax is simple but confuse for non experts)
Scheme is simple.
Our own language will also be simple. The goal is that the core language can
be explained one very few pages.
> - we should choice as our official scripting language one that is simple
> and that can be included in lyx so users don't have to install a bunch
> of libraries.
This is met in the dual approach, because we bundle both languages (and without
bloat, because SIOD is small, and an interpreter for a simple language is also
small: I think it could be written in much less than 10K.).
> - this language would be aimed only to those that knows what are they
> doing, given that we would replace with it some critical lyxfunc
> functions.
>
> - some critical and unsecure functions would be allowed only by
> this way and not by the server.
Since we embed, we control this completely. And also, we provide the complete
"sand-box" scripting language that can not do anything harmful.
The security aspect of the simple language is as simple as saying whether the
"eval-scheme" function is allowed or not.
For the Scheme language, we need a bit more work, but this is needed for any
scripting language.
> - the server should be powerful and safe enough to allow people wrote
> their own external script front end using whatever they want, all of them
> optative to the rest of the users.
Indeed. For instance, one LyXServer command would be to execute this piece of
Scheme, or this piece of the Simple Evil Scripting Language.
So what do you think? Should I download SIOD and try to embed it into 1.1?
And then, over time, the simply Evil Scripting Language would be developed over
time.
The main problem remaining with this solution is to decide how to design the
simple language. But since we have another language that seems to fit the
needs of many developers, the disagreements would be managable, and a big
conflict very unlikely to develop. If someone disagrees too much, we could
disarm them by pointing to Scheme, which arguably solves all problems except
for syntax.
In particular, I think I could get away with designing the language more or
less the way I think it should be.
Notice that all of this seems like a very ambitious project, and in a way it
is, because the perspectives of a scripting language are far. But the amount
of work involved in this project is not extreme: I embedded Python in one
hour. SIOD can be embedded in one hour too. Then, the hooks for the LyX data
structures can be done in a weekend, and we have the fundation.
The simple scripting language can be implemented in another weekend (especially
if we can dig up an interpreter for a simple language, written in Scheme, and
hack that, but this should be easy.)
The thing that would take the most energy would be to document it all.
Greets,
Asger
