Ralf Wildenhues wrote:
* John E. Malmberg wrote on Fri, Nov 18, 2005 at 09:26:14PM CET:
Because of differences between *NIX and OpenVMS in this area, it looks
like it may not be possible to make generating shared images on OpenVMS
totally transparent to a build procedure with out some cooperation with
the maintainers of the package.
I'm not sure yet. For a large subset of packages, it may be possible.
Item 3 below from my first post is the sticky point for shared images.
3. For dot-n releases, in order to provide upward compatibility so that
new shared libraries can be used by older programs, the list of public
symbols must always be in the same order, with new symbols being added
to the end.
Ouch. This is a new requirement. So in general the author of the
library has to provide the list of symbols, it can't be generated.
Right?
Mostly right. A generated approach can be taken as long as the
generating program has the previous generation of the file as a base and
the generated output is distributed with the source, and understands the
rules for generating the output.
It does cause problems if the project spins off a new branch that needs
backwards compatibility, as the list of symbols must be maintained by
the head branch.
For full compatibility, Microsoft Windows DLL files may have the same
requirement. Microsoft Windows DLL routines can be invoked either by
ordinal or by name. Change the order of the symbols to be exported, and
it changes the ordinal value, thus breaking any routine that calls by
ordinal.
And on a Microsoft based system, it appears that call by ordinal would
be faster than call by symbol name.
I do not know if the Microsoft Linker sets up images to load their DLLs
by ordinal or by name.
Effectively the OpenVMS linker sets up calling routines by ordinal, not
by name.
This probably means that we should have a new version_type for OpenVMS.
Special handing is needed if routine foo(x,y,z) changes to be foo(p,d,q)
in a newer dot-n version. The old version of foo would get a new name,
but retain it's position in the symbol table, and it needs to still
return something, if only an ENOSYS error code if it was really bad to
continue to use it. The new version of foo() would go to the end of the
table.
So for OpenVMS, some history needs to be maintained for the full
advantages of shared images to be used.
Hmm. This feature does not map on features commonly found in other
shared library systems (altough of course there are systems with
versioned symbols). So, probably the first support for OpenVMS in
Libtool should disallow and need not support symbols with changed
interface, since for portable programs, you cannot use them either.
We should then strive to add support for this together with adding
support for versioned symbols on other systems.
It is pretty hard to "disallow" this feature because with out a history,
it is impossible for the build procedure to automatically know if it is
building a shared image that is incompatible with the previous version.
The current hack around this it to set the flags so that a relink is
always required for a new version. This removes one of the benefits of
using a shared image. Especially if it is something that only one copy
is expected to be run at a time.
Supplying a old/new version of a public routine is something that a
programmer needs to do. I do not think that most change control systems
will flag if someone makes an incompatible change to a program's parameters.
Also, public symbols on OpenVMS are traditionally case insensitive and
stored in uppercase and limited to 31 characters. They can also be case
sensitive. I have the ability to export symbols in both exact case and
uppercase.
The 31 character limit is a special case that I am trying to get some
tools to make easier. What happens is that if a public symbol is more
than 31 characters, the C compiler shortens to be much smaller and adds
a CRC type value to the end to keep it unique. That is the symbol name
that I have to put in the symbol table file.
Of course the current C standard states that only the first 31
characters of a public symbol are significant, but that the restriction
is planned to be removed in a future version of the standard.
Other issues:
Normally the eventual location of the shared library will reside is not
used at build time. A construct known as a logical name is used to find
it if the shared image is not in the /SYS$SHARE directory. For shared
libraries not supplied by the operating system, normally a logical name
is used.
A logical name works like a symbolic link, except that to the file
system it looks like it is always present in the in either the current
working directory or in the root directory when it is explicitly
referenced. A logical name can supersede access to a file of the same name.
When an application is built against a test shared library instead of a
production one, it makes a local logical name for this.
There is a naming convention for the logical names and program names
where prefixes can be reserved to prevent conflicts so that multiple
libraries with similar names can coexist. These prefixes are not
directory paths.
OK. I think this might need a bit of code, but is doable.
Note Libtool (in default mode) wants the user to be able to use the
uninstalled libraries, and has means to relink libraries and/or programs
upon either execution or installation.
With OpenVMS, no relink is needed, just change what the logical name is
pointing to. It is like if you could have a symbolic link mean
something different to a process tree than it does to the rest of the
system.
This is close to the issue that brought me to this conference, the
generated libtool script of one of the packages I am trying to build is
looking for the destination library path to be set up, and it does not
have any meaning on OpenVMS, so the caller of libtool did not set it up.
The build script on OpenVMS sets a logical name that points to the
uninstalled libraries, and the install script sets it to point to the
installed libraries. The linked image does not change.
A linker option file contains the shared libraries that will be
referenced by the image being built.
$ type svn_client.opt;3
LIBSVN_SHARE/SHARE
APACHE$APR_SHR/SHARE
APACHE$APU_SHR/SHARE
[End-of-file]
During the build, LIBSVN_SHARE points to image that was just built. At
run-time, a system wide definition is installed to point to the
production version of the library.
Essentially sometime before the LIBSVN_SHARE is used to build something,
the build script must do something like "DCL DEFINE/JOB LIBSVN_SHARE
LCL_ROOT:[LIB]LIBSVN_SHARE.EXE". This definition must be cleaned up
after the build is complete because it will persist for the life of the
job tree containing the build script. And I also need the name to be in
VMS syntax, which is hard to do from a bash shell script at this time.
In a traditional OpenVMS build environment, all the building takes place
in a single process, so a temporary logical name is used that
automatically self destructs when the script ends.
Also in this case the prefix "APACHE$" is reserved to to files that are
part of the Apache web server. The "$" in the name indicates the name
has been registered.
By changing APACHE$APR_SHR definition, I can change either the test or
production version of svn_client to use a compatible version of that
library.
Well, Libtool has postinstall_cmds and finish_cmds and all that, and you
can stick pretty much arbitrary code in there.
Of course that would require a programmer familiar with VMS to put in
such support for each project.
Incremental linking does not exist on OpenVMS. I can fake it, but all
that does is use more disk space.
Well, it's needed mostly to cope with too large command line lengths.
Does OpenVMS have a high limit there?
Current versions support 4096 characters. Older only 1024.
What has been done in the past for large number of files is either to
populate an object library, or to put the filenames in a linker option
file. Simply a list of files, usually one per line for readability. I
would have to check if commas are required.
Another question for you: I remember a VMS version for Autoconf being
announced on its mailing list some time ago. Do you use this in order
to get working configure scripts on OpenVMS? Is GNU Autoconf portable
to OpenVMS these days? I also wonder how much of the shell scripting
of Libtool will just break on VMS..
I tried to run a very recent version of Autoconf. It failed because it
seems to require enhancements to Perl on OpenVMS. Currently Perl on
OpenVMS assumes a DCL shell environment. I am working on that, but it
will take some time.
Currently for the project I need to run Autoconf on, I have an NFS
client on a LINUX system that runs it for me after refreshing the
changes in that directory from the previous day.
A couple of notes, before you start hacking: we generally like patches
against the CVS HEAD version of Libtool. Unintrusive patches may be
backported to branch-1-5. Especially for libltdl, working with CVS HEAD
should be easier.
A CVS client has been ported to OpenVMS. I will see if I can find it.
Otherwise is there ftp access to the unpacked head files?
For large contributions, the FSF requires either a copyright assignment
or disclaimer; the assignment paperwork takes a while, so it may be
helpful to start early with this (and to check whether you may have any
issues with this). Other than that, patch discussions are best handled
on the libtool-patches mailing list.
For large contributions to open source projects, I have to go through a
central corporate office.
Feel free to ask questions about anything you don't understand, show
whatever changes you're working with -- we may be able to give useful
hints.
This is a fragment of the option file that I manually generated for zlib
1.2.3. An image linked against it will continue to work with newer
versions as long as the second number on the GSMATCH is equal to or
greater than the value present, and the first number is the same.
I encoded the minor version numbers in that section, but it might be
more appropriate to use the commit numbers from a central change control
library. Using the major version number as the first number may not be
a good idea, and I may change that again before I submit that change to
the ZLIB maintainers.
The symbols are exported both in exact case and in upper case so that
the users of the library do not need to change their build procedures to
require exact case, but the library can be called by modules that need
to be built using exact case. The uppercase version of the symbol is
the alias assigned by the linker.
[start of fragment]
GSMATCH=LEQUAL,1,200300
CASE_SENSITIVE=YES
symbol_vector=(zlibVersion=PROCEDURE)
symbol_vector=(ZLIBVERSION/zlibVersion=PROCEDURE)
[end-of-fragment]
I am not all that skilled with bash shell scripting. I have been on
vacation for the past week, so I have to get back to the specific build
issues that I was seeing to be more specific. I also am having to
create/fix the VMS versions of some UNIX utilities.
Regards,
-John
[EMAIL PROTECTED]
Personal Opinion Only
_______________________________________________
http://lists.gnu.org/mailman/listinfo/libtool
