My point was that anything you do in Pharo will lower performance quite
a bit, but it wont be noticable anyway unless its in a big loop or
something.
Second that Pharo cannot debug such issues which is a huge deal when you
work with OS stuff and OS calls.
You are far better doing this outside UFFI and straight inside a C IDE
because you will have an army of tools that will make your life a ton
easier.
Hence why I talked about performance, C coding and Pharo crashes.
In my case I had to write and test the code in C to port it to UFFI
because it was impossible to resolve the bugs from inside Pharo. Also
crashing Pharo all the time got really annoying and I managed to corrupt
a couple of images too in the process.
Of course debugging C code from a C IDE is quite similar to how you work
with Pharo.
I also got an experience with loading libraries because I worked with
Shared memory mapped files which is the technology OS uses to load DLLs.
Its really low level stuff that come directly from OS kernel. It was a
ton of fun.
Also C is easier because obviously there is a ton of docs and all you
have to do is copy paste. In case of UFFI you have to first figure out
the C code and then how to convert it to UFFI so its double the effort
for little gain. Its much better thing to just wrap C code and minimize
the usage of UFFI.
The benefits of Pharo are lost anyway as soon as you drop so low level.
On Thu, Mar 16, 2017 at 3:05 PM <raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>> wrote:
Indeed, experimenting using some combination of AddDllDirectory(),
SetDllDirectory() and LoadLibraryEx() was in my plan for next week.
Thanks
On 16/03/17 11:59, Mark Bestley wrote:
> Yes the calling process can alter where these indirect DLLs are loaded
> from and so should be able to be done in UFFI and directly from
Smalltalk
>
> The OS will load the DLL specified then it runs DllMain and doing that
> causes the OS to load any DLLs that it calls
>
> MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>
> With both implicit and explicit linking, Windows first searches for
> "known DLLs", such as Kernel32.dll and User32.dll. Windows then
searches
> for the DLLs in the following sequence:
> The directory where the executable module for the current process is
> located.
> The current directory.
> The Windows system directory. The GetSystemDirectory function
retrieves
> the path of this directory.
> The Windows directory. The GetWindowsDirectory function retrieves the
> path of this directory.
> The directories listed in the PATH environment variable.
>
>
> This can be changed using the call LoadLibraryEx instead of
LoadLibrary
> Flags can be set to change the directories used
>
>
https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>
>
>
> Also see AddDllDirectory
>
https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>
>
> Mark
>
> On 15/03/2017 17:22, Raffaello Giulietti wrote:
>> Hi Dimitris,
>>
>> I don't get your point about proper C coding, C performance or Pharo
>> crashes. It is not the focus of this thread.
>>
>> How dependencies between libraries are resolved at load-time is
dictated
>> by the OS. If the OS offers a working mechanism to specify
folders that
>> the OS uses to search for libraries at load-time, well, I think that
>> Pharo, hence the UFFI, should offer it as well and in a more
fashionable
>> format.
>> UFFI is quite elegant in many respects but I wouldn't call it a
>> minimalist design (zero hand holding, if I understand you correctly).
>> But I'm happy the extra elegance is there because it makes life more
>> comfortable.
>>
>> Again, I still have to experiment with SetDllDirectory() in the
case of
>> Windows, so I'm not in a position to say something authoritative yet.
>>
>>
>>
>>
>>
>>
>>
>> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>>> Please note the moment you use a FFI of any language you step to a C
>>> territory. VMs tend not to touch the code because the code is
already
>>> in machine code format.
>>>
>>> There is zero reason for UFFI to worry about the dependencies of a
>>> library loaded, this must happen at the library level that UFFI does
>>> not focus on.
>>>
>>> What you describe here is standard practice because the vast
majority
>>> of libraries depend on other libraries. In the end UFFI or any
FFI in
>>> any language is not there to replace proper C coding but rather to
>>> allow you to utilize C libraries. You cannot hope to achieve
>>> performance comparable to C if you try to use a language that is
not C
>>> to do standard things like dependency management.
>>>
>>> If the library fail to load it wont be because of UFFI it will be
>>> because the library is buggy at handling its dependencies. That
means
>>> that the library will be buggy even if you use it from C directly.
>>>
>>> Even you do not know what you doing at C level, you going to crash
>>> Pharo countless times anyway and the least of your worries will be
>>> handling dependencies which far more obvious than some other nasty C
>>> bugs.
>>>
>>> I think its a great idea that UFFI does zero hand holding. This
way it
>>> gives the clear message that if you do not understand C then you
>>> should not be using the UFFI in the first place.
>>>
>>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>>> <esteba...@gmail.com <mailto:esteba...@gmail.com>
>>> <mailto:esteba...@gmail.com <mailto:esteba...@gmail.com>>> wrote:
>>>
>>>
>>> > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>> <raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>>>
>>> wrote:
>>> >
>>> > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>> >>
>>> >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>> <raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>>>
>>> wrote:
>>> >>>
>>> >>> Hi Esteban,
>>> >>>
>>> >>> I understand this is the current status of the UFFI, so
I can
>>> certainly use the workarounds discussed below.
>>> >>>
>>> >>> But I hope UFFI will soon offer a more "declarative" way to
>>> specify the search folders, kind of LD_LIBRARY_PATH
mechanism but
>>> in the UFFI.
>>> >>
>>> >> that will NOT solve the problem of dependencies of libraries
>>> indirectly references.
>>> >> There is no way to do that (indirect reference solving) on an
>>> executing program that I know.
>>> >> So is not “current status of UFFI”: it will never provide
that.
>>> >>
>>> >
>>> > Do you mean that SetDllDirectory() has no influence when
invoked
>>> between Pharo's start and the LoadLibrary() call done by the
UFFI?
>>>
>>> I really have no idea. You can try to add it to your app.
>>>
>>> Esteban
>>>
>>> >
>>> >
>>> >
>>> >
>>> >> All that, unless someone come and says: "look, Esteban is
like
>>> this”.
>>> >>
>>> >> I spent last three weeks (not full time of course) trying
to do
>>> exactly that for our current linux vm, to force libgit2 to
use the
>>> version of libssh2 we want (and not the version on system)…
and I
>>> failed. I solved some cases, but not all. So at the end I
opted to
>>> change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>> Only way I found I can force the lookup paths.
>>> >>
>>> >> cheers,
>>> >> Esteban
>>> >>
>>> >>>
>>> >>>
>>> >>>
>>> >>> Greetings
>>> >>> Raffaello
>>> >>>
>>> >>>
>>> >>>
>>> >>>
>>> >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>> >>>> Hi,
>>> >>>>
>>> >>>> UFFI cannot do what you want.
>>> >>>> If I understand well, you have:
>>> >>>>
>>> >>>> Pharo -> libA.dll -> libB.dll
>>> >>>>
>>> >>>> Pharo does a LoadLibrary(libA.dll), but has no control
on how
>>> libA calls libB… and is not possible for us to influence it more
>>> than the predefined platform ways.
>>> >>>>
>>> >>>> One posible workaround (just possible, no idea if it will
>>> work) is to perform a load of libB before is required by libB.
>>> Then hopefully the loader will understand is same library
and will
>>> answer same handler (but not sure, because if it
discriminates by
>>> path… then you’re in the same position). Anyway, if you want to
>>> try it, it would be something like this:
>>> >>>>
>>> >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’.
>>> >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’.
>>> >>>>
>>> >>>> then you can try to execute functions of libA…
>>> >>>>
>>> >>>> cheers,
>>> >>>> Esteban
>>> >>>>
>>> >>>>
>>> >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>> <raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>>>
>>> wrote:
>>> >>>>>
>>> >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>> >>>>>>
>>> <raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>>
>>> >>>>>>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>
>>> <mailto:raffaello.giulie...@lifeware.ch
<mailto:raffaello.giulie...@lifeware.ch>>>>
>>> wrote:
>>> >>>>>>
>>> >>>>>> Hi Ben,
>>> >>>>>>
>>> >>>>>> my understanding is that SetDllDirectory only
affects the
>>> search
>>> >>>>>> path of libraries that are loaded at *run-time* with
>>> LoadLibrary.
>>> >>>>>>
>>> >>>>>> What I'm asking for is a mechanism that works at
>>> *load-time*, when a
>>> >>>>>> library I'm accessing directly depends on another one
>>> which I'm not
>>> >>>>>> targeting directly.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I don't quite follow. I'm not intimate with Windows
>>> dynamic loading, so
>>> >>>>>> these questions are an opportunity for me to learn...
>>> >>>>>>
>>> >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>> of OTHER.DLL ?
>>> >>>>>>
>>> >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>> YOUR.DLL before
>>> >>>>>> its uses the function from OTHER.DLL?
>>> >>>>>>
>>> >>>>>
>>> >>>>> During the build of YOUR.DLL, you can specify that it
should
>>> depend on OTHER.LIB (kind of binary which lists function headers
>>> and static data declarations for OTHER.DLL). Linking
resolution is
>>> then done at load-time of YOUR.DLL, not by invoking
>>> LoadLibrary("OTHER.DLL").
>>> >>>>>
>>> >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>> of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>> to load OTHER.DLL because of the linking information gathered
>>> during the build. Notice that the linking information does *not*
>>> include a specific path for OTHER.DLL.
>>> >>>>>
>>> >>>>> (You don't have to mention standard *.LIBs like
Kernel32.lib
>>> because they are included by default. But of course, Windows
knows
>>> where to find the corresponding .dll.)
>>> >>>>>
>>> >>>>> OTHER.DLL is searched using the strategy described in the
>>> MSDN page you mention. My expectation is/was that UFFI
has/had an
>>> API for specifying the search paths where to find OTHER.DLL in a
>>> platform-independent way. In other words, I would expect the
UFFI
>>> to do the call of SetDllDirectory() or whatever is needed on
>>> Windows for me or to use other mechanisms on other platforms.
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile
time?
>>> >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>> FFI, so neither
>>> >>>>>> is OTHER.DLL,
>>> >>>>>> and Dynamic-Link Library Search Order [1] says... "If
a DLL
>>> with the
>>> >>>>>> same module name
>>> >>>>>> is already loaded in memory, the system uses the loaded
>>> DLL, no matter
>>> >>>>>> which directory it is in.
>>> >>>>>> The system does not search for the DLL."
>>> >>>>>>
>>> >>>>>> So maybe before calling any of YOUR.DLL,
>>> >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>> >>>>>> and later when you call into YOUR.DLL,
>>> >>>>>> OTHER.DLL is already loaded.
>>> >>>>>>
>>> >>>>>
>>> >>>>> Yes, this might work but is rather contrived.
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>> Otherwise it would seems(?) your options are either
Manifests
>>> >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>> Pharo.
>>> >>>>>>
>>> >>>>>
>>> >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my
control?
>>> The only thing I know is where the .dll and the
corresponding .lib
>>> + .h are.
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> Thanks for your interest
>>> >>>>> Raffaello
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>>> But I'll try whether SetDllDirectory also affects
>>> load-time searches.
>>> >>>>>>
>>> >>>>>> Anyway, I hope future versions of the UFFI to offer
>>> setting the
>>> >>>>>> search paths more easily.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Greetings
>>> >>>>>> Raffaello
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >
>>> >
>>>
>>>
>>
>
>
