My point was that anything you do in Pharo will lower performance quite a bit, but it wont be noticable anyway unless its in a big loop or something.
Second that Pharo cannot debug such issues which is a huge deal when you work with OS stuff and OS calls. You are far better doing this outside UFFI and straight inside a C IDE because you will have an army of tools that will make your life a ton easier. Hence why I talked about performance, C coding and Pharo crashes. In my case I had to write and test the code in C to port it to UFFI because it was impossible to resolve the bugs from inside Pharo. Also crashing Pharo all the time got really annoying and I managed to corrupt a couple of images too in the process. Of course debugging C code from a C IDE is quite similar to how you work with Pharo. I also got an experience with loading libraries because I worked with Shared memory mapped files which is the technology OS uses to load DLLs. Its really low level stuff that come directly from OS kernel. It was a ton of fun. Also C is easier because obviously there is a ton of docs and all you have to do is copy paste. In case of UFFI you have to first figure out the C code and then how to convert it to UFFI so its double the effort for little gain. Its much better thing to just wrap C code and minimize the usage of UFFI. The benefits of Pharo are lost anyway as soon as you drop so low level. On Thu, Mar 16, 2017 at 3:05 PM <raffaello.giulie...@lifeware.ch> wrote: > Indeed, experimenting using some combination of AddDllDirectory(), > SetDllDirectory() and LoadLibraryEx() was in my plan for next week. > > Thanks > > > > On 16/03/17 11:59, Mark Bestley wrote: > > Yes the calling process can alter where these indirect DLLs are loaded > > from and so should be able to be done in UFFI and directly from Smalltalk > > > > The OS will load the DLL specified then it runs DllMain and doing that > > causes the OS to load any DLLs that it calls > > > > MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says > > > > With both implicit and explicit linking, Windows first searches for > > "known DLLs", such as Kernel32.dll and User32.dll. Windows then searches > > for the DLLs in the following sequence: > > The directory where the executable module for the current process is > > located. > > The current directory. > > The Windows system directory. The GetSystemDirectory function retrieves > > the path of this directory. > > The Windows directory. The GetWindowsDirectory function retrieves the > > path of this directory. > > The directories listed in the PATH environment variable. > > > > > > This can be changed using the call LoadLibraryEx instead of LoadLibrary > > Flags can be set to change the directories used > > > > > https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx > > > > > > > > Also see AddDllDirectory > > > https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx > > > > > > Mark > > > > On 15/03/2017 17:22, Raffaello Giulietti wrote: > >> Hi Dimitris, > >> > >> I don't get your point about proper C coding, C performance or Pharo > >> crashes. It is not the focus of this thread. > >> > >> How dependencies between libraries are resolved at load-time is dictated > >> by the OS. If the OS offers a working mechanism to specify folders that > >> the OS uses to search for libraries at load-time, well, I think that > >> Pharo, hence the UFFI, should offer it as well and in a more fashionable > >> format. > >> UFFI is quite elegant in many respects but I wouldn't call it a > >> minimalist design (zero hand holding, if I understand you correctly). > >> But I'm happy the extra elegance is there because it makes life more > >> comfortable. > >> > >> Again, I still have to experiment with SetDllDirectory() in the case of > >> Windows, so I'm not in a position to say something authoritative yet. > >> > >> > >> > >> > >> > >> > >> > >> On 2017-03-15 17:41, Dimitris Chloupis wrote: > >>> Please note the moment you use a FFI of any language you step to a C > >>> territory. VMs tend not to touch the code because the code is already > >>> in machine code format. > >>> > >>> There is zero reason for UFFI to worry about the dependencies of a > >>> library loaded, this must happen at the library level that UFFI does > >>> not focus on. > >>> > >>> What you describe here is standard practice because the vast majority > >>> of libraries depend on other libraries. In the end UFFI or any FFI in > >>> any language is not there to replace proper C coding but rather to > >>> allow you to utilize C libraries. You cannot hope to achieve > >>> performance comparable to C if you try to use a language that is not C > >>> to do standard things like dependency management. > >>> > >>> If the library fail to load it wont be because of UFFI it will be > >>> because the library is buggy at handling its dependencies. That means > >>> that the library will be buggy even if you use it from C directly. > >>> > >>> Even you do not know what you doing at C level, you going to crash > >>> Pharo countless times anyway and the least of your worries will be > >>> handling dependencies which far more obvious than some other nasty C > >>> bugs. > >>> > >>> I think its a great idea that UFFI does zero hand holding. This way it > >>> gives the clear message that if you do not understand C then you > >>> should not be using the UFFI in the first place. > >>> > >>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano > >>> <esteba...@gmail.com > >>> <mailto:esteba...@gmail.com>> wrote: > >>> > >>> > >>> > On 15 Mar 2017, at 15:44, Raffaello Giulietti > >>> <raffaello.giulie...@lifeware.ch > >>> <mailto:raffaello.giulie...@lifeware.ch>> > >>> wrote: > >>> > > >>> > On 2017-03-15 15:20, Esteban Lorenzano wrote: > >>> >> > >>> >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti > >>> <raffaello.giulie...@lifeware.ch > >>> <mailto:raffaello.giulie...@lifeware.ch>> > >>> wrote: > >>> >>> > >>> >>> Hi Esteban, > >>> >>> > >>> >>> I understand this is the current status of the UFFI, so I can > >>> certainly use the workarounds discussed below. > >>> >>> > >>> >>> But I hope UFFI will soon offer a more "declarative" way to > >>> specify the search folders, kind of LD_LIBRARY_PATH mechanism but > >>> in the UFFI. > >>> >> > >>> >> that will NOT solve the problem of dependencies of libraries > >>> indirectly references. > >>> >> There is no way to do that (indirect reference solving) on an > >>> executing program that I know. > >>> >> So is not “current status of UFFI”: it will never provide that. > >>> >> > >>> > > >>> > Do you mean that SetDllDirectory() has no influence when invoked > >>> between Pharo's start and the LoadLibrary() call done by the UFFI? > >>> > >>> I really have no idea. You can try to add it to your app. > >>> > >>> Esteban > >>> > >>> > > >>> > > >>> > > >>> > > >>> >> All that, unless someone come and says: "look, Esteban is like > >>> this”. > >>> >> > >>> >> I spent last three weeks (not full time of course) trying to do > >>> exactly that for our current linux vm, to force libgit2 to use the > >>> version of libssh2 we want (and not the version on system)… and I > >>> failed. I solved some cases, but not all. So at the end I opted to > >>> change zeroconf to define LD_LIBRARY_PATH before calling pharo. > >>> Only way I found I can force the lookup paths. > >>> >> > >>> >> cheers, > >>> >> Esteban > >>> >> > >>> >>> > >>> >>> > >>> >>> > >>> >>> Greetings > >>> >>> Raffaello > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> On 2017-03-15 14:52, Esteban Lorenzano wrote: > >>> >>>> Hi, > >>> >>>> > >>> >>>> UFFI cannot do what you want. > >>> >>>> If I understand well, you have: > >>> >>>> > >>> >>>> Pharo -> libA.dll -> libB.dll > >>> >>>> > >>> >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how > >>> libA calls libB… and is not possible for us to influence it more > >>> than the predefined platform ways. > >>> >>>> > >>> >>>> One posible workaround (just possible, no idea if it will > >>> work) is to perform a load of libB before is required by libB. > >>> Then hopefully the loader will understand is same library and will > >>> answer same handler (but not sure, because if it discriminates by > >>> path… then you’re in the same position). Anyway, if you want to > >>> try it, it would be something like this: > >>> >>>> > >>> >>>> DynamicLoader loadLibrary: 'path/to/libB.dll’. > >>> >>>> DynamicLoader loadLibrary: 'path/to/libA.dll’. > >>> >>>> > >>> >>>> then you can try to execute functions of libA… > >>> >>>> > >>> >>>> cheers, > >>> >>>> Esteban > >>> >>>> > >>> >>>> > >>> >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti > >>> <raffaello.giulie...@lifeware.ch > >>> <mailto:raffaello.giulie...@lifeware.ch>> > >>> wrote: > >>> >>>>> > >>> >>>>> On 2017-03-15 13:51, Ben Coman wrote: > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti > >>> >>>>>> > >>> <raffaello.giulie...@lifeware.ch > >>> <mailto:raffaello.giulie...@lifeware.ch> > >>> >>>>>> > >>> <mailto:raffaello.giulie...@lifeware.ch > >>> <mailto:raffaello.giulie...@lifeware.ch>>> > >>> wrote: > >>> >>>>>> > >>> >>>>>> Hi Ben, > >>> >>>>>> > >>> >>>>>> my understanding is that SetDllDirectory only affects the > >>> search > >>> >>>>>> path of libraries that are loaded at *run-time* with > >>> LoadLibrary. > >>> >>>>>> > >>> >>>>>> What I'm asking for is a mechanism that works at > >>> *load-time*, when a > >>> >>>>>> library I'm accessing directly depends on another one > >>> which I'm not > >>> >>>>>> targeting directly. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> I don't quite follow. I'm not intimate with Windows > >>> dynamic loading, so > >>> >>>>>> these questions are an opportunity for me to learn... > >>> >>>>>> > >>> >>>>>> You can't mean when Pharo loads, because it wouldn't know > >>> of OTHER.DLL ? > >>> >>>>>> > >>> >>>>>> Do you mean that LoadLibrary is not called explicitly by > >>> YOUR.DLL before > >>> >>>>>> its uses the function from OTHER.DLL? > >>> >>>>>> > >>> >>>>> > >>> >>>>> During the build of YOUR.DLL, you can specify that it should > >>> depend on OTHER.LIB (kind of binary which lists function headers > >>> and static data declarations for OTHER.DLL). Linking resolution is > >>> then done at load-time of YOUR.DLL, not by invoking > >>> LoadLibrary("OTHER.DLL"). > >>> >>>>> > >>> >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means > >>> of LoadLibrary(), I guess), Windows (not the UFFI) also attempts > >>> to load OTHER.DLL because of the linking information gathered > >>> during the build. Notice that the linking information does *not* > >>> include a specific path for OTHER.DLL. > >>> >>>>> > >>> >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib > >>> because they are included by default. But of course, Windows knows > >>> where to find the corresponding .dll.) > >>> >>>>> > >>> >>>>> OTHER.DLL is searched using the strategy described in the > >>> MSDN page you mention. My expectation is/was that UFFI has/had an > >>> API for specifying the search paths where to find OTHER.DLL in a > >>> platform-independent way. In other words, I would expect the UFFI > >>> to do the call of SetDllDirectory() or whatever is needed on > >>> Windows for me or to use other mechanisms on other platforms. > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time? > >>> >>>>>> But still, YOUR.DLL it not loaded until called via Pharo > >>> FFI, so neither > >>> >>>>>> is OTHER.DLL, > >>> >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL > >>> with the > >>> >>>>>> same module name > >>> >>>>>> is already loaded in memory, the system uses the loaded > >>> DLL, no matter > >>> >>>>>> which directory it is in. > >>> >>>>>> The system does not search for the DLL." > >>> >>>>>> > >>> >>>>>> So maybe before calling any of YOUR.DLL, > >>> >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL) > >>> >>>>>> and later when you call into YOUR.DLL, > >>> >>>>>> OTHER.DLL is already loaded. > >>> >>>>>> > >>> >>>>> > >>> >>>>> Yes, this might work but is rather contrived. > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>>> Otherwise it would seems(?) your options are either > Manifests > >>> >>>>>> or Dynamic-Link Library Redirection, both done outside of > >>> Pharo. > >>> >>>>>> > >>> >>>>> > >>> >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control? > >>> The only thing I know is where the .dll and the corresponding .lib > >>> + .h are. > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> Thanks for your interest > >>> >>>>> Raffaello > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>> > >>> >>>>>> But I'll try whether SetDllDirectory also affects > >>> load-time searches. > >>> >>>>>> > >>> >>>>>> Anyway, I hope future versions of the UFFI to offer > >>> setting the > >>> >>>>>> search paths more easily. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Greetings > >>> >>>>>> Raffaello > >>> >>>>>> > >>> >>>>> > >>> >>>>> > >>> >>>> > >>> >>>> > >>> >>> > >>> >>> > >>> >> > >>> >> > >>> > > >>> > > >>> > >>> > >> > > > > > > >
