I just spend another hour with the rotten lib opencv.

Before you try anything else please patch it to use our ffmpeg and not download some random binaries.

The library also installs itself to x64\vc16\bin which does not match any sane layout.

On 15/02/2020 10:57, Gilles Caulier wrote:
Compare side by side the build and the factory opencv detection properties :

build.kde.org (working) :

01:53:02  -- OpenCV ARCH: x64
01:53:02  -- OpenCV RUNTIME: vc16
01:53:02  -- OpenCV STATIC: OFF
01:53:02  -- Found OpenCV:
C:/Craft/CI-Qt514/windows-msvc2019_64-cl-debug (found version "4.1.2")
found components:  core objdetect imgproc imgcodecs dnn flann
01:53:02  -- Found OpenCV 4.1.2 in
C:/Craft/CI-Qt514/windows-msvc2019_64-cl-debug/x64/vc16/lib
01:53:02  -- You might need to add
C:\Craft\CI-Qt514\windows-msvc2019_64-cl-debug\x64\vc16\bin to your
PATH to be able to run your applications.
01:53:02  -- OpenCV Root directory is:
C:/Craft/CI-Qt514/windows-msvc2019_64-cl-debug
01:53:02  -- OpenCV: Found version 4.1.2 (required: 3.3.0)
01:53:02  -- OpenCV headers:
C:/Craft/CI-Qt514/windows-msvc2019_64-cl-debug/include
01:53:02  -- OpenCV libs   :
opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann

binary-factory.kde.org (not working) :

3:45:27  -- OpenCV ARCH: x64
03:45:27  -- OpenCV RUNTIME: vc15
03:45:27  -- OpenCV STATIC: OFF
03:45:27  -- Found OpenCV:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl (found version "4.1.2")
found components: core objdetect imgproc imgcodecs dnn flann
03:45:27  -- Found OpenCV 4.1.2 in
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib
03:45:27  -- You might need to add
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\x64\vc15\bin to your
PATH to be able to run your applications.
03:45:27  -- OpenCV Root directory is:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl
03:45:27  -- OpenCV: Found version 4.1.2 (required: 3.3.0)
03:45:27  -- OpenCV headers:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/include
03:45:27  -- OpenCV libs   :
opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann

Important diff : compiler versions, debug mode, paths.

Note : In my office, we drop MSVC 2017 in favor to 2019 because it
play better with Cmake. There is a special reason to have different
Microsoft compiler on both workflow ?

Gilles

Le sam. 15 févr. 2020 à 04:47, Gilles Caulier
<caulier.gil...@gmail.com> a écrit :
Ok, yesterday, i patch the opencv python scrip to revert back to 4.1.2
version to see if DK link better on core object : no more...

03:45:27  -- OpenCV ARCH: x64
03:45:27  -- OpenCV RUNTIME: vc15
03:45:27  -- OpenCV STATIC: OFF
03:45:27  -- Found OpenCV:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl (found version "4.1.2")
found components: core objdetect imgproc imgcodecs dnn flann
03:45:27  -- Found OpenCV 4.1.2 in
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib
03:45:27  -- You might need to add
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\x64\vc15\bin to your
PATH to be able to run your applications.
03:45:27  -- OpenCV Root directory is:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl
03:45:27  -- OpenCV: Found version 4.1.2 (required: 3.3.0)
03:45:27  -- OpenCV headers:
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/include
03:45:27  -- OpenCV libs   :
opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann

...

04:38:58  FAILED: bin/digikamcore.dll lib/digikamcore.lib
04:38:58  cmd.exe /C "cmd.exe /C
"C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E __create_def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def.objs
&& cd C:\_\1ddac771\RelWithDebInfo-master" &&
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E vs_link_dll --intdir=core\app\CMakeFiles\digikamcore.dir
--rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
--mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp  /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def   && cd ."
04:38:58  LINK Pass 1: command
"C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def /MANIFEST
/MANIFESTFILE:core\app\CMakeFiles\digikamcore.dir/intermediate.manifest
core\app\CMakeFiles\digikamcore.dir/manifest.res" failed (exit code
1104) with the following output:
04:38:58  LINK : fatal error LNK1104: cannot open file
'opencv_core-NOTFOUND.obj'

So it setup back 4.2.0. The problem is in another place...

Gilles

Le jeu. 13 févr. 2020 à 12:07, Ben Cooksley <bcooks...@kde.org> a écrit :
On Thu, Feb 13, 2020 at 11:45 PM Gilles Caulier
<caulier.gil...@gmail.com> wrote:
Right, but the Q is : why with the CI, openCV detection work as expected?

There is a solution to drop definitively the cmake macro in DK to
handle openCV. With 4.x, OpenCV start to deploy the library
configuration through the target cmake scripts which is the modern
way.

I tried with openCV 4.1 to use only the target library scripts
provided by OpenCV and to drop all the macro in DK, but it do not work
well. With 4.2, this problem is certainly solved but i need to check
all case (Linux, MacOS, MinGW, and now MSVC).

The current code work everywhere, even in CI MSVC. So I ask the Q
again: why the binary build infrastructure has this kind of problems ?
Likely because the CI system is using the older OpenCV 4.1.x series,
as it was deployed before the update to OpenCV 4.2.
The Binary Factory on the other hand is using the current latest state
of Craft - which is OpenCV 4.2.

Cheers,
Ben

Gilles

Le jeu. 13 févr. 2020 à 05:11, Hannah von Reth <vonr...@kde.org> a écrit :
 From the logs:

-- OpenCV libs   : 
opencv_core;opencv_objdetect;opencv_imgproc;opencv_imgcodecs;opencv_dnn;opencv_flann

With modern cmake I'd expect absolute paths here, or at least the full names, 
-lfoo does not work on Windows.
Also, have a look on how opencv is installed.

00:19:58.081  -- Installing: 
C:/Craft/BinaryFactory/windows-msvc2017_64-cl/build/libs/opencv/image-RelWithDebInfo-4.2.0/Craft/BinaryFactory/windows-msvc2017_64-cl/x64/vc15/lib/opencv_core420.lib

Cheers,

Hannah

On 13.02.20 10:52, Gilles Caulier wrote:

Hannah,

In digiKam, there is a legacy cmake wrapper for OpenCV when we switch
from 1.x to 2.x, and later 3.x. OpenCV team has badly written the
cmake code to share information with client application. It's now
really better with 3.x and now 4.x.

In DK, code is here :

https://invent.kde.org/kde/digikam/blob/master/core/CMakeLists.txt#L238

https://invent.kde.org/kde/digikam/blob/master/core/cmake/modules/MacroOpenCV.cmake

In the past, this wrapper was really very complicated with plenty of
sub script due to the mess done by OpenCv to deploy the library
configuration. Now it's really more simpler but, perhaps it still a
little bug in this macro,

If yes, why CI build with MSVC work as expected when ?

Best

Gilles

Le jeu. 13 févr. 2020 à 04:00, Hannah von Reth <vonr...@kde.org> a écrit :

This doesn't look like a mix of build types, anyhow we only provide
release builds with symbols.

the big question is why does it try to to use an obj instead of a lib?

I'd check how you locate the lib and how you pass it to the target.

Some find scripts fail with msvc as they expect the wrong name for a
lib, msvc does not prepend everything with a "lib" by default so you
might need to add libopencv_core to the possible names.

I'm just guessing here but maybe it helps.


Cheers,

Hannah

On 13.02.20 04:57, Gilles Caulier wrote:

digiKAm start to compile now, but :

18:58:51  [1234/2395] Linking CXX shared library bin\digikamcore.dll
18:58:51  FAILED: bin/digikamcore.dll lib/digikamcore.lib
18:58:51  cmd.exe /C "cmd.exe /C
"C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E __create_def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def
C:\_\1ddac771\RelWithDebInfo-master\core\app\CMakeFiles\digikamcore.dir\exports.def.objs
&& cd C:\_\1ddac771\RelWithDebInfo-master" &&
C:\Craft\BinaryFactory\windows-msvc2017_64-cl\dev-utils\cmake-base\bin\cmake.exe
-E vs_link_dll --intdir=core\app\CMakeFiles\digikamcore.dir
--rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
--mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  --
C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp  /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def   && cd ."
18:58:51  LINK Pass 1: command
"C:\PROGRA~2\MICROS~1\2019\PROFES~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\link.exe
/nologo @CMakeFiles\digikamcore.rsp /out:bin\digikamcore.dll
/implib:lib\digikamcore.lib /pdb:bin\digikamcore.pdb /dll /version:7.0
/machine:x64 /debug /INCREMENTAL
/DEF:core\app\CMakeFiles\digikamcore.dir\exports.def /MANIFEST
/MANIFESTFILE:core\app\CMakeFiles\digikamcore.dir/intermediate.manifest
core\app\CMakeFiles\digikamcore.dir/manifest.res" failed (exit code
1104) with the following output:
18:58:51  LINK : fatal error LNK1104: cannot open file
'opencv_core-NOTFOUND.obj'

This is typically a problem to handle debug version of shared library.
non debug is found but not debug version.

It due to a mix of build package type. MSVC require a complete package
build with the same type. I can reproduce this kind of dysfunction in
my office when i build my working codes...

Best

Gilles

Le mer. 12 févr. 2020 à 15:04, Gilles Caulier
<caulier.gil...@gmail.com> a écrit :

I pushed this  commit :

https://commits.kde.org/craft-blueprints-kde/51f8cd97d1346364303bd79eaede7ff9b16d7e0e

We will see if build is better tomorrow morning

Gilles

Le mer. 12 févr. 2020 à 07:47, Gilles Caulier
<caulier.gil...@gmail.com> a écrit :

Le mer. 12 févr. 2020 à 04:11, Ben Cooksley <bcooks...@kde.org> a écrit :

On Sun, Feb 9, 2020 at 10:22 PM Gilles Caulier <caulier.gil...@gmail.com> wrote:

Hi Ben,

Hi Gilles,

By experience as we use openCV in digiKAm since more than 10 years
now, Using the LTS version is always a good deal to prevent problem.

openCV team always touch a lot of code (too much) when a new devel
branch is started.

For all bundles that we manage, OpenCV 3.4.x is used for production.
It well maintained and changes are smaller in time.

https://invent.kde.org/kde/digikam/blob/master/project/bundles/3rdparty/ext_opencv/CMakeLists.txt#L99

OpenCV dependency is very tedious. Updating this library is always a
source of dysfunctions. Always. So upgrading must be checked safety in
all conditions.

My 10cts€ tips (:=)))...

Q: It's possible to customize OpenCV dependency only for digiKam or
it's too much complicated ? I suppose that all these king of shared
libraries installed on CI are common for all projects...

Please note that CI is very different to the Binary Factory.
The CI system does not update system level dependencies without manual
action being taken by a Sysadmin. These builds are currently operating
fine.

The Binary Factory is driven by the Craft Blueprints, and will update
as those are updated.
I can confirm however that a given package (such as OpenCV) can only
be in the Craft Blueprints once, and the default version is specified
in that same Blueprint.

I have checked and it appears that frei0r

frei0r python script as the opencv dependency comments actually...

and Digikam are the only
ones that make use of OpenCV, so the version in use can likely be
tweaked to keep them both happy.
As Hannah updated it you will need to check with her why it was
updated (there may very well be good reason to do so)

The current OpenCV version in Craft is 4.2.0, prior to the update it
was 4.1.2 (which built fine) - I suspect the failure may have more to
do with the changes to make it use a system wide Protobuf than the
updates in OpenCV itself however, as the missing symbols belong to
Protobuf.

So, we can setup Opencv to compile internal protobuf instead. There is
a cmake configuration option for that:

-DBUILD_PROTOBUF=OFF

...to set here :

https://github.com/KDE/craft-blueprints-kde/blob/master/libs/opencv/opencv.py#L18

Gilles

Reply via email to