On 9/7/21 12:22 PM, Ben Cooksley wrote:
On Tue, Sep 7, 2021 at 8:48 PM Vlad Zahorodnii <vlad.zahorod...@kde.org
<mailto:vlad.zahorod...@kde.org>> wrote:
On 9/5/21 3:18 PM, David Faure wrote:
> On dimanche 5 septembre 2021 12:26:50 CEST Ben Cooksley wrote:
>> On Sun, Sep 5, 2021 at 10:22 PM David Faure <fa...@kde.org
<mailto:fa...@kde.org>> wrote:
>>> For frameworks, I think we should be able to write a one-time
script that
>>> generates .kde-ci.yml files using the dependencies listed in
kde-build-
>>> metadata (and the platforms listed in metainfo.yaml) ?
>>
>> Yes, that should work nicely (although that information now lives in
>> sysadmin/repo-metadata, dependencies folder)
>
> All done for Frameworks.
>
> The script that collects platforms from metainfo.yaml won't be
useful for
> other KDE modules, but the script that collects deps from
dependency-data-kf5-
> qt5 is attached, in case it's useful to anyone else.
Is there a file that maps legacy project paths to gitlab project paths?
dependency-data-kf5-qt5 lists projects with their legacy paths.
The individual project YAML files in sysadmin/repo-metadata contain this
information.
The legacy project path can be found under 'projectpath' while the
Gitlab paths are under 'repopath'
Thanks! I've attached a quick and dirty python script that parses
project dependencies from repo-metadata and prints corresponding gitlab
project paths.
Example usage
python project-deps.py --repo-metadata
/data/projects/src/repo-metadata/ kde/workspace/kwin
Output
Dependencies:
'requires':
'frameworks/extra-cmake-modules': '@stable'
'plasma/kdecoration': '@stable'
'plasma/kscreenlocker': '@stable'
'plasma/kwayland-integration': '@stable'
'plasma/breeze': '@stable'
'plasma/kwayland-server': '@stable'
Cheers,
Vlad
Cheers,
Ben
>> Once we've transitioned across to the .kde-ci.yml files the existing
>> dependency metadata and branch group files will no longer be
required.
>> (We'll need to work out a way of exporting this information for easy
>> consumption by kdesrc-build and other clients before it can be
removed
>> though)
>
> OK. Duplication is bad so I agree :)
>
#!/usr/bin/env python3
import argparse
import glob
import os
import yaml
def build_legacy_to_gitlab_table(repo_metadata):
table = dict()
repo_root = os.path.join(repo_metadata, "projects-invent/**/**/metadata.yaml")
for filename in glob.glob(repo_root):
with open(filename, "r") as f:
document = yaml.safe_load(f)
legacy_path = document["projectpath"]
gitlab_path = document["repopath"]
if not legacy_path:
print(filename + " is invalid")
elif not gitlab_path:
print(filename + " is invalid")
else:
table[legacy_path] = gitlab_path
return table
def parse_dependencies(repo_metadata, legacy_project_path):
deps_file = os.path.join(repo_metadata, "dependencies", "dependency-data-kf5-qt5")
dependencies = list()
needle = legacy_project_path + ":"
with open(deps_file, "r") as f:
for line in f:
if line.startswith("*:"):
dep = line[len("*:"):].strip()
elif line.startswith(needle):
dep = line[len(needle):].strip()
else:
continue
if dep.startswith("-"):
dep = dep[1:]
dependencies.remove(dep)
else:
dependencies.append(dep)
return dependencies
def print_dependencies(mapping_table, dependencies):
gitlab_deps = list()
for dependency in dependencies:
if dependency in mapping_table:
gitlab_deps.append(mapping_table[dependency])
elif dependency != "third-party/Qt5":
print(dependency + " has no matching gitlab repo")
if gitlab_deps:
print("Dependencies:")
print(" 'requires':")
for dep in gitlab_deps:
print(f" '{dep}': '@stable'")
parser = argparse.ArgumentParser(description='kde ci dependency builder')
parser.add_argument('--repo-metadata',
help='the file path to sysadmin/repo-metadata repo')
parser.add_argument('projectpath', help='projectpath')
args = parser.parse_args()
repo_metadata = args.repo_metadata
projectpath = args.projectpath
mapping_table = build_legacy_to_gitlab_table(repo_metadata)
legacy_dependencies = parse_dependencies(repo_metadata, projectpath)
print_dependencies(mapping_table, legacy_dependencies)
