On Mon, 16. Aug 20:42, Michael Niedermayer wrote: > On Sun, Aug 15, 2021 at 01:53:23PM -0400, Andriy Gelman wrote: > > On Sun, 15. Aug 17:15, Michael Niedermayer wrote: > > > On Sun, Aug 15, 2021 at 08:18:17AM -0400, Andriy Gelman wrote: > > > > On Sun, 15. Aug 11:17, Nicolas George wrote: > > > > > Is it possible to add hooks in Patchwork to warn people automatically > > > > > when their commit message does not match standards? > > > > > > > > > > If it is possible, I volunteer to write it. > > > > > > > > > > > > > Yes, nice idea. > > > > > > > > If the commit message is invalid I could add a warning similar to: > > > > https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210809102919.387-1-an...@khirnov.net/ > > > > and trigger an automated email to the author. > > > > > > > > Feel free to send me the parsing part (shell or python is ok) and I'll > > > > add it > > > > in. > > > > I'll also aim to put all this code in a repo somewhere so that others > > > > can add to it. > > > > > > If you want i can add a git repo to https://git.ffmpeg.org/gitweb > > > i just need to know name, who should have write access > > > (@ffmpeg_developers is > > > possible, if its something else then you will have to maintain the list of > > > people who have access), where commitlog mails should go. But you can > > > also put it on github > > > that said if anyone else wants basic git repos for any ffmpeg parts which > > > arent in teh main git, we can add it to git.ffmpeg.org too > > > > > > thx > > > > Thanks, a repo on gitweb with write access for @ffmpeg_developers sounds > > good. > > ok, we need a name for it, and should commit mails be sent to ffmpeg-devel ?
Michael set up the repo here: https://git.ffmpeg.org/gitweb/patchwork_job_runner.git I've pushed the python code that can be used to setup a custom CI job runner for patchwork. Attached is a README patch if anyone wants to comment. -- Andriy
>From 9481394dc834f89c4941c1b0dd57b27d9d9a7f5f Mon Sep 17 00:00:00 2001 From: Andriy Gelman <andriy.gel...@gmail.com> Date: Mon, 30 Aug 2021 13:21:21 -0400 Subject: [PATCH] Add a README --- README | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 README diff --git a/README b/README new file mode 100644 index 0000000..597bceb --- /dev/null +++ b/README @@ -0,0 +1,57 @@ +This repo has python helper to setup custom CI jobs (i.e. different +OS, architectures, etc) for FFmpeg or any other project that uses patchwork (see +https://github.com/getpatchwork/patchwork for more details). + +The script periodically queries the patchwork site for new patches. CI jobs are +run on new patches and the results are posted to the patchwork site. +In the context of FFmpeg, this includes running ./configure, make build, and make fate. + +-- CI jobs -- +The CI jobs are setup in class Job. This class has functions setup(), build(), +and unit_test(). In our implementation the functions start docker +containers which run ./configure, make build, and make fate. The containers are +launched using subprocess.run() which captures stdout, stderr, and success of +the process. The return struct is forwarded to the calling functions which +determines how to process the output (i.e. posts result to patchwork, notify +user by email). Custom jobs can therefore be created modifying the job class. Use of containers is +recomended for isolating the process. + +--- Caching results --- +The code currently uses a mysql database to track information and cache job +results. The config of the database are set by the environment variables: +PATCHWORK_DB_{HOST,USER,PASSWORD}. + +--- Automated emails --- +If a CI job fails, an automated email is triggered to the patch author with a +link to the patchwork site where the warning or error is shown. To prevent +spamming the author, only one email is triggered per patch series. An email is +also only sent if the parent commit builds successfully. Thus if current +origin/master doesn't build, an email will not be sent (unless a commit fixes +the issue and breaks it another commit of the series). The environment variables +for connecting to an SMTP server are +PATCHWORK_SMTP_{HOST,PORT}, PATCHWORK_{USER_EMAIL,PASSWORD_EMAIL}. +The environment variable PATCHWORK_CC_EMAIL is used to add a cc email address. + +--- Patchwork authentication --- +An account (on https://patchwork.ffmpeg.org) and proper permission are needed to post +CI results back to the patchwork site. Email your patchwork site maintainer in +(FFmpeg/MAINTERNERS) with your username if you want the permissions added to your account. +After the permissions are set up, an API token can be obtained after logging in +to the patchwork site. The environment variable PATCHWORK_TOKEN stores the api +token. The variable PATCHWORK_HOST needs to be set to patchwork.ffmpeg.org or +another patchwork site. + +-- Other environemnt variables -- +The following variables are used by the docker container for the CI job: +PATCHWORK_{UID,GID} set the uid/gid of the docker container. +PATCHWORK_PROJECT_ROOT_PATH is the path to the main FFmpeg directory. + +-- Steps to setup a CI job runner -- +1. Create an account on patchwork.ffmpeg.org and email your patchwork maintainer to setup +permissions. +2. Modify class Job in job.py for your CI job setup. +3. Export the environment variables described above. +4. Start a mysql server instance, i.e. using docker: +$ docker run --name "ffmpeg_mysql" -e MYSQL_ROOT_PASSWORD=${PATCHWORK_DB_PASSWORD} --rm -v sqldb:/var/lib/mysql -p 3306:3306 mysql:5.7 +5. Start the job runner with: +$ python patchwork_runner.py -- 2.32.0
_______________________________________________ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
