On 4/10/20 10:06 AM, Andrew McCreight wrote:
On Thu, Apr 9, 2020 at 6:51 PM Sarah Clements <scleme...@mozilla.com> wrote:
Hello,
As part of maintenance work and performance improvements to Treeherder,
we're making changes to some of the data that's stored in our database. On
*April
30th*, we will no longer be retrieving uploaded artifacts for jobs and
storing them in the JobDetail table.
Uploaded artifacts can be retrieved via a taskcluster API, with the
appropriate rootUrl, taskId and runId. TaskId and runId (synonymous with
retryId in Treeherder) can be surfaced from any of our /jobs/ API's. See
this
<https://github.com/mnoorenberghe/mozscreenshots/issues/43#issue-592994567>
for implementation details.
As somebody who has dumped files to MOZ_UPLOAD_DIR and used the job detail
tab to download those files (just this week, in fact!), but has only a
vague idea of what the "taskcluster API" even is, let alone a "rootURL" or
the rest of it, is there any introductory documentation about how I might
retrieve files written to MOZ_UPLOAD_DIR? Am I going to have to write some
kind of Python script to retrieve my files in the future? That GitHub issue
seems to require a lot of background information that I do not have.
I'm in a similar boat. I very frequently set additional options on my
try pushes in order to generate nondefault files in MOZ_UPLOAD_DIR. It
is unclear to me from this description whether I will still be able to
see those files in the treeherder UI after this change.
We're also planning to deprecate the parsing and storage of TinderboxPrint
log lines in the JobDetail table. They'll no longer be shown in
Treeherder's job detail panel but you can still read them in the raw logs
via the log-viewer.
If you have any questions or concerns (or need a longer timeline to make
changes that rely on this data), don't hesitate to reach out.
Similarly, I also regularly depend on being able to emit log lines from
my jobs that will be easily visible in treeherder (*without* reading the
entire raw log), both for my own use and others'. Is there a replacement
for this functionality? I am not at all sad to see the specific
implementation go ("TinderboxPrint" plus an opaque set of rules for what
can go after it), and would be happy to convert my jobs over to a
different system. If that involves something like writing out structured
json to a separate file, that's still fine with me. (I'm only speaking
for myself; some people may depend on being able to simply drop in
`fprintf(stderr, "TinderboxPrint: ...")` into their code and have it
show up.)
Needing to load the raw log files isn't great because (1) it takes a
noticeable amount of time for large files (I'm sure this is much worse
for some locations), and (2) when you're doing things like this, you may
very well need to be loading several of them.
If there *is* an intended replacement, I could suggest other things that
I'd find useful (or would like to preserve with the current setup). For
example, I have a job that checks for multiple types of errors, and
wants to output a link to documentation specific to the class of error
that was detected. Not critical functionality, but it definitely makes
for a better workflow. I am currently using TinderboxPrint for this, in
a way that once worked but has now regressed. Example: see the
'documentation' link in Job Details at
https://treeherder.mozilla.org/#/jobs?repo=try&author=sfink%40mozilla.com&fromchange=39a2e0ffb371099b5ffd42cb7932367a183d6128&selectedJob=295880330
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
