Hi Steve, You will still be able to see all uploaded artifacts in the Treeherder job detail panel. The difference is that we will be retrieving them from taskcluster directly rather than storing them in our database and retrieving them that way.
Regarding your use of log lines, if you can create a structured artifact (JSON, as you mentioned) that would probably be the way to go. If it's stats, that should be reported to Perfherder. If that doesn't work for your particular scenario, then we can loop in other people (maybe :jmaher) who have more knowledge of tests/harnesses/etc to figure out how to surface the info you need. In general, we don't really have an intended replacement per say for TinderboxPrint. It's more a matter of figuring out who is using it, whether we already have that data stored elsewhere in Treeherder and if not, how we can surface it using existing avenues. Let me know if you need a longer timeline for implementing this. I can extend the deadline by another month or two (*just for TinderboxPrint deprecation*) so everyone who needs to can get alternatives implemented can. Sarah On Fri, Apr 10, 2020 at 11:03 AM Steve Fink <sf...@mozilla.com> wrote: > On 4/10/20 10:06 AM, Andrew McCreight wrote: > > On Thu, Apr 9, 2020 at 6:51 PM Sarah Clements <scleme...@mozilla.com> > wrote: > > > >> Hello, > >> > >> As part of maintenance work and performance improvements to Treeherder, > >> we're making changes to some of the data that's stored in our database. > On *April > >> 30th*, we will no longer be retrieving uploaded artifacts for jobs and > >> storing them in the JobDetail table. > >> > >> Uploaded artifacts can be retrieved via a taskcluster API, with the > >> appropriate rootUrl, taskId and runId. TaskId and runId (synonymous with > >> retryId in Treeherder) can be surfaced from any of our /jobs/ API's. See > >> this > >> < > https://github.com/mnoorenberghe/mozscreenshots/issues/43#issue-592994567> > >> for implementation details. > >> > > As somebody who has dumped files to MOZ_UPLOAD_DIR and used the job > detail > > tab to download those files (just this week, in fact!), but has only a > > vague idea of what the "taskcluster API" even is, let alone a "rootURL" > or > > the rest of it, is there any introductory documentation about how I might > > retrieve files written to MOZ_UPLOAD_DIR? Am I going to have to write > some > > kind of Python script to retrieve my files in the future? That GitHub > issue > > seems to require a lot of background information that I do not have. > > I'm in a similar boat. I very frequently set additional options on my > try pushes in order to generate nondefault files in MOZ_UPLOAD_DIR. It > is unclear to me from this description whether I will still be able to > see those files in the treeherder UI after this change. > > > > >> We're also planning to deprecate the parsing and storage of > TinderboxPrint > >> log lines in the JobDetail table. They'll no longer be shown in > >> Treeherder's job detail panel but you can still read them in the raw > logs > >> via the log-viewer. > >> > >> If you have any questions or concerns (or need a longer timeline to make > >> changes that rely on this data), don't hesitate to reach out. > > Similarly, I also regularly depend on being able to emit log lines from > my jobs that will be easily visible in treeherder (*without* reading the > entire raw log), both for my own use and others'. Is there a replacement > for this functionality? I am not at all sad to see the specific > implementation go ("TinderboxPrint" plus an opaque set of rules for what > can go after it), and would be happy to convert my jobs over to a > different system. If that involves something like writing out structured > json to a separate file, that's still fine with me. (I'm only speaking > for myself; some people may depend on being able to simply drop in > `fprintf(stderr, "TinderboxPrint: ...")` into their code and have it > show up.) > > Needing to load the raw log files isn't great because (1) it takes a > noticeable amount of time for large files (I'm sure this is much worse > for some locations), and (2) when you're doing things like this, you may > very well need to be loading several of them. > > If there *is* an intended replacement, I could suggest other things that > I'd find useful (or would like to preserve with the current setup). For > example, I have a job that checks for multiple types of errors, and > wants to output a link to documentation specific to the class of error > that was detected. Not critical functionality, but it definitely makes > for a better workflow. I am currently using TinderboxPrint for this, in > a way that once worked but has now regressed. Example: see the > 'documentation' link in Job Details at > > https://treeherder.mozilla.org/#/jobs?repo=try&author=sfink%40mozilla.com&fromchange=39a2e0ffb371099b5ffd42cb7932367a183d6128&selectedJob=295880330 > > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
