On Thu, 2011-09-22 at 13:27 +0200, Koen Kooi wrote: > Op 22 sep 2011, om 12:20 heeft Richard Purdie het volgende geschreven: > > > On Thu, 2011-09-22 at 10:28 +0200, Koen Kooi wrote: > >> Op 22 sep. 2011, om 09:03 heeft Richard Purdie het volgende > > This just means there aren't appropriate git:// -> git:// mappings in > > people's mirrors files (including Yocto). If git:// -> git:// mappings > > don't work, we should fix that but they should. > > You mean something like git.kernel.org/scm/torvalds/linux-2.6.git -> > github.com/torvalds/linux.git ?
Yes. Thinking about this, the current fetcher might not be optimised in making this as efficient as possible but it should work and if it doesn't, we should make it. > > Now someone has explained the issue, yes, I can see why it is causing > > problems. I do think some of them are of people's own choosing but I'm > > trying to be flexible and not make judgement on that. > > > > So we can do something along the lines of what you after but there are > > the following considerations: > > > > a) The versioned tarballs do need to be different to what we had > > before > > in that the tarball will be of a .git directory containing the git > > metadata, not just a checkout. Why? This is so we can incrementally > > update a git repo if the user changes the source revision as one > > example. We're dealing with the git fetcher and limiting ourselves > > to a > > flat one dimension doesn't make sense. > > Agreed on that. The bitbake patch I sent does at least do that :) Right :) I just want to be clear we're doing something different to what was there before (and why). > > b) We need to agree what order the tarballs on the server should be > > searched for. The "obvious" order would be: > > > > 1. Specific Revision X tarball > > 2. generic mirror tarball > > 3. git clone > > > > The problems come about when we have an existing git checkout which > > doesn't contain the revision we want. > > There currently is a bug in the fetcher that makes it hard to use now > that kernel.org is down. The situation: > > I have a recipe that pulls from git (e.g. cpufrequtils.git from > kernel.org) with SRCREV fixed to e.g. deadbeef. DL_DIR/git/ > git.kernel.org.utils.cpufrequtils has revision deadbeef. When I do a - > c fetch -f it will try to do an update and fail since kernel.org is > down. I would have expected it to check the existing bare clone for > deadbeef first and not try updating if it is present. Agreed, it shouldn't be hitting the network unless the revision is floating. That is bad :/. > Anyway, I agree on the search order above. > > > Currently we can just update and > > assume that will bring in the revision. If we now support people doing > > rebases and other things, we now at least in theory have to: > > > > 1. See if the revision exists locally > > 2. Try pulling > > 3. If that doesn't work look for a versioned tarball > > 4. Blow away all fetcher git status from DL_DIR > > 5. Extract the tarball > > Since git supports local fetches the above can be: > > 1) see if rev exists > 2) try pulling > 3) try versioned tarball > 4) extract tarball into tmpdir, fetch objects into DL_DIR/git2 repo > > That would stop things from being blown away. It does allow the > versioned snapshots to differ, but they will always have the rev they > were versioned with. This is a much better approach. Its quite a big difference from anything we've done before but the approach should fit in better. I need to think on it more... > > My big concern with this is rather than having some general > > archictecture and standard way of doing this, the fetcher once again > > becomes a set of special case if A do B but if C do D first then do E > > but don't forget to do A if Z happens type code. > > I know virtually nothing of the fetcher code, but I think treating > every tarball as a local mirror that gets fetched into DL_DIR/git2/ > repodir would avoid a lot of special casing. Yes, its nicer. We keep falling into the trap of thinking about the non-distributed SCM cases but we have more capability with git and it makes sense to use it. > > So what I'm saying is I don't want the fetcher to become the > > unmaintainable mess the that the code paths in fetch1 were. I don't > > honestly know how to add a code path like the above and avoid this :( > > > > The patch on the bitbake list doesn't consider any of this > > complexity or > > the corner cases > > Sadly it's the best thing I can accomplish with my python skills :( It > does solve my #1 need, which is an easy way (ls | grep) to check for > GPL compliance. Ok. Can you at least file a bug in the yocto bugzilla with a summary of this discussion please? :) That should ensure the issue gets more attention since with the best intentions, I can't remember everything. Cheers, Richard _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
