Just tried the externalsrc feature. Works perfectly. Exactly what I was
looking for. Thanks so much, Rudolf!
--Russ
On Thu, Jul 25, 2019 at 4:59 PM Rudolf J Streif <rudolf.str...@ibeeto.com>
wrote:
> Inlining below.
> On 7/25/19 8:14 AM, Russell Peterson wrote:
>
> I think I have a somewhat better understanding of what is going on.
>
> First off, I was confused by the fact that the original error message I
> saw from do_unpack referenced the file (URL) at
> DL_DIR/git2/original_github_url. What I didn't understand at the time was
> that while that file existed,* it was actually a link* to
> DL_DIR/git2/local_url. So, do_unpack wasn't looking at the wrong bare
> repository as I thought. It was unhappy because it didn't see the git
> commit in the local repo that it was looking for. I use AUTOREV for
> SRCREV. Apparently, even with PREMIRROR set and matched, bitbake will
> still fetch the git HEAD commit hash from the original URL in the recipe.
> The local git repo doesn't contain that commit. When I sync the local git
> repo to the github repo things work as expected.
>
> Correct. If the source repo or the correct commit cannot be found from the
> PREMIRROR, bitbake will use SRC_URI from the recipe. You can block that
> behavior by setting BB_NO_NETWORK = "1" in which case bitbake will not be
> able to connect to any remote server. However, that does not help if your
> PREMIRROR is a network server. In this case use BB_ALLOWED_HOSTS to allow
> access to only the servers you want.
>
>
> Here is my problem. What I am ultimately trying to do here is have 2 git
> repos. 1 public, 1 private. For internal development we use the private
> git repo (a local gerrit server). Just before we ship we update the public
> github repo. The recipe always points to the github (public) repo so we
> don't need to mess with that before a release. This way we can do nightly
> builds (I'm just talking nightly builds done by Jenkins... not developers
> using eSDK and devtool) and test. Customers using yocto will be given a
> recipe that points to github and, importantly, those customers that do NOT
> use yocto simply fetch the project from github and manually build it in
> their own environment with their own tools.
>
> That is a common problem. I do this all the time. To do this add to your
> conf/local.conf file:
>
> INHERIT += "externalsrc"
> EXTERNALSRC_pn-myrecipe = "/path/to/source/tree"
>
> Alternatively, you could use a bbappend file for you recipe in a
> development layer, myrecipe.bbappend:
>
> inherit externalsrc
> EXTERNALSRC = "/path/to/source/tree"
>
> Now, that is exactly what devtool does for you but you can of course do it
> manually.
>
> :rjs
>
>
>
> I can't be the only one doing this. Is there a best practices here
> (private vs. public repo)?
>
> --Russ
>
>
> On Wed, Jul 24, 2019 at 3:55 PM Rudolf J Streif <rudolf.str...@ibeeto.com>
> wrote:
>
>> Russell,
>>
>> That is exactly what devtool and the externalsrc class do. PREMIRROR is
>> the wrong approach for that.
>>
>> :rjs
>> On 7/24/19 12:53 PM, Russell Peterson wrote:
>>
>> Hi, Rudolf.
>>
>> I apologize for not being clear. The idea here is that my recipe points
>> to github while, for my local development environment, I set a premirror to
>> match a specific github repository and translate it to a local directory.
>> That works. The fetch matches the PREMIRROR and places a copy of the LOCAL
>> git repo in my DL_DIR. The problem is, the do_unpack task is looking for
>> the git repo in the DL_DIR (git2/etc...)... but it's looking for the git
>> repo from github, not my local repo that the fetch task just put in DL_DIR.
>>
>> There are numerous reasons I'm doing this including the fact that I
>> cannot put a reference to my local repo in the bb file since I ship that
>> recipe to my customers and a local pathname is meaningless to them. I
>> prefer to simply modify my local.conf with a PREMIRROR value that has a
>> specific regular expression that matches this particular github repo (and
>> hence does NOT effect all recipes). This is why I wanted to use the
>> PREMIRROR function. Unfortunately, it does not appear to work or at least
>> not work as I expected.
>>
>> --Russ
>>
>>
>>
>>
>> On Wed, Jul 24, 2019 at 2:57 PM Rudolf J Streif <rudolf.str...@ibeeto.com>
>> wrote:
>>
>>> Hi Russell,
>>>
>>> devtool and eSDK are different things. The purpose of PREMIRRORS is to
>>> set a mirror for all recipes. It's a way for organizations to control where
>>> their YP builds download sources from. It's not intended to be used for a
>>> single recipe. There is no need for that. You simply set SRC_URI in your
>>> recipe to your local git repo. That is what devtool does after downloading
>>> the sources from a remote repo. If you already have the remote repo cloned
>>> locally you can just point devtool to it.
>>>
>>> You can do it manually by creating your recipe and setting SRC_URI like
>>> this:
>>>
>>> SRC_URI = "git:///local/path/${PN};protocol=file"
>>>
>>> SRCREV = "${AUTOREV}"
>>>
>>> S = "${WORKDIR}/git"
>>>
>>> PREMIRRORS is only relevant for do_fetch not for do_unpack.
>>>
>>> :rjs
>>>
>>>
>>> On 7/24/19 11:28 AM, Russell Peterson wrote:
>>>
>>> Hi, Rudolf.
>>>
>>> Thanks for the reply. Yes, I am aware of the eSDK functionality,
>>> however, I have some unique requirements that I am trying to work around.
>>> Regardless... what I am doing should work, no? I simply want to use a
>>> local git repo (the directory itself hence protocol=file) instead of what
>>> the recipe specifies. Looks like the fetch is working but the do_unpack
>>> task is ignoring PREMIRRORS (or at least the localpath variable seems
>>> wrong).
>>>
>>> --Russ
>>>
>>>
>>> On Wed, Jul 24, 2019 at 12:19 PM Rudolf J Streif <
>>> rudolf.str...@ibeeto.com> wrote:
>>>
>>>> Russell,
>>>>
>>>> You don't need PREMIRROR for this functionality. It's not exactly
>>>> intended for that use.
>>>>
>>>> The simplest way to achieve what you are looking for is to use devtool.
>>>> If I understand you correctly you are downloading sources from a remote
>>>> repo on GitHub but want to have them locally to make modifications? If so
>>>> use:
>>>>
>>>> devtool add myrecipe localsrc fetchuri
>>>>
>>>> from your build environment (you have to source oe-init-build-env
>>>> first). devtool then fetches the source from fetchuri into a directory
>>>> localsrc as a git repo and automatically creates the recipe for it.
>>>>
>>>> :rjs
>>>>
>>>>
>>>> On 7/23/19 1:49 PM, Russell Peterson wrote:
>>>>
>>>> Hello,
>>>>
>>>> I am looking to have bitbake pick up files for a particular recipe from
>>>> a local git repository using the PREMIRROR functionality.
>>>>
>>>> Basically, the recipe (bb file) points to github but in my local build
>>>> I add PREMIRROR_prepend = "git://.*/.*
>>>> git:///local/path/BASENAME;protocol=file\n"
>>>>
>>>> I will probably make the git regular expression more exact for my
>>>> specific github repo but this works for now.
>>>>
>>>> This all works (as I deleted the github download from the local
>>>> download directory) because I can see in the do_fetch log and the correct
>>>> (local) repo was found and placed in the DL_DIR.
>>>>
>>>> Problem is, do_unpack fails because it appears to be looking for the
>>>> original (github) SRC_URI. Then it complains about "no up to date source
>>>> found: clone or directory not available or not up to date (shallow clone
>>>> not enabled)"
>>>>
>>>> Any help on what I am missing would be appreciated.
>>>>
>>>> Regards,
>>>>
>>>> Russell
>>>>
>>>>
>>>>
>>>> --
>>>> -----
>>>> Rudolf J Streif
>>>> CEO/CTO ibeeto
>>>> +1.855.442.3396 x700
>>>>
>>>> --
>>> -----
>>> Rudolf J Streif
>>> CEO/CTO ibeeto
>>> +1.855.442.3396 x700
>>>
>>> --
>> -----
>> Rudolf J Streif
>> CEO/CTO ibeeto
>> +1.855.442.3396 x700
>>
>> --
> -----
> Rudolf J Streif
> CEO/CTO ibeeto
> +1.855.442.3396 x700
>
>
--
_______________________________________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
