Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Daniel Golle 
On Tue, Jan 01, 2019 at 08:46:25PM +0100, Petr Štetiar wrote:
> Daniel Golle  [2019-01-01 17:56:25]:
> 
> Hi,
> 
> > On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
> > > Daniel Golle  [2018-12-29 06:51:32]:
> > > 
> > > >  config KERNEL_AIO
> > > >  config KERNEL_FHANDLE
> > > >  config KERNEL_FANOTIFY
> > > > +   default y if !SMALL_FLASH
> > > 
> > > What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.
> > 
> > This is not necessarily only used on NAS devices. systemd requires
> > FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
> > needs AIO. Both could well run on a modern router or SBC having USB or
> > an SDCARD slot.
> 
> to me it's still just NAS and container use cases. I'm afraid, that adding
> more bloat to kernels for devices with USB and MMC/SD card slots would be
> rejected also.

Well, most modern routers come with USB and/or MMC/SD and Samba
installed in their stock-rom. Apart from that, a lot of useful things
can be done with network namespaces -- we are just not implementing
them because the kernel doesn't have that feature enabled.
(think: have some service connect over VPN, others directly over WAN)

> 
> > > >  config KERNEL_CGROUPS
> > > >  config KERNEL_CPUSETS
> > > >  config KERNEL_CGROUP_CPUACCT
> > > >  config KERNEL_RESOURCE_COUNTERS
> > > >  config KERNEL_MEMCG
> > > >  config KERNEL_MEMCG_KMEM
> > > >  menuconfig KERNEL_CGROUP_SCHED
> > > >   config KERNEL_FAIR_GROUP_SCHED
> > > >   config KERNEL_RT_GROUP_SCHED
> > > >  config KERNEL_NAMESPACES
> > > >  config KERNEL_LXC_MISC
> > > >  config KERNEL_SECCOMP_FILTER
> > > >  config KERNEL_SECCOMP
> > > > -   default n
> > > > +   default y if !SMALL_FLASH
> > > 
> > > What about `FEATURES += containers` ?
> > 
> > From what I understood FEATURES is supposed to reflect hardware
> > capabilities
> 
> Well, almost. I've found `squashfs` and `ext4` in there also.

True, and I don't think those two symbols make sense when talking
about FEATURES -- all devices do support squashfs without exception,
ext4 can only work on block-type storage, ie. devices booting from
(e)MMC or a SATA drive. So imho we could drop the squashfs symbol
and replace ext4 with a 'block_root' feature which imho would be
more meaningful.

> 
> >  -- all the above are generic software features useful on any device having
> >  the capacity (ie. flash and RAM) to make use of them.
> 
> But as other Daniel already suggested, !SMALL_FLASH isn't proper group
> selection either. Speaking about the capacity, did you measured how much those
> features add to the kernel images?
> 
> > > Daniel Engberg  [2018-12-30 10:21:46]:
> > > 
> > > > however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config 
> > > > KERNEL_LXC_MISC,
> > > > KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and 
> > > > more or
> > > > less only used on x86*?
> > > 
> > > There are other quite powerful platforms like mvebu, imx6, ipq etc. where 
> > > you
> > > could use this as well.
> > 
> > I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),
> 
> Ok, looking at IMAGE_SIZE values for mt7621 yields following results:
> 
>  IMAGE_SIZE := 6016k TP-LINK RE350 v1
>  IMAGE_SIZE := 7798784   I-O DATA WN-GX300GR
>  IMAGE_SIZE := $(ralink_default_fw_size_4M)  MT7621 EVB
>  IMAGE_SIZE := $(ralink_default_fw_size_8M)  AP-MT7621A-V60 EVB
> 
> So it wouldn't be wise to add more bloat into kernels for those devices.

I don't think adding ~ 100kb to devices with 8MB of flash or more is a
problem. Regarding that evaluation board with only 4MB of NOR flash:
this is not a real-world device but rather an evalution platform, so
I wouldn't care much about users having to build their own stripped-
down version for that, because it's probably only a few hundred of
those EVBs ever made and most of them are by now collecting dust on
a shelf and will never be used for anything again.

> 
> > running Debian inside LXC containers (and it's annoying that I can't
> > use regular OpenWrt releases or buildbot-generated snapshots on the).
> 
> I would like to do the same, but on the other hand I also understand, why your
> patch as it is wouldn't be accepted. Our containers/NAS use cases are still
> very rare, thus it makes no sense to enable those features by default to every
> other !SMALL_FLASH target. As you can see on mt7621, it's not marked as
> SMALL_FLASH, yet there are devices which might be considered SMALL_FLASH.

One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
a more sane flash partition layout to gain another megabyte or so.

> 
> Now I realize, that it couldn't be handled with FEATURES anyway, as this is
> always too broad group selection and ideally you want to enable those features
> for specific devices only, meaning different kernels, images etc.

Why specific devices? Wouldn't all devices with the resources (which
boils down to !SMALL

Re: [OpenWrt-Devel] GCC 7.4.0 ld LTO plugin issues with SDK

2019-01-07 Thread Daniel F. Dickinson 
On 2019-01-03 2:21 a.m., Daniel F. Dickinson wrote:
> On 2019-01-02 8:58 p.m., Daniel F. Dickinson wrote:
>> On 2019-01-02 3:06 p.m., Daniel F. Dickinson wrote:
>>> Hi,
>>>
>>> With the SDK I'm getting errors with LTO -- at least for json-c LD is
>>> trying to load the *target's* LTO plugin instead of host.
>>>
>>> 7.4.0_musl_eabi/bin/../libexec/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/liblto_plugin.so:
>>> error loading plugin:
>>> /home/daniel/Build/openwrt-sdk/cshoredwrt-sdk-19.01-1-brcm2708-bcm2708_gcc-7.4.0_musl_eabi.Linux-x86_64/staging_dir/toolchain-arm_arm1176jzf-s+vfp_gcc-7.4.0_musl_eabi/bin/../libexec/gcc/arm-openwrt-linux-muslgnueabi/7.4.0/liblto_plugin.so:
>>> invalid ELF header
>> So it turns out that for the SDK that it turns libraries into
>> library.bin and the original *.so* becomes a bash script.  Apparently
>> GCC doesn't like that, at least for it's own plugins.
> Ok, building the SDK on Debian 9 solves the issue.  The SDK built on
> Ubuntu 18.10 has the issue even if used on Debian 9, so it's the
> building of the SDK where the issue, whatever it is, occurs.  As I have
> a handy Debian VM I'm not going to worry too much about it at the
> moment, unless there is a patch someone needs me to test.
So this may be a path too long issue ... I've run into other problems
with the sdk using default path, and not in the base of my home
directory (i.e. long path) that 'go away' when I shorten the path.  It
looks like somewhere in the toolchain there is some limit being
exceeded.  Unfortunately I have not identified where, only noticed it
because of the following build failure with glib2:

|c-7.4.0_musl_eabi.Linux-x86_64/build_dir/hostpkg/glib-2.58.2/gobject'
Making all in gio make[6]: Entering directory
'/home/daniel/Build/openwrt-sdk/cshoredwrt-sdk-19.01-1-brcm2708-bcm2708_gcc-7.4.0_musl_eabi.Linux-x86_64/build_dir/hostpkg/glib-2.58.2/gio'
GEN xdp-dbus.c GEN xdp-dbus.h cp gnextstepsettingsbackend.m
gnextstepsettingsbackend.c cp gosxcontenttype.m gosxcontenttype.c cp
gosxappinfo.m gosxappinfo.c cp gcocoanotificationbackend.m
gcocoanotificationbackend.c GEN gconstructor_as_data.h GEN
gioenumtypes.h /usr/bin/env:
'/home/daniel/Build/openwrt-sdk/cshoredwrt-sdk-19.01-1-brcm2708-bcm2708_gcc-7.4.0_musl_eabi.Linux-x86_64/staging_':
No such file or directory Makefile:5039: recipe for target
'gioenumtypes.h' failed make[6]: *** [gioenumtypes.h] Error 127 make[6]:
Leaving directory
'/home/daniel/Build/openwrt-sdk/cshoredwrt-sdk-19.01-1-brcm2708-bcm2708_gcc-7.4.0_musl_eabi.Linux-x86_64/build_dir/hostpkg/glib-2.58.2/gio'
Makefile:1254: recipe for target 'all-recursive' failed make[5]: ***
[all-recursive] Error 1 make[5]: Leaving directory
'/home/daniel/Build/openwrt-sdk/cshoredwrt-sdk-19.01-1-brcm2708-bcm2708_gcc-7.4.0_musl_eabi.Linux-x86_64/build_dir/hostpkg/glib-2.58.2'
Makefile:881: recipe for target 'all' failed make[4]: *** [all] Error 2|


___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Daniel F. Dickinson 
On 2019-01-07 4:03 a.m., Daniel Golle wrote:
> On Tue, Jan 01, 2019 at 08:46:25PM +0100, Petr Štetiar wrote:
>> Daniel Golle  [2019-01-01 17:56:25]:
>>
>> Hi,
>>
>>> On Sun, Dec 30, 2018 at 11:26:58AM +0100, Petr Štetiar wrote:
 Daniel Golle  [2018-12-29 06:51:32]:

>  config KERNEL_AIO
>  config KERNEL_FHANDLE
>  config KERNEL_FANOTIFY
> + default y if !SMALL_FLASH
 What about `FEATURES += nas` to make it clear and don't abuse SMALL_FLASH.
>>> This is not necessarily only used on NAS devices. systemd requires
>>> FHANDLE and FANOTIFY (eg. when running inside LXC container), lvm2
>>> needs AIO. Both could well run on a modern router or SBC having USB or
>>> an SDCARD slot.
>> to me it's still just NAS and container use cases. I'm afraid, that adding
>> more bloat to kernels for devices with USB and MMC/SD card slots would be
>> rejected also.
> Well, most modern routers come with USB and/or MMC/SD and Samba
> installed in their stock-rom. Apart from that, a lot of useful things
> can be done with network namespaces -- we are just not implementing
> them because the kernel doesn't have that feature enabled.
> (think: have some service connect over VPN, others directly over WAN)
>
I think really small devices are rare now...and that's what SMALL_FLASH
is for (perhaps quantify SMALL would be better though).


>  config KERNEL_CGROUPS
>  config KERNEL_CPUSETS
>  config KERNEL_CGROUP_CPUACCT
>  config KERNEL_RESOURCE_COUNTERS
>  config KERNEL_MEMCG
>  config KERNEL_MEMCG_KMEM
>  menuconfig KERNEL_CGROUP_SCHED
>   config KERNEL_FAIR_GROUP_SCHED
>   config KERNEL_RT_GROUP_SCHED
>  config KERNEL_NAMESPACES
>  config KERNEL_LXC_MISC
>  config KERNEL_SECCOMP_FILTER
>  config KERNEL_SECCOMP
> - default n
> + default y if !SMALL_FLASH
 What about `FEATURES += containers` ?
>>> From what I understood FEATURES is supposed to reflect hardware
>>> capabilities
>> Well, almost. I've found `squashfs` and `ext4` in there also.
> True, and I don't think those two symbols make sense when talking
> about FEATURES -- all devices do support squashfs without exception,
> ext4 can only work on block-type storage, ie. devices booting from
> (e)MMC or a SATA drive. So imho we could drop the squashfs symbol
> and replace ext4 with a 'block_root' feature which imho would be
> more meaningful.
+1
>>>  -- all the above are generic software features useful on any device having
>>>  the capacity (ie. flash and RAM) to make use of them.
>> But as other Daniel already suggested, !SMALL_FLASH isn't proper group
>> selection either. Speaking about the capacity, did you measured how much 
>> those
>> features add to the kernel images?
>>
 Daniel Engberg  [2018-12-30 10:21:46]:

> however KERNEL_CGROUPS, config KERNEL_NAMESPACES, config KERNEL_LXC_MISC,
> KERNEL_SECCOMP_FILTER are very limited use cases to my knowledge and more 
> or
> less only used on x86*?
 There are other quite powerful platforms like mvebu, imx6, ipq etc. where 
 you
 could use this as well.
>>> I use LXC on oxnas/ox820 (ARM11mpcore) and ramips/mt7621 (MIPS1004Kc),
>> Ok, looking at IMAGE_SIZE values for mt7621 yields following results:
>>
>>  IMAGE_SIZE := 6016k TP-LINK RE350 v1
>>  IMAGE_SIZE := 7798784   I-O DATA WN-GX300GR
>>  IMAGE_SIZE := $(ralink_default_fw_size_4M)  MT7621 EVB
>>  IMAGE_SIZE := $(ralink_default_fw_size_8M)  AP-MT7621A-V60 EVB
>>
>> So it wouldn't be wise to add more bloat into kernels for those devices.
> I don't think adding ~ 100kb to devices with 8MB of flash or more is a
> problem. Regarding that evaluation board with only 4MB of NOR flash:
> this is not a real-world device but rather an evalution platform, so
> I wouldn't care much about users having to build their own stripped-
> down version for that, because it's probably only a few hundred of
> those EVBs ever made and most of them are by now collecting dust on
> a shelf and will never be used for anything again.
+1
>>> running Debian inside LXC containers (and it's annoying that I can't
>>> use regular OpenWrt releases or buildbot-generated snapshots on the).
>> I would like to do the same, but on the other hand I also understand, why 
>> your
>> patch as it is wouldn't be accepted. Our containers/NAS use cases are still
>> very rare, thus it makes no sense to enable those features by default to 
>> every
>> other !SMALL_FLASH target. As you can see on mt7621, it's not marked as
>> SMALL_FLASH, yet there are devices which might be considered SMALL_FLASH.
> One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> a more sane flash partition layout to gain another megabyte or so.
>
>> Now I realize, that it couldn't be handled with FEATURES anyway, as this is
>> always too broad group selection and ideally you want to enable those 
>> features
>> for specific devices only, meaning different

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Petr Štetiar 
Daniel Golle  [2019-01-07 10:03:09]:

Hi,

> One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> a more sane flash partition layout to gain another megabyte or so.

I've looked only at mt7621, so this was just example from one subtarget of
ramips target. So I tend to believe, that there's quite more such cases hidden
in the tree. Please correct me if I'm wrong.

> Why specific devices? Wouldn't all devices with the resources (which
> boils down to !SMALL_FLASH) be potentially more useful with those
> kernel features enabled? 

You currently can't use !SMALL_FLASH, because this is target/subtarget
specific feature, not per device feature. I think, that in order to use this
feature, you would need to convert/fix all devices like that TP-Link RE350
from all (sub)targets into tiny subtarget and then you could freely use
!SMALL_FLASH.

> I know first hand that this is a reason for businesses I have been working
> for to switch over to OpenEmbedded instead of OpenWrt (and then hire people
> like me to port all the OpenWrt stuff they need into their OE build...).

This is interesting, so instead of adding few config options and building own
images which is quite trivial task, someone is rather moving to completely
different {build,eco}system? Well, you can do that, so why not. And yes, I'm
aware about meta-openwrt OE layer.

-- ynezz

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Jonas Gorski 
On Mon, 7 Jan 2019 at 11:42, Petr Štetiar  wrote:
>
> Daniel Golle  [2019-01-07 10:03:09]:
>
> Hi,
>
> > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > a more sane flash partition layout to gain another megabyte or so.
>
> I've looked only at mt7621, so this was just example from one subtarget of
> ramips target. So I tend to believe, that there's quite more such cases hidden
> in the tree. Please correct me if I'm wrong.
>
> > Why specific devices? Wouldn't all devices with the resources (which
> > boils down to !SMALL_FLASH) be potentially more useful with those
> > kernel features enabled?
>
> You currently can't use !SMALL_FLASH, because this is target/subtarget
> specific feature, not per device feature. I think, that in order to use this
> feature, you would need to convert/fix all devices like that TP-Link RE350
> from all (sub)targets into tiny subtarget and then you could freely use
> !SMALL_FLASH.

I agree with not abusing small_flash for that. It has a clear defined
meaning, and shouldn't have unrelated side effects.

I think a new opt-in symbol for those targets with hardware
virtualization support and/or beefy enough cpus would make more sense.
Those virtualization options (probably) don't come for free, they will
have also a memory and performance impact even when not actively used.
How much that is (and if this assumption is true) would be nice to
have in the PR/patch for it.


Regards
Jonas

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Daniel Golle 
On Mon, Jan 07, 2019 at 01:17:34PM +, Jonas Gorski wrote:
> On Mon, 7 Jan 2019 at 11:42, Petr Štetiar  wrote:
> >
> > Daniel Golle  [2019-01-07 10:03:09]:
> >
> > Hi,
> >
> > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > a more sane flash partition layout to gain another megabyte or so.
> >
> > I've looked only at mt7621, so this was just example from one subtarget of
> > ramips target. So I tend to believe, that there's quite more such cases 
> > hidden
> > in the tree. Please correct me if I'm wrong.
> >
> > > Why specific devices? Wouldn't all devices with the resources (which
> > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > kernel features enabled?
> >
> > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > specific feature, not per device feature. I think, that in order to use this
> > feature, you would need to convert/fix all devices like that TP-Link RE350
> > from all (sub)targets into tiny subtarget and then you could freely use
> > !SMALL_FLASH.
> 
> I agree with not abusing small_flash for that. It has a clear defined
> meaning, and shouldn't have unrelated side effects.

So what else would the SMALL_FLASH symbol be used for then?
A quick grep reveals that currently already quite a few kernel config
defaults are set according to SMALL_FLASH, see

origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_SWAP
origin/master:Config-kernel.in- bool "Support for paging of anonymous memory 
(swap)"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_KALLSYMS
origin/master:Config-kernel.in- bool "Compile the kernel with symbol table 
information"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-
origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
origin/master:Config-kernel.in- bool "Compile the kernel with debug information"
origin/master:Config-kernel.in: default y if !SMALL_FLASH
--
origin/master:Config-kernel.in-config KERNEL_ELF_CORE
origin/master:Config-kernel.in- bool "Enable process core dump support"
origin/master:Config-kernel.in- select KERNEL_COREDUMP
origin/master:Config-kernel.in: default y if !SMALL_FLASH
...

> 
> I think a new opt-in symbol for those targets with hardware
> virtualization support and/or beefy enough cpus would make more sense.
> Those virtualization options (probably) don't come for free, they will
> have also a memory and performance impact even when not actively used.
> How much that is (and if this assumption is true) would be nice to
> have in the PR/patch for it.

This is not about virtualization and none of the features selected
requires any special hardware support apart from the few extra
kilobytes of flash and memory. You are still right, it doesn't come
all for free at runtime in terms of CPU cycles, but the impact is
hardly measurable.

But sure, I understand that this can be opt-in, so lets call it
'full_kernel' or something like that and have target maintainers
decide themselves. In the picture I get after browsing through
all targets, it would still end up such that
full_kernel == !small_flash is true for all cases.


And sure, I can carry out some size measurements and compare memory
allocation after boot. I'm sure the run-time CPU impact is hardly
measurable at all.

> 
> 
> Regards
> Jonas

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Jonas Gorski 
On Mon, 7 Jan 2019 at 14:21, Daniel Golle  wrote:
>
> On Mon, Jan 07, 2019 at 01:17:34PM +, Jonas Gorski wrote:
> > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar  wrote:
> > >
> > > Daniel Golle  [2019-01-07 10:03:09]:
> > >
> > > Hi,
> > >
> > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > a more sane flash partition layout to gain another megabyte or so.
> > >
> > > I've looked only at mt7621, so this was just example from one subtarget of
> > > ramips target. So I tend to believe, that there's quite more such cases 
> > > hidden
> > > in the tree. Please correct me if I'm wrong.
> > >
> > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > kernel features enabled?
> > >
> > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > specific feature, not per device feature. I think, that in order to use 
> > > this
> > > feature, you would need to convert/fix all devices like that TP-Link RE350
> > > from all (sub)targets into tiny subtarget and then you could freely use
> > > !SMALL_FLASH.
> >
> > I agree with not abusing small_flash for that. It has a clear defined
> > meaning, and shouldn't have unrelated side effects.
>
> So what else would the SMALL_FLASH symbol be used for then?
> A quick grep reveals that currently already quite a few kernel config
> defaults are set according to SMALL_FLASH, see
>
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_SWAP
> origin/master:Config-kernel.in- bool "Support for paging of anonymous memory 
> (swap)"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> origin/master:Config-kernel.in- bool "Compile the kernel with symbol table 
> information"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-
> origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> origin/master:Config-kernel.in- bool "Compile the kernel with debug 
> information"
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> --
> origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> origin/master:Config-kernel.in- bool "Enable process core dump support"
> origin/master:Config-kernel.in- select KERNEL_COREDUMP
> origin/master:Config-kernel.in: default y if !SMALL_FLASH
> ...

Most of these option only influence the size of the kernel, and have
no further runtime side effects. Also small_flash has impact on the
compression options used.

>
> >
> > I think a new opt-in symbol for those targets with hardware
> > virtualization support and/or beefy enough cpus would make more sense.
> > Those virtualization options (probably) don't come for free, they will
> > have also a memory and performance impact even when not actively used.
> > How much that is (and if this assumption is true) would be nice to
> > have in the PR/patch for it.
>
> This is not about virtualization and none of the features selected
> requires any special hardware support apart from the few extra
> kilobytes of flash and memory. You are still right, it doesn't come
> all for free at runtime in terms of CPU cycles, but the impact is
> hardly measurable.
>
> But sure, I understand that this can be opt-in, so lets call it
> 'full_kernel' or something like that and have target maintainers
> decide themselves. In the picture I get after browsing through
> all targets, it would still end up such that
> full_kernel == !small_flash is true for all cases.

"Full kernel" really has no real meaning and would describe
everything. The name should clearly describe the (non-default) feature
set it enables.

> And sure, I can carry out some size measurements and compare memory
> allocation after boot. I'm sure the run-time CPU impact is hardly
> measurable at all.

Please do so especially on low end devices. Maybe something like
routing throughput with/without.


Regards
Jonas

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

Re: [OpenWrt-Devel] [PATCH] config: enable some useful features on !SMALL_FLASH devices

2019-01-07 Thread Daniel Golle 
On Mon, Jan 07, 2019 at 02:39:26PM +, Jonas Gorski wrote:
> On Mon, 7 Jan 2019 at 14:21, Daniel Golle  wrote:
> >
> > On Mon, Jan 07, 2019 at 01:17:34PM +, Jonas Gorski wrote:
> > > On Mon, 7 Jan 2019 at 11:42, Petr Štetiar  wrote:
> > > >
> > > > Daniel Golle  [2019-01-07 10:03:09]:
> > > >
> > > > Hi,
> > > >
> > > > > One. The MT7621 EVB. The TP-LINK RE350 v1 can probably be converted to
> > > > > a more sane flash partition layout to gain another megabyte or so.
> > > >
> > > > I've looked only at mt7621, so this was just example from one subtarget 
> > > > of
> > > > ramips target. So I tend to believe, that there's quite more such cases 
> > > > hidden
> > > > in the tree. Please correct me if I'm wrong.
> > > >
> > > > > Why specific devices? Wouldn't all devices with the resources (which
> > > > > boils down to !SMALL_FLASH) be potentially more useful with those
> > > > > kernel features enabled?
> > > >
> > > > You currently can't use !SMALL_FLASH, because this is target/subtarget
> > > > specific feature, not per device feature. I think, that in order to use 
> > > > this
> > > > feature, you would need to convert/fix all devices like that TP-Link 
> > > > RE350
> > > > from all (sub)targets into tiny subtarget and then you could freely use
> > > > !SMALL_FLASH.
> > >
> > > I agree with not abusing small_flash for that. It has a clear defined
> > > meaning, and shouldn't have unrelated side effects.
> >
> > So what else would the SMALL_FLASH symbol be used for then?
> > A quick grep reveals that currently already quite a few kernel config
> > defaults are set according to SMALL_FLASH, see
> >
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_SWAP
> > origin/master:Config-kernel.in- bool "Support for paging of anonymous 
> > memory (swap)"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_KALLSYMS
> > origin/master:Config-kernel.in- bool "Compile the kernel with symbol table 
> > information"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-
> > origin/master:Config-kernel.in-config KERNEL_DEBUG_INFO
> > origin/master:Config-kernel.in- bool "Compile the kernel with debug 
> > information"
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > --
> > origin/master:Config-kernel.in-config KERNEL_ELF_CORE
> > origin/master:Config-kernel.in- bool "Enable process core dump support"
> > origin/master:Config-kernel.in- select KERNEL_COREDUMP
> > origin/master:Config-kernel.in: default y if !SMALL_FLASH
> > ...
> 
> Most of these option only influence the size of the kernel, and have
> no further runtime side effects. Also small_flash has impact on the
> compression options used.

They sure do, cache size on small CPUs is a very finite resource
and having a kernel with debug symbols will make things slower, of
course. SWAP also makes every single malloc call more expensive and
is just as well only useful on devices with block storage (ok, and
zramswap, but lets not talk about that).

> 
> >
> > >
> > > I think a new opt-in symbol for those targets with hardware
> > > virtualization support and/or beefy enough cpus would make more sense.
> > > Those virtualization options (probably) don't come for free, they will
> > > have also a memory and performance impact even when not actively used.
> > > How much that is (and if this assumption is true) would be nice to
> > > have in the PR/patch for it.
> >
> > This is not about virtualization and none of the features selected
> > requires any special hardware support apart from the few extra
> > kilobytes of flash and memory. You are still right, it doesn't come
> > all for free at runtime in terms of CPU cycles, but the impact is
> > hardly measurable.
> >
> > But sure, I understand that this can be opt-in, so lets call it
> > 'full_kernel' or something like that and have target maintainers
> > decide themselves. In the picture I get after browsing through
> > all targets, it would still end up such that
> > full_kernel == !small_flash is true for all cases.
> 
> "Full kernel" really has no real meaning and would describe
> everything. The name should clearly describe the (non-default) feature
> set it enables.

But they are not even necessarily related, just closer to the vanilla
default config which is used eg. by Debian and most other Linux distros
so projects like LVM2 started to rely on them.
My goal here is to bring modern generic kernel features into OpenWrt,
they are quite unrelated apart from being left out for space reasons
and because for a minimal router/AP they are unneeded.

> 
> > And sure, I can carry out some size measurements and compare memory
> > allocation after boot. I'm sure the run-time CPU impact is hardly
> > measurable at all.
> 
> Please do so especially on low end devices. Maybe something like
> routing throughput with/withou

Re: [OpenWrt-Devel] [PATCH] mpc85xx: add support for Sophos RED 15w Rev.1

2019-01-07 Thread Christian Lamparter 
On Sunday, January 6, 2019 5:29:54 PM CET David Bauer wrote:
> Hardware
> 
> CPU:  Freescale P1010 PowerPC
> RAM:  128M DDR3
> NAND: 128MiB
> ETH:  RTL8211F SGMII PHY
>   RTL8367B 5-port RGMII switch
>   (not connected to SoC - unmanaged)
> WiFi: SparkLan WPEA-121N
>- Atheros AR9382 2T2R abgn
> USB:  1x USB 2.0
> LED:  System, Router, Internet, Tunnel controllable
>   LAN1-4, WAN, Power non-controllable
> BTN:  None
> 
> Installation
> 
> 1. Power on the device while attached to the Console port.
> 
> 2. Halt the U-Boot by pressing Enter when prompted.
> 
> 3. Set the correct bootcmd for booting OpenWRT:
>  > setenv bootargs_owrt "setenv bootargs console=ttyS0,115200"
>  > setenv bootcmd "run bootargs_owrt;
>nand read 0x100 0x30 0x80;
>bootm 0x100;"
>  > saveenv
> 
> 5. Rename OpenWRT initramfs image to 'kernel.bin' and place it in a
>TFTP server root-directory served on 192.168.1.2/24. Connect your
>computer to one of the LAN-ports.
> 
> 4. Boot OpenWRT initramfs image with
>  > run bootargs_owrt; tftpboot 0x100 192.168.1.2:kernel.bin;
>bootm 0x100;
> 
> 6. (Optional)
>Make a Backup of 'sophos-os1', 'sophos-os2' and 'sophos-data' in case
>you ever want to go back to the vendor firmware.
> 
> 7. Create Ubi Volume on mtd4 by executing
>  > ubiformat /dev/mtd4 -y
> 
> 8. Transfer OpenWRT sysupgrade image to the device via SCP and install it
>with
>  > sysupgrade -n 
> 
> Back to Stock
> =
> If you want to go back to the stock firmware, here is the bootcmd of the
> vendor firmware:
>  > setenv bootargs console=ttyS0,115200 root=/dev/mtdblock5;
>nand read 0xc0 0x0030  0x10;
>nand read 0x100 0x0040 0x0080;
>bootm 0x100 - 0xc0
> 
> Set it via 'setenv' from the U-Boot shell and don't forget to save it
> using 'saveenv'!
> 
> After this, boot the OpenWRT initramfs image just like you would for
> installation. Write back the three vendor partitions using mtd. Reboot
> the device afterwards.
> 
> Signed-off-by: David Bauer 
> ---
> diff --git a/target/linux/mpc85xx/config-4.14 
> b/target/linux/mpc85xx/config-4.14
> index 5bed475664..e81ff1cda0 100644
> --- a/target/linux/mpc85xx/config-4.14
> +++ b/target/linux/mpc85xx/config-4.14
> @@ -88,6 +88,7 @@ CONFIG_ETHERNET_PACKET_MANGLE=y
>  CONFIG_FIXED_PHY=y
>  # CONFIG_FORCE_SMP is not set
>  CONFIG_FSL_BOOKE=y
> +# CONFIG_FSL_CORENET_CF is not set
>  # CONFIG_FSL_DPAA is not set
>  CONFIG_FSL_EMB_PERFMON=y
>  # CONFIG_FSL_FMAN is not set
> @@ -213,7 +214,16 @@ CONFIG_MPIC_TIMER=y
>  CONFIG_MPILIB=y
>  # CONFIG_MTD_CFI is not set
>  CONFIG_MTD_M25P80=y
> +CONFIG_MTD_NAND=y
> +CONFIG_MTD_NAND_ECC=y
> +CONFIG_MTD_NAND_FSL_IFC=y
>  CONFIG_MTD_SPI_NOR=y
> +CONFIG_MTD_UBI=y
> +CONFIG_MTD_UBI_BEB_LIMIT=20
> +CONFIG_MTD_UBI_BLOCK=y
> +# CONFIG_MTD_UBI_FASTMAP is not set
> +# CONFIG_MTD_UBI_GLUEBI is not set
> +CONFIG_MTD_UBI_WL_THRESHOLD=4096
>  # CONFIG_MVME2500 is not set
>  # CONFIG_NEED_DMA_MAP_STATE is not set
>  # CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK is not set
> @@ -306,6 +316,8 @@ CONFIG_QUICC_ENGINE=y
>  CONFIG_RAS=y
>  # CONFIG_RCU_NEED_SEGCBLIST is not set
>  # CONFIG_RCU_STALL_COMMON is not set
> +CONFIG_REALTEK_PHY=y
> +CONFIG_RED_15W_REV1=y
>  CONFIG_REGMAP=y
>  CONFIG_REGMAP_I2C=y
>  CONFIG_REGMAP_SPI=y
> @@ -348,6 +360,10 @@ CONFIG_TINY_SRCU=y
>  # CONFIG_TQM8555 is not set
>  # CONFIG_TQM8560 is not set
>  # CONFIG_TWR_P102x is not set
> +CONFIG_UBIFS_FS=y
> +CONFIG_UBIFS_FS_ADVANCED_COMPR=y
> +CONFIG_UBIFS_FS_LZO=y
> +CONFIG_UBIFS_FS_ZLIB=y
>  CONFIG_UCC=y
>  CONFIG_UCC_FAST=y
>  CONFIG_UCC_GETH=y
Since you add this new device as a subtarget of generic, why not just add the
REALTEK_PHY, NAND, UBI + UBIFS support into 
target/linux/mpc85xx/generic/config-default ?
This way you won't need to patch target/linux/mpc85xx/config-4.14 and 
target/linux/mpc85xx/config-4.19 as much and the HiveAP 330 does not end up
with stuff it doesn't need.

(The nand feature can also be placed into generic/target.mk)?

(However, make sure to add the 
# CONFIG_RED_15W_REV1 is not set
to the config-4.14 and config-4.19)

> diff --git a/target/linux/mpc85xx/config-4.19 
> b/target/linux/mpc85xx/config-4.19
> index 0aaefe84ea..63cd5c6c01 100644
> --- a/target/linux/mpc85xx/config-4.19
> +++ b/target/linux/mpc85xx/config-4.19
> @@ -86,6 +86,7 @@ CONFIG_ETHERNET_PACKET_MANGLE=y
>  # CONFIG_FHCI_DEBUG is not set
>  CONFIG_FIXED_PHY=y
>  CONFIG_FSL_BOOKE=y
> +# CONFIG_FSL_CORENET_CF is not set
>  # CONFIG_FSL_DPAA is not set
>  CONFIG_FSL_EMB_PERFMON=y
>  # CONFIG_FSL_FMAN is not set
> @@ -199,7 +200,16 @@ CONFIG_MPIC_TIMER=y
>  CONFIG_MPILIB=y
>  # CONFIG_MTD_CFI is not set
>  CONFIG_MTD_M25P80=y
> +CONFIG_MTD_NAND=y
> +CONFIG_MTD_NAND_ECC=y
> +CONFIG_MTD_NAND_FSL_IFC=y
>  CONFIG_MTD_SPI_NOR=y
> +CONFIG_MTD_UBI=y
> +CONFIG_MTD_UBI_BEB_LIMIT=20
> +CONFIG_MTD_UBI_BLOCK=y
> +# CONFIG_MTD_UBI_FASTMAP is not set
> +# CONFIG_MTD_UBI_GLUEBI is

Re: [OpenWrt-Devel] flock clash with msmtpq-ng-mta

2019-01-07 Thread Marcel Telka 
On Sat, Jan 05, 2019 at 03:54:52AM -0500, Daniel F. Dickinson wrote:
> On 2019-01-04 2:14 a.m., Marcel Telka wrote:
> > Hi,
> >
> > After upgrade from LEDE 17.01.4 to OpenWrt 18.06.1 I noticed that
> > mailq from
> > the msmtpq-ng-mta package no longer works:
> >
> So I have bad news -- I'm upstream and I deprecated / abandoned the
> msmtpq-ng as a bad idea.  I could probably be convinced to bring it back
> if there were use cases that actually made sense (what I realized is
> that the queuing was of limited use on flash system because a reboot
> loses all queued messages anyway in the default (RAM) config, and I
> wouldn't want to queue onto flash for limited writes concerns).  If
> there was a good case I think I'd fixup the shell script only until I
> had replaced it with a decent C implementation of a similar concept
> (preferably using a reasonably small smtp library to do avoid
> reinventing the wheel).

Okay, thanks for the info.

My use case is simple: I have a site (small household) with only one device
running all the time (a cheap openwrt router).  I would like to run some
non-critical cron jobs on the router and I'd like to get email notifications
from them once completed.  I found that simple mail queue support as provided
by msmtpq-ng-mta is good enough to cover rare internet connection issues.  I do
not need to have a bullet proof system with zero lost mails, so in-RAM queue is
good enough.  When I did my research few years back to find the best solution I
just found msmtpq-ng-mta and started to use it.  It perfectly fits my needs.  A
possible solution with added some small additional device (like a Raspberry Pi)
is just an overkill for this use case and cannot be reasonably justified.

Is there any other simple solution that could be used for sending mails,
preferably with simple queuing support, and with ability to submit mails using
TLS and SMTP AUTH?


Thanks.

-- 
+---+
| Marcel Telka   e-mail:   mar...@telka.sk  |
|homepage: http://telka.sk/ |
|jabber:   mar...@jabber.sk |
+---+

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel

[OpenWrt-Devel] [PATCH v2] mpc85xx: add support for Sophos RED 15w Rev.1

2019-01-07 Thread David Bauer 
Hardware

CPU:  Freescale P1010 PowerPC
RAM:  128M DDR3
NAND: 128MiB
ETH:  RTL8211F SGMII PHY
  RTL8367B 5-port RGMII switch
  (not connected to SoC - unmanaged)
WiFi: SparkLan WPEA-121N
   - Atheros AR9382 2T2R abgn
USB:  1x USB 2.0
LED:  System, Router, Internet, Tunnel controllable
  LAN1-4, WAN, Power non-controllable
BTN:  None

Installation

1. Power on the device while attached to the Console port.

2. Halt the U-Boot by pressing Enter when prompted.

3. Set the correct bootcmd for booting OpenWRT:
 > setenv bootargs_owrt "setenv bootargs console=ttyS0,115200"
 > setenv bootcmd "run bootargs_owrt;
   nand read 0x100 0x30 0x80;
   bootm 0x100;"
 > saveenv

5. Rename OpenWRT initramfs image to 'kernel.bin' and place it in a
   TFTP server root-directory served on 192.168.1.2/24. Connect your
   computer to one of the LAN-ports.

4. Boot OpenWRT initramfs image with
 > run bootargs_owrt; tftpboot 0x100 192.168.1.2:kernel.bin;
   bootm 0x100;

6. (Optional)
   Make a Backup of 'sophos-os1', 'sophos-os2' and 'sophos-data' in case
   you ever want to go back to the vendor firmware.

7. Create Ubi Volume on mtd4 by executing
 > ubiformat /dev/mtd4 -y

8. Transfer OpenWRT sysupgrade image to the device via SCP and install it
   with
 > sysupgrade -n 

Back to Stock
=
If you want to go back to the stock firmware, here is the bootcmd of the
vendor firmware:
 > setenv bootargs console=ttyS0,115200 root=/dev/mtdblock5;
   nand read 0xc0 0x0030  0x10;
   nand read 0x100 0x0040 0x0080;
   bootm 0x100 - 0xc0

Set it via 'setenv' from the U-Boot shell and don't forget to save it
using 'saveenv'!

After this, boot the OpenWRT initramfs image just like you would for
installation. Write back the three vendor partitions using mtd. Reboot
the device afterwards.

Signed-off-by: David Bauer 
---
v2:
 - move kernel configuration to subtarget
 - move nand feature to subtarget
 - move partition layout to "fixed-partition" subnode
 - add SPDX license-identifier to red15w_rev1.c
 - use pr_info instead of printk in red15w_rev1.c

 target/linux/mpc85xx/Makefile |   2 +-
 target/linux/mpc85xx/base-files/etc/diag.sh   |   3 +
 .../etc/hotplug.d/ieee80211/10-fix-wifi-mac   |  20 ++
 .../base-files/lib/upgrade/platform.sh|  13 +-
 target/linux/mpc85xx/config-4.14  |   1 +
 target/linux/mpc85xx/config-4.19  |   1 +
 .../arch/powerpc/boot/dts/red-15w-rev1.dts| 207 ++
 .../arch/powerpc/platforms/85xx/red15w_rev1.c |  87 
 target/linux/mpc85xx/generic/config-default   |  16 ++
 target/linux/mpc85xx/generic/target.mk|   1 +
 target/linux/mpc85xx/image/Makefile   |  12 +
 target/linux/mpc85xx/p1020/target.mk  |   1 -
 .../104-powerpc-85xx-red-15w-rev1.patch   |  33 +++
 .../103-powerpc-85xx-red-15w-rev1.patch   |  33 +++
 14 files changed, 427 insertions(+), 3 deletions(-)
 create mode 100644 
target/linux/mpc85xx/base-files/etc/hotplug.d/ieee80211/10-fix-wifi-mac
 create mode 100644 
target/linux/mpc85xx/files/arch/powerpc/boot/dts/red-15w-rev1.dts
 create mode 100644 
target/linux/mpc85xx/files/arch/powerpc/platforms/85xx/red15w_rev1.c
 create mode 100644 
target/linux/mpc85xx/patches-4.14/104-powerpc-85xx-red-15w-rev1.patch
 create mode 100644 
target/linux/mpc85xx/patches-4.19/103-powerpc-85xx-red-15w-rev1.patch

diff --git a/target/linux/mpc85xx/Makefile b/target/linux/mpc85xx/Makefile
index 552f16e82f..0affc38f61 100644
--- a/target/linux/mpc85xx/Makefile
+++ b/target/linux/mpc85xx/Makefile
@@ -10,7 +10,7 @@ ARCH:=powerpc
 BOARD:=mpc85xx
 BOARDNAME:=Freescale MPC85xx
 CPU_TYPE:=8540
-FEATURES:=squashfs
+FEATURES:=squashfs ramdisk
 MAINTAINER:=Imre Kaloz 
 SUBTARGETS:=generic p1020
 
diff --git a/target/linux/mpc85xx/base-files/etc/diag.sh 
b/target/linux/mpc85xx/base-files/etc/diag.sh
index 3d56d14b08..e344d4b756 100644
--- a/target/linux/mpc85xx/base-files/etc/diag.sh
+++ b/target/linux/mpc85xx/base-files/etc/diag.sh
@@ -9,6 +9,9 @@ get_status_led() {
aerohive,hiveap-330)
status_led="hiveap-330:green:tricolor0"
;;
+   sophos,red-15w-rev1)
+   status_led="red-15w-rev1:green:system"
+   ;;
tplink,tl-wdr4900-v1)
status_led="tp-link:blue:system"
;;
diff --git 
a/target/linux/mpc85xx/base-files/etc/hotplug.d/ieee80211/10-fix-wifi-mac 
b/target/linux/mpc85xx/base-files/etc/hotplug.d/ieee80211/10-fix-wifi-mac
new file mode 100644
index 00..3092cebfc9
--- /dev/null
+++ b/target/linux/mpc85xx/base-files/etc/hotplug.d/ieee80211/10-fix-wifi-mac
@@ -0,0 +1,20 @@
+#!/bin/ash
+
+[ "$ACTION" == "add" ] || exit 0
+
+PHYNBR=${DEVPATH##*/phy}
+
+[ -n $PHYNBR ] || exit 0
+
+. /lib/functions.sh
+. /lib/functions/system.sh
+
+board=$(board_name)
+
+case "$board" in
+sophos,red-15w-rev1)
+   echo $(mtd_get_mac_ascii u-boot-env

[OpenWrt-Devel] [PATCH experimental] quilt: Build a kernel git tree

2019-01-07 Thread Daniel Santos 
This is is mostly for those who've wanted something like this.  It still
needs cleanup.

This patch enables OpenWRT to re-construct a kernel git tree instead of
just extracting and patching the kernel sources to an essentially
temporary directory.  It's VERY helpful when you're trying to backport,
git-blame, or just understand why a change was made.  Details are in the
patch message and Kconfig documentation.

If devs are interested in this patch for upstream I can clean it up. 
As-is, it spits you to a shell if anything goes wrong, so only works
with -j1.

Daniel

>From 5c73c936e84b0eeb9f595b39488deceb5cb509ab Mon Sep 17 00:00:00 2001
From: Daniel Santos 
Date: Tue, 30 Oct 2018 20:54:00 -0500
Subject: [PATCH] Add option to build external kernel tree (experimental)

This patch modifies quilt and adds the root .config options that can be
configured to cause the OpenWRT / quilt build to re-construct a real git
tree using a base git reference and then applying the kernel patches
using git am or git apply.

You only need to enable this and run the build one time, after which you
should generally disable the option and configure OpenWRT to *use* the
external kernel directory, which inhibits OpenWRT from reconstructing
(extracting and patching) your kernel tree.

This patch still needs cleanup (removal of commented out code, etc.)
land probably also refinement.  Also, it is not perfect at
reconstructing the logical patch history due to the "files" directories
that must be applied as a set of mega-patches (one for generic and one
for the arch files), but it's step in the right direction.

Signed-off-by: Daniel Santos 
---
 config/Config-devel.in  | 28 +
 include/quilt.mk|  6 ++
 scripts/patch-kernel.sh | 45 -
 3 files changed, 74 insertions(+), 5 deletions(-)

diff --git a/config/Config-devel.in b/config/Config-devel.in
index fd7c3ead1e..06110a14da 100644
--- a/config/Config-devel.in
+++ b/config/Config-devel.in
@@ -73,6 +73,34 @@ menuconfig DEVEL
 		string "Use external kernel tree" if DEVEL
 		default ""
 
+	config EXTERNAL_KERNEL_TREE_BUILD_GIT
+		bool "Build git tree from OpenWRT patches." if DEVEL
+		depends on !KERNEL_GIT_CLONE_URI
+		default n
+		help
+		  Uses git to clone the upstream kernel and then apply OpenWRT
+		  patches with git-am or git-apply to create a proper Linux
+		  kernel git tree.
+
+  TODO: needs more explanation.
+
+	config EXTERNAL_KERNEL_TREE_BUILD_GIT_URI
+		bool "Upstream git URI" if DEVEL
+		depends on EXTERNAL_KERNEL_TREE_BUILD_GIT
+		default "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git";
+		help
+		  Where to clone the Linux tree from.  You probably want the
+		  default unless you already have a local copy you want to
+		  clone.
+
+	config EXTERNAL_KERNEL_TREE_BUILD_GIT_REF
+		bool "Local referenceBuild git tree from OpenWRT patches." if DEVEL
+		depends on EXTERNAL_KERNEL_TREE_BUILD_GIT
+		default ""
+		help
+		  Use git clone --dissociate --reference-if-able 
+		  when cloning to reduce network traffic.
+
 	config KERNEL_GIT_CLONE_URI
 		string "Enter git repository to clone" if DEVEL
 		default ""
diff --git a/include/quilt.mk b/include/quilt.mk
index 61dcc7964c..1f52d53176 100644
--- a/include/quilt.mk
+++ b/include/quilt.mk
@@ -95,7 +95,13 @@ endef
 kernel_files=$(foreach fdir,$(GENERIC_FILES_DIR) $(FILES_DIR),$(fdir)/.)
 define Kernel/Patch/Default
 	$(if $(QUILT),rm -rf $(PKG_BUILD_DIR)/patches; mkdir -p $(PKG_BUILD_DIR)/patches)
+
+	#echo "copy files is next"
+	#read
 	$(if $(kernel_files),$(CP) $(kernel_files) $(LINUX_DIR)/)
+
+	#echo "remove (old?) rejects and patch..."
+	#read
 	find $(LINUX_DIR)/ -name \*.rej -or -name \*.orig | $(XARGS) rm -f
 	if [ -d $(GENERIC_PLATFORM_DIR)/patches$(if $(wildcard $(GENERIC_PLATFORM_DIR)/patches-$(KERNEL_PATCHVER)),-$(KERNEL_PATCHVER)) ]; then \
 		echo "generic patches directory is present. please move your patches to the pending directory" ; \
diff --git a/scripts/patch-kernel.sh b/scripts/patch-kernel.sh
index c2b7e72049..b52d16f61c 100755
--- a/scripts/patch-kernel.sh
+++ b/scripts/patch-kernel.sh
@@ -18,7 +18,20 @@ if [ ! -d "${patchdir}" ] ; then
 echo "Aborting.  '${patchdir}' is not a directory."
 exit 1
 fi
-
+
+fixit() {
+op="$1"
+file="$2"
+patch="$3"
+shift 3
+echo "$op failed on $file ($patch).  $@"
+/bin/bash
+while [ -n "`git status --porcelain`" ]; do
+	echo "git repo not clean, please try again"
+	/bin/bash
+done
+}
+
 for i in ${patchdir}/${patchpattern} ; do 
 case "$i" in
 	*.gz)
@@ -37,12 +50,34 @@ for i in ${patchdir}/${patchpattern} ; do
 [ -d "${i}" ] && echo "Ignoring subdirectory ${i}" && continue	
 echo ""
 echo "Applying ${i} using ${type}: " 
-${uncomp} ${i} | ${PATCH:-patch} -f -p1 -d ${targetdir}
-if [ $? != 0 ] ; then
-echo "Patch failed!  Please fix $i!"
-	exit 1
+if true; then
+	if ! ${uncomp} ${i} |